The black hole accretion code
 Oliver Porth^{1}Email authorView ORCID ID profile,
 Hector Olivares^{1},
 Yosuke Mizuno^{1},
 Ziri Younsi^{1},
 Luciano Rezzolla^{1, 2},
 Monika Moscibrodzka^{3},
 Heino Falcke^{3} and
 Michael Kramer^{4}
https://doi.org/10.1186/s4066801700202
© The Author(s) 2017
Received: 5 December 2016
Accepted: 6 April 2017
Published: 3 May 2017
Abstract
We present the black hole accretion code (BHAC), a new multidimensional generalrelativistic magnetohydrodynamics module for the MPIAMRVAC framework. BHAC has been designed to solve the equations of ideal generalrelativistic magnetohydrodynamics in arbitrary spacetimes and exploits adaptive mesh refinement techniques with an efficient blockbased approach. Several spacetimes have already been implemented and tested. We demonstrate the validity of BHAC by means of various one, two, and threedimensional test problems, as well as through a close comparison with the HARM3D code in the case of a torus accreting onto a black hole. The convergence of a turbulent accretion scenario is investigated with several diagnostics and we find accretion rates and horizonpenetrating fluxes to be convergent to within a few percent when the problem is run in three dimensions. Our analysis also involves the study of the corresponding thermal synchrotron emission, which is performed by means of a new generalrelativistic radiative transfer code, BHOSS. The resulting synthetic intensity maps of accretion onto black holes are found to be convergent with increasing resolution and are anticipated to play a crucial role in the interpretation of horizonscale images resulting from upcoming radio observations of the source at the Galactic Center.
1 Introduction
Accreting black holes (BHs) are amongst the most powerful astrophysical objects in the Universe. A substantial fraction of the gravitational binding energy of the accreting gas is released within tens of gravitational radii from the BH, and this energy supplies the power for a rich phenomenology of astrophysical systems including active galactic nuclei, Xray binaries and gammaray bursts. Since the radiated energy originates from the vicinity of the BH, a fully generalrelativistic treatment is essential for the modelling of these objects and the flows of plasma in their vicinity.
Depending on the mass accretion rate, a given system can be found in various spectral states, with different radiation mechanisms dominating and varying degrees of coupling between radiation and gas (Fender et al. 2004; Markoff 2005). Some supermassive BHs, including the primary targets of observations by the EventHorizonTelescope Collaboration (EHTC^{1}), i.e., Sgr A* and M87, are accreting well below the Eddington accretion rate (Marrone et al. 2007; Ho 2009). In this regime, the accretion flow advects most of the viscously released energy into the BH rather than radiating it to infinity. Such optically thin, radiatively inefficient and geometrically thick flows are termed advectiondominated accretion flows (ADAFs, see (Narayan and Yi 1994; Narayan and Yi 1995; Abramowicz et al. 1995; Yuan and Narayan 2014)) and can be modelled without radiation feedback. Next to the ADAF, two additional radiatively inefficient accretion flows (RIAFs) exist: The advectiondominated inflowoutflow solution (ADIOS) (Blandford and Begelman 1999; Begelman 2012) and the convectiondominated accretion flow (CDAF) (Narayan et al. 2000; Quataert and Gruzinov 2000), which include respectively, the physical effects of outflows and convection. Analytical and semianalytical approaches are reasonably successful in reproducing the main features in the spectra of ADAFs [see, e.g., Yuan et al. (2003)]. However, numerical generalrelativistic magnetohydrodynamic (GRMHD) simulations are essential to gain an understanding of the detailed physical processes at play in the Galactic Centre and other lowluminosity compact objects.
Modern BH accretiondisk theory suggests that angular momentum transport is due to MHD turbulence driven by the magnetorotational instability (MRI) within a differentially rotating disk (Balbus and Hawley 1991; Balbus and Hawley 1998). Recent nonradiative GRMHD simulations of BH accretion systems in an ADAF regime have resolved these processes and reveal a flow structure that can be decomposed into a disk, a corona, a diskwind and a highly magnetized polar funnel [see, e.g., Villiers and Hawley (2003); McKinney and Gammie (2004); McKinney (2006); McKinney and Blandford (2009)]. The simulations show complex timedependent behaviour in the disk, corona and wind. Depending on BH spin, the polar regions of the flow contain a nearly forcefree, Poyntingfluxdominated jet [see, e.g., Blandford and Znajek (1977); McKinney and Gammie (2004); Hawley and Krolik (2006); McKinney (2006)].
In addition to having to deal with highly nonlinear dynamics that spans a large range in plasma parameters, the numerical simulations also need to follow phenomena that occur across multiple physical scales. For example, in the MHD paradigm, jet acceleration is an intrinsically inefficient process that requires a few thousand gravitational radii to reach equipartition of the energy fluxes (Komissarov et al. 2007; Barkov and Komissarov 2008) (purely hydrodynamical mechanisms can however be far more efficient (Aloy and Rezzolla 2006)). Jetenvironment interactions like the prominent HST1 feature of the radiogalaxy M87 (Biretta et al. 1989; Stawarz et al. 2006; Asada and Nakamura 2012) occur on scales of \(\sim5\times 10^{5}\) gravitational radii. Hence, for a selfconsistent picture of accretion and ejection, jet formation and recollimation due to interaction with the environment [see, e.g., Mizuno et al. (2015)], numerical simulations must capture horizonscale processes, as well as parsecscale interactions with an overall spatial dynamic range of ∼10^{5}. The computational cost of such largescale gridbased simulations quickly becomes prohibitive. Adaptive mesh refinement (AMR) techniques promise an effective solution for problems where it is necessary to resolve small and large scale dynamics simultaneously.
Another challenging scenario is presented by radiatively efficient geometrically thin accretion disks that mandate extreme resolution in the equatorial plane in order to resolve the growth of MRI instabilities. Typically this is dealt with by means of stretched grids that concentrate resolution where needed (Avara et al. 2016; Sądowski 2016). However, when the disk is additionally tilted with respect to the spin axis of the BH (Fragile et al. 2007; McKinney et al. 2013), lack of symmetry forbids such an approach. Here an adaptive mesh that follows the warping dynamics of the disk can be of great value. The list of scenarios where AMR can have transformative qualities due to the lack of symmetries goes on, the modelling of stardisk interactions (Giannios and Sironi 2013), starjet interactions (Barkov et al. 2010), tidal disruption events (Tchekhovskoy et al. 2014), complex shock geometries (Nagakura and Yamada 2008; Meliani et al. 2017), and intermittency in driventurbulence phenomena (Radice and Rezzolla 2013; Zrake and MacFadyen 2013), will benefit greatly from adaptive mesh refinement.
Over the past few years, the development of generalrelativistic numerical codes employing the \(3+1\) decomposition of spacetime and conservative ‘Godunov’ schemes based on approximate Riemann solvers (Rezzolla and Zanotti 2013; Font 2003; Martí and Müller 2015) have led to great advances in numerical relativity. Many generalrelativistic hydrodynamic (HD) and MHD codes have been developed (Hawley et al. 1984; Koide et al. 2000; De Villiers and Hawley 2003; Gammie et al. 2003; Baiotti et al. 2005; Duez et al. 2005; Anninos et al. 2005; Antón et al. 2006; Mizuno et al. 2006; Del Zanna et al. 2007; Giacomazzo and Rezzolla 2007; Radice and Rezzolla 2012; Radice et al. 2014; McKinney et al. 2014; Etienne et al. 2015; White and Stone 2015; Zanotti and Dumbser 2015; Meliani et al. 2017) and applied to study a variety of problems in highenergy astrophysics. Some of these implementations provide additional capabilities that incorporate approximate radiation transfer [see, e.g., Sądowski et al. (2013); McKinney et al. (2013); Takahashi et al. (2016)] and/or nonideal MHD processes [see, e.g., Dionysopoulou et al. (2013); Foucart et al. (2016)]. Although these codes have been applied to many astrophysical scenarios involving compact objects and matter [for recent reviews see, e.g., Martí and Müller (2015); Baiotti and Rezzolla (2016)], full AMR is still not commonly utilised and exploited [with the exception of Anninos et al. (2005); Zanotti et al. (2015); White and Stone (2015)]. BHAC attempts to fill this gap by providing a fullyadaptive multidimensional GRMHD framework that features stateoftheart numerical schemes.
Qualitative aspects of BH accretion simulations are codeindependent [see, e.g., Villiers and Hawley (2003); Gammie et al. (2003); Anninos et al. (2005)], but quantitative variations raise questions regarding numerical convergence of the observables (Shiokawa et al. 2012; White and Stone 2015). In preparation for the upcoming EHTC observations, a large international effort, whose European contribution is represented in part by the BlackHoleCam project^{2} (Goddi et al. 2016), is concerned with forward modelling of the future event horizonscale interferometric observations of Sgr A* and M87 at submillimeter (EHTC; (Doeleman et al. 2009)) and nearinfrared wavelengths (VLTI GRAVITY; (Eisenhauer et al. 2008)). To this end, GRMHD simulations have been coupled to generalrelativistic radiative transfer (GRRT) calculations [see, e.g., Mościbrodzka et al. (2009); Dexter et al. (2009); Chan et al. (2015); Gold et al. (2016); Dexter et al. (2012); Mościbrodzka et al. (2016)]. In order to assess the credibility of these radiative models, it is necessary to assess the quantitative convergence of the underlying GRMHD simulations. In order to demonstrate the utility of BHAC for the EHTC sciencecase, we therefore validate the results obtained with BHAC against the HARM3D code (Gammie et al. 2003; Noble et al. 2009) and investigate the convergence of the GRMHD simulations and resulting observables obtained with the GRRT postprocessing code BHOSS (Younsi et al. 2017).
The structure of the paper is as follows. In Section 2 we describe the governing equations and numerical methods. In Section 3 we show numerical tests in specialrelativistic and generalrelativistic MHD. In Section 4 the results of 2D and 3D GRMHD simulations of magnetised accreting tori are presented. In Section 5 we briefly describe the GRRT postprocessing calculation and the resulting image maps from the magnetised torus simulation shown in Section 4. In Section 6 we present our conclusions and outlook.
Throughout this paper, we adopt units where the speed of light, \(c=1\), the gravitational constant, \(G=1\), and the gas mass is normalised to the central compact object mass. Greek indices run over space and time, i.e., \((0,1,2,3)\), and Roman indices run over space only, i.e., \((1,2,3)\). We assume a \((,+,+,+)\) signature for the spacetime metric. Selfgravity arising from the gas is neglected.
2 GRMHD formulation and numerical methods
In this section we briefly describe the covariant GRMHD equations, introduce the notation used throughout this paper, and present the numerical approach taken in our solution of the GRMHD system. The computational infrastructure underlying BHAC is the versatile opensource MPIAMRVAC toolkit (Keppens et al. 2012; Porth et al. 2014).
Indepth derivations of the covariant fluid and magnetofluid dynamical equations can be found in the textbooks by (Landau and Lifshitz 2004; Weinberg 1972; Rezzolla and Zanotti 2013). We follow closely the derivation of the GRMHD equations by (Del Zanna et al. 2007). This is very similar to the ‘Valencia formulation’, cf. (Rezzolla and Zanotti 2013) and (Antón et al. 2006). The general considerations of the ‘\(3+1\)’ split of spacetime are discussed in greater detail in (Misner et al. 1973; Gourgoulhon 2007; Alcubierre 2008).
2.1 \(3+1\) split of spacetime
In the following, purely spatial vectors (e.g., \(v^{0}=0\)) are denoted by Roman indices. Note that \(\Gamma=(1v^{2})^{1/2}\) with \(v^{2}=v_{i} v^{i}\) just as in special relativity.
2.2 Finite volume formulation
For the temporal update, we interpret the semidiscrete form (40) as an ordinary differential equation in time for each cell and employ a multistep RungeKutta scheme to evolve the average state in the cell \(\bar{\boldsymbol{U}}_{{i,j,k}}\), a procedure also known as ‘method of lines’. At each substep, the pointwise interface fluxes \(\boldsymbol{F^{i}}\) are obtained by performing a limited reconstruction operation of the cellaveraged state \(\bar{\boldsymbol{U}}\) to the interfaces (see Section 2.8) and employing approximate Riemann solvers, e.g., HLL or TVDLF (Section 2.9).
Several temporal update schemes are available: simple predictorcorrector, thirdorder RungeKutta (RK) RK3 (Gottlieb and Shu 1998) and the strongstability preserving sstep, pthorder RK schemes \(\operatorname{SSPRK}(s,p)\) schemes: \(\operatorname{SSPRK}(4,3)\), \(\operatorname{SSPRK}(5,4)\) due to (Spiteri and Ruuth 2002).^{5}
2.3 Metric datastructure
Elements of the metric datastructure
Symbol  Identifier  Index list 

\(g_{\mu\nu}\)  m%g(mu,nu)  m%nnonzero, m%nonzero(inonzero) 
α  m%alpha   
\(\beta^{i}\)  m%beta(i)  m%nnonzeroBeta, m%nonzeroBeta(inonzero) 
\(\sqrt{\gamma}\)  m%sqrtgamma   
\(\gamma^{ij}\)  m%gammainv(i,j)   
\(\beta_{i}\)  m%betaD(i)   
\(\partial_{k}\gamma_{ij}\)  m%dgdk(i,j,k)  m%nnonzeroDgDk, m%nonzeroDgDk(inonzero) 
\(\partial_{j}\beta^{i}\)  m%DbetaiDj(i,j)  m%nnonzeroDbetaiDj, m%nonzeroDbetaiDj(inonzero) 
\(\partial_{j}\alpha\)  m%DalphaDj(j)  m%nnonzeroDalphaDj, m%nonzeroDalphaDj(inonzero) 
0  m%zero   
As a consequence, only 14 grid functions are required for the Schwarzschild coordinates and 29 grid functions need to be allocated in the KerrSchild (KS) case. This is still less than half of the 68 grid functions which a bruteforce approach would yield. The need for efficient storage management becomes apparent when we consider that the metric is required in the barycenter as well as on the interfaces, thus multiplying the required grid functions by a factor of four for threedimensional simulations (yielding 116 grid functions in the KS case).
In order to eliminate the errorprone process of implementing complicated functions for metric derivatives, BHAC can obtain derivatives by means of an accurate complexstep numerical differentiation (Squire and Trapp 1998). This elegant method takes advantage of the CauchyRiemann differential equations for complex derivatives and achieves full doubleprecision accuracy, thereby avoiding the stepsize dilemma of common finitedifferencing formulae (Martins et al. 2003). The small price to pay is that at the initialisation stage, metric elements are provided via functions of the complexified coordinates. However, the intrinsic complex arithmetic of Fortran90 allows for seamless implementation.
To promote full flexibility in the spacetime, we always calculate the inverse metric \(\gamma^{ij}\) using the standard LU decomposition technique (Press et al. 2007). As a result, GRMHD simulations on any metric can be performed after providing only the nonzero elements of the threemetric \(\gamma_{ij}(x^{1},x^{2},x^{3})\), the lapse function \(\alpha(x^{1},x^{2},x^{3})\) and the shift vector \(\beta^{i}(x^{1},x^{2},x^{3})\). As an additional convenience, BHAC can calculate the required elements and their derivatives entirely from the fourmetric \(g_{\mu\nu}(x^{0},x^{1},x^{2},x^{3})\).
2.4 Equations of state

Ideal gas: \(h(\rho,p)= 1 + \dfrac{\hat{\gamma}}{\hat {\gamma}1} \dfrac{p}{\rho}\) with adiabatic index γ̂.

Synge gas: \(h(\Theta) = \dfrac{K_{3}(\Theta^{1})}{K_{2}(\Theta^{1})}\), where the relativistic temperature is given by \(\Theta=p/\rho\) and \(K_{n}\) denotes the modified Bessel function of the second kind. In fact, we use an approximation to the previous expression that does not contain Bessel functions [see Meliani et al. (2004); Keppens et al. (2012)].

Isentropic flow: Assumes an ideal gas with the additional constraint \(p=\kappa\rho^{\hat{\gamma}}\), where the pseudoentropy κ may be chosen arbitrarily. This allows one to omit the energy equation entirely and only the reduced set \(\boldsymbol{P}=\{\rho,v^{j},B^{j}\}\) is solved.
2.5 Divergence cleaning and augmented Faraday’s law
2.6 Fluxinterpolated constrained transport
Equation (47) is closely related to the integral over the surface of a volume containing eight cells in 3D (see Appendix D for the derivation), and it reduces to equation (27) from (Toth 2000) in the special case of Cartesian coordinates. As mentioned before, this scheme can maintain \(\boldsymbol{\nabla \cdot B}=0\) to machine precision only if it was already zero at the initial condition. The corresponding curl operator used to setup initial conditions is derived in Appendix D.
In its current form, BHAC cannot handle both constrained transport and AMR. The reason is that special prolongation and restriction operators are required in order to avoid the creation of divergence when refining or coarsening. Due to the lack of information about the magnetic flux on cell faces, the problem of finding such divergencepreserving prolongation operators becomes underdetermined. However, storing the faceallocated (staggered) magnetic fluxes and applying the appropriate prolongation and restriction operators requires a large change in the code infrastructure on which we will report in an accompanying work.
2.7 Coordinates
Coordinates available in BHAC
Coordinates  Identifier  Num. derivatives  Init. \(\boldsymbol{g_{\mu\nu}}\) 

Cartesian  cart  No  No 
BoyerLindquist  bl  No  No 
KerrSchild  ks  No  No 
Modified KerrSchild  mks  No  No 
Cartesian KerrSchild  cks  Yes  Yes 
Rezzolla & Zhidenko parametrization (Rezzolla and Zhidenko 2014)  rz  Yes  No 
Horizon penetrating Rezzolla & Zhidenko coordinates  rzks  Yes  Yes 
HartleThorne (Hartle and Thorne 1968)  ht  Yes  Yes 
2.7.1 Modified KerrSchild coordinates
Modified KS coordinates were introduced by e.g., (McKinney and Gammie 2004) with the purpose of stretching the grid radially and being able to concentrate resolution in the equatorial region.
With these transformations, we obtain the new metric \(g_{\mathrm{MKS}} = J^{\mathrm{T}} g_{\mathrm{KS}} J\). Note that whenever the parameters \(h=0\) and \(R_{0}=0\) are set, our MKS coordinates reduce to the standard logarithmic KerrSchild coordinates.
2.7.2 Rezzolla & Zhidenko parametrization
2.8 Available reconstruction schemes
The secondorder finite volume algorithm (40) requires numerical fluxes centered on the interface midpoint. As in any Godunovtype scheme [see e.g., Toro (1999), Komissarov (1999)], the fluxes are in fact computed by solving (approximate) Riemann problems at the interfaces (see Section 2.9). Hence for higher than firstorder accuracy, the fluid variables need to be reconstructed at the interface by means of an appropriate spatial interpolation. Our reconstruction strategy is as follows. (1) Compute primitive variables \(\bar{\boldsymbol{P}}\) from the averages of the conserved variables \(\bar{\boldsymbol{U}}\) located at the cell barycenter. (2) Use the reconstruction formulae to obtain two representations for the state at the interface, one with a leftbiased reconstruction stencil \(\boldsymbol{P}^{\mathrm{L}}\) and the other with a rightbiased stencil \(\boldsymbol{P}^{\mathrm{R}}\). (3) Convert the now pointwise values back to their conserved states \(\boldsymbol{U}^{\mathrm{L}}\) and \(\boldsymbol{U}^{\mathrm{R}}\). The latter two states then serve as input for the approximate Riemann solver.
A large variety of reconstruction schemes are available, which can be grouped into standard secondorder total variation diminishing (TVD) schemes like ‘minmod’, ‘vanLeer’, ‘monotonizedcentral’, ‘woodward’ and ‘koren’ [see Keppens et al. (2012), for details] and higher order methods like the thirdorder methods ‘PPM’ (Colella and Woodward 1984), ‘LIMO3’ (Čada and Torrilhon 2009) and the fifthorder monotonicity preserving scheme ‘MP5’ due to (Suresh and Huynh 1997). While the overall order of the scheme will remain secondorder, the higher accuracy of the spatial discretisation usually reduces the diffusion of the scheme and improves accuracy of the solution [see, e.g., Porth et al. (2014)]. For typical GRMHD simulations with nearevacuated funnel/atmosphere regions, we find the PPM reconstruction scheme to be a good compromise between high accuracy and robustness. For simple flows, e.g., the stationary toroidal field torus discussed in Section 3.4, the compact stencil LIMO3 method is recommended.
2.9 Characteristic speed and approximate Riemann solvers
2.10 Primitive variable recovery
It is wellknown that the nonlinear inversion \(\boldsymbol{P}(\boldsymbol{U})\) is the Achilles heel of any relativistic (M)HD code and sophisticated schemes with multiple backup strategies have been developed over the years as a consequence (e.g., Noble et al. (2006), Faber et al. (2007), Noble et al. (2009), Etienne et al. (2012), Galeazzi et al. (2013), Hamlin and Newman (2013)). Here we briefly describe the methods used throughout this work and refer to the previously mentioned references for a more detailed discussion.
2.10.1 Primary inversions
In addition to the 1D scheme, we have implemented the ‘2DW’ method of (Noble et al. 2006; Del Zanna et al. 2007). The 2DW inversion simultaneously solves the nonlinear Eqs. (25) and the square of the threemomentum \(S^{2}\), following (29) by means of a NewtonRaphson scheme on the two variables ξ and \(v^{2}\). Among all inversions tested by (Noble et al. 2006), the 2DW method was reported as the one with the smallest failure rate. We find the same trend, but also find that the lead of 2DW over 1D is rather minor in our tests.
With two distinct inversions that might fail under different circumstances, one can act as a backup strategy for the other. Typically we first attempt a 2DW inversion and switch to the 1D method when no convergence is found. The next layer of backup can be provided by the entropy method as described in the next section.
2.10.2 Entropy switch
In the rare instances where the entropy inversion also fails to converge to a physical solution, the code is normally stopped. To force a continuation of the simulation, last resort measures that depend on the physical scenario can be employed. Often the simulation can be continued when the faulty cell is replaced with averages of the primitive variables of the neighbouring healthy cells as described in (Keppens et al. 2012). In the GRMHD accretion simulations described below, failures could happen occasionally in the highly magnetised evacuated ‘funnel’ region close to the outer horizon where the floors are frequently applied. We found that the best strategy is then to replace the faulty density and pressure values with the floor values and set the Eulerian velocity to zero. Note that in order to avoid generating spurious \(\boldsymbol{\nabla\cdot B}\), the last resort measures should never modify the magnetic fields of the simulation.
2.11 Adaptive mesh refinement
The computational grid employed in BHAC is provided by the MPIAMRVAC toolkit and constitutes a fully adaptive block based (oct) tree with a fixed refinement factor of two between successive levels. That is, the domain is first split into a number of blocks with equal amount of cells (e.g., 10^{3} computational cells per block). Each block can be refined into two (1D), four (2D) or eight (3D) childblocks with an again fixed number of cells. This process of refinement can be repeated ad libitum and the datastructure can be thought of a forest (collection of trees). All operations on the grid, for example timeupdate, IO and problem initialisation are scheduled via a loop over a spacefilling curve. We adopt the Morton Zorder curve for ease of implementation via a simple recursive algorithm.
3 Numerical tests
3.1 Shock tube test with gauge effect
The first code test is considered in flat spacetime and therefore no metric source terms are involved. Herein we perform onedimensional MHD shock tube tests with gauge effects by considering gauge transformations of the spacetime. Shock tube tests are wellknown tests for code validation and emphasise the nonlinear behaviour of the equations, as well as the ability to resolve discontinuities in the solutions [see, e.g., Antón et al. (2006), Del Zanna et al. (2007)].
Shock tube with gaugeeffect setups
Case  α  \(\boldsymbol{\beta^{i}}\)  \(\boldsymbol{\gamma_{11}}\)  \(\boldsymbol{\gamma_{22}}\)  \(\boldsymbol{\gamma_{33}}\) 

A  1  (0,0,0)  1  1  1 
B  2  (0,0,0)  1  1  1 
C  1  (0.4,0,0)  1  1  1 
D  1  (0,0,0)  4  1  1 
E  1  (0,0,0)  1  4  1 
F  2  (0.4,0,0)  4  9  1 
In the simulations, an ideal gas EOS is employed with an adiabatic index of \(\hat{\gamma}=2\). The 1D problem is run on a uniform grid in xdirection using 1,024 cells spanning over \(x\in[1/2,1/2]\). The simulations are terminated at \(t=0.4\). For the spatial reconstruction, we adopt the second order TVD limiter due to Koren (Koren 1993). Furthermore, RK3 timeintegration is used with Courant number set to 0.4.
Case A is the reference solution without modification of fluxes due to the threemetric, lapse or shift.^{7} By means of simple transformations of flatspacetime, all other cases can be matched with the reference solution. Case B will coincide with solution A if B is viewed at \(t/2=0.2\). Case C will agree with case A when it is shifted in positive xdirection by \(\delta x = \beta^{x}t=0.16\). For case D, we rescale the domain as \(x\in[1/4,1/4]\) and initialise the contravariant vectors as \(B^{\prime x} = B^{x}/2\). The state at \(t=0.4\) should agree with case A when the domain is multiplied by the scale factor \(h_{x} = 2\). For case E we initialise \(B^{\prime y}=B^{y}/2\) and case F is initialised similarly as \(B^{\prime x} = B^{x}/2\), \(B^{\prime y}=B^{y}/3\).
3.2 Boosted loop advection
In order to test the implementation of the GLMGRMHD system, we perform the advection of a forcefree fluxtube with poloidal and toroidal components of the magnetic field in a flat spacetime.
This configuration is then boosted to the frame moving at velocity \(\boldsymbol{v}=\sqrt{2}(v_{c},v_{c},0)\) and we test values of \(v_{c}\) between 0.5c and 0.99c.
3.3 Magnetised spherical accretion
A useful stress test for the conservative algorithm in a generalrelativistic setting is spherical accretion onto a Schwarzschild BH with a strong radial magnetic field (Gammie et al. 2003). The steadystate solution is known as the Michel accretion solution (Michel 1972) and represents the extension to general relativity of the corresponding Newtonian solution by (Bondi 1952). The steadystate spherical accretion solution in general relativity is described in a number of works [see, e.g., Hawley et al. (1984), Rezzolla and Zanotti (2013)]. It is easy to show that the solution is not affected when a radial magnetic field of the form \(B^{r}\propto\gamma^{1/2}\) is added (De Villiers and Hawley 2003). This test challenges the robustness of the code and of the inversion procedure \(\boldsymbol{P}(\boldsymbol{U})\) in particular. The calculation of the initial condition follows that outlined in (Hawley et al. 1984). Here, we parametrize the field strength via \(\sigma=b^{2}/\rho\) at the inner edge of the domain (\(r=1.9\mbox{ M}\)). The simulation is setup in the equatorial plane using MKS coordinates corresponding to a domain of \(r_{\mathrm{KS}}\in[1.9,20]\mbox{ M}\). The analytic solution remains fixed at the inner and outer radial boundaries. We run two cases, case 1 with magnetisation up to \(\sigma=10^{3}\) and case 2 with a very high magnetisation reaching up to \(\sigma=10^{4}\). Since the problem is only 1D, the constraint \(\mathbf{\nabla\cdot B}=0\) has a unique solution which gets preserved via the FCT algorithm.
3.4 Magnetised equilibrium torus
As a final validation of the code in the GRMHD regime, we perform the simulation of a magnetised equilibrium torus around a spinning BH. A generalisation of the steadystate solution of the standard hydrodynamical equilibrium torus with constant angular momentum [see, e.g., Fishbone and Moncrief (1976), Hawley et al. (1984), Font and Daigne (2002)] to MHD equilibria with toroidal magnetic fields was proposed by (Komissarov 2006). This steadystate solution is important since it constitutes a rare case of a nontrivial analytic solution in GRMHD.^{9}
Parameters for the MHD equilibrium torus test
Case  \(\boldsymbol{l_{0}}\)  \(\boldsymbol{r_{c}}\)  \(\boldsymbol{W_{\mathrm{in}}}\)  \(\boldsymbol{W_{c}}\)  \(\boldsymbol{\omega_{c}}\)  \(\boldsymbol{\beta_{c}}\) 

A  2.8  4.62  −0.030  −0.103  1.0  0.1 
Initially, the velocity of the atmosphere outside of the torus is set to zero in the Eulerian frame, with density and gas pressure set to very small values of \(\rho=\rho_{\min} r_{\mathrm{BL}}^{3/2}\), \(p=p_{\min} r_{\mathrm{BL}}^{5/2}\) with \(\rho_{\min}=10^{5}\) and \(p_{\min}=10^{7}\). It is important to note that the atmosphere is free to evolve and only densities and pressures are floored according to the initial state. In the simulations we use the HLL approximate Riemann solver, third order LIMO3 reconstruction, twostep time update, and a CFL number of 0.5. We impose outflow conditions on the inner and outer boundaries of the radial direction and reflecting boundary conditions in the θ direction. As the magnetic field is purely toroidal, and will remain so during the timeevolution of this axisymmetric case, no particular \(\mathbf{\nabla\cdot B=0}\) treatment is used.
3.5 Differences between FCT and GLM
Having implemented two methods for divergence control, we took the opportunity to compare the results of simulations using both methods. We analysed three tests: a relativistic OrszagTang vortex, magnetised Michel accretion, and magnetised accretion from a FishboneMoncrief torus. Although much less indepth, this comparison is in the same spirit as those performed in previous works in nonrelativistic MHD (Toth 2000; Balsara and Kim 2004; Mocz et al. 2016). The wellknown work by (Toth 2000) compares seven divergencecontrol methods, including an early nonconservative divergencecleaning method known as the eightwave method (Powell 1994), and three CT methods, finding that FCT is among the three most accurate schemes for the test problems studied. In (Balsara and Kim 2004), three divergencecleaning schemes and one CT scheme were applied to the same test problem of supernovainduced MHD turbulence in the interstellar medium. It was found that the three divergencecleaning methods studied suffer from, among other problems, spurious oscillations in the magnetic energy, which is attributed to the nonlocality introduced by the loss of hyperbolicity in the equations. Finally, in (Mocz et al. 2016), a nonstaggered version of CT adapted to a moving mesh is compared to the divergencecleaning Powell scheme (Powell et al. 1999), an improved version of the eightwave method. They observe greater numerical stability and accuracy, and a better preservation of the magnetic field topology for the CT scheme. In their tests, the Powell scheme suffers from an artificial growth of the magnetic field. This is explained to be a result of the scheme being nonconservative.
3.5.1 OrszagTang vortex
3.5.2 Spherical accretion
3.5.3 Accreting torus
To summarise this small section on the comparison between both divergencecontrol techniques, we found from the three tests performed that FCT seems to be less diffusive than GLM, is able to preserve for a longer time a stationary solution, and seems to create less spurious structures in the magnetic field. However, it still has the inconvenient property that it is not possible to implement a cellentered version of it whilst fully incorporating AMR. As mentioned previously, we are currently working on a staggered implementation adapted to AMR, and this will be described in a separate work.
4 Torus simulations
4.1 Initial conditions
As with any fluid code, vacuum regions must be avoided and hence we apply floor values for the restmass density (\(\rho_{\mathrm{fl}} = 10^{5} r^{3/2}\)) and the gas pressure (\(p_{\mathrm{fl}} = 1/3\times10^{7} r^{5/2}\)). In practice, for all cells which satisfy \(\rho\le\rho_{\mathrm{fl}} \) we set \(\rho=\rho_{\mathrm{fl}}\), in addition if \(p \le p_{\mathrm{fl}}\), we set \(p = p_{\mathrm{fl}}\).
The simulations are performed using horizon penetrating logarithmic KS coordinates (corresponding to our set of modified KS coordinates with \(h=0\) and \(R_{0}=0\)). In the 2D cases, the simulation domain covers \(r_{\mathrm{KS}} \in[0.96 r_{\mathrm{h}}, 2{,}500\mbox{ M}]\) and \(\theta\in[0,\pi]\), where \(r_{\mathrm{h}} \simeq1.35\mbox{ M}\). In the 3D cases, we slightly excise the axial region \(\theta\in[0.02\pi,0.98\pi]\) and adopt \(\phi\in[0, 2\pi]\). We set the boundary conditions in the horizon and at \(r=2{,}500\mbox{ M}\) to zero gradient in primitive variables. The θboundary is handled as follows: when the domain extends all the way to the poles (as in our 2D cases), we adopt ‘hard’ boundary conditions, thus setting the flux through the pole manually to zero. For the excised cone in the 3D cases, we use reflecting ‘soft’ boundary conditions on primitive variables.
The timeupdate is performed with a twostep predictor corrector based on the TVDLF fluxes and PPM reconstruction. Furthermore, we set the CFL number to 0.4 and use the FCT algorithm. Typically, the runs are stopped after an evolution for \(t=5{,}000\mbox{ M}\), ensuring that no signal from the outflow boundaries can disturb the inner regions. To check convergence, we adopt the following resolutions: \(N_{r}\times N_{\theta}\in \{256\times128,512\times256,1{,}024\times512 \}\) in the 2D cases and \(N_{r}\times N_{\theta}\times N_{\phi}\in\{128\times64\times64, 192\times 96\times96, 256\times128\times128, 384\times192\times192\}\) in the 3D runs. In the following, the runs are identified via their resolution in θdirection. For the purpose of validation, we ran the 2D cases also with the HARM3D code (Noble et al. 2009).^{10}
4.2 2D results
4.2.1 Comparison to HARM3D
For validation purposes we simulated the same initial conditions with the HARM3D code. Wherever possible, we have made identical choices for the algorithm used in both codes, that is: PPM reconstruction, TVDLF Riemann solver and a two step time update. It is important to note that the outer radial boundary differs in both codes: while the HARM3D setup implements outflow boundary conditions at \(r=50\mbox{ M}\), in the BHAC runs the domain and radial grid is doubled in the logarithmic KerrSchild coordinates, yielding identical resolution in the region of interest. This ensures that no boundary effects compromise the BHAC simulation. Next to the boundary conditions, also the initial random perturbation varies in both codes which can amount to a slightly different dynamical evolution.
After verifying good agreement in the qualitative evolution, we quantify with both codes Ṁ and \(\phi_{B}\) according to Eqs. (95) and (96). The results are shown in Figure 14. Onsettime of accretion, magnitude and overall behaviour are in excellent agreement, despite the chaotic nature of the turbulent flow. We also find the same trend with respect to the resolutiondependence of the results: upon doubling the resolution, the accretion rate \(\langle\dot{M}\rangle\), averaged over \(t\in[1{,}000,2{,}000]\), increases significantly by a factor of 1.908 and 1.843 for BHAC and HARM, respectively. For \(\langle\phi_{B}\rangle\), the factors are 1.437 and 1.484. At a given resolution, the values for \(\langle \dot{M}\rangle\) and \(\langle\phi_{B}\rangle\) agree between the two codes within their standard deviations. Furthermore, we have verified that these same resolution variations are within the runtorun deviations due to a different random number seed for the initial perturbation.
4.3 3D results
Four different numerical resolutions were run which allows a first convergence analysis of the magnetised torus accretion scenario. Based on the convergence study, we can estimate which numerical resolutions are required for meaningful observational predictions derived from GRMHD simulations of this type.
Since we attempt to solve the set of dissipationfree ideal MHD equations, convergence in the strict sense cannot be achieved in the presence of a turbulent cascade [see also the discussion in Sorathia et al. (2012), Hawley et al. (2013)].^{11} Instead, given sufficient scale separation, one might hope to find convergence in quantities of interest like the disk averages and accretion rates. The convergence of various indicators in similar GRMHD torus simulations was addressed for example by (Shiokawa et al. 2012). The authors found signs for convergence in most quantifications when adopting a resolution of \(192\times192\times128\), however no convergence was found in the correlation length of the magnetic field. Hence the question as to whether GRMHD torus simulations can be converged with the available computational power is still an open one.
Although the simulations were run until \(t=5{,}000\mbox{ M}\), in order to enable comparison with the 2D simulations, we deliberately set the averaging time to \(t\in[1{,}000\mbox{ M}, 2{,}000\mbox{ M}]\). Figure 18 shows that as the resolution is increased, the diskaveraged 3D data approaches the much higher resolution 2D results shown in Figure 15, indicating that the dynamics are dominated by the axisymmetric MRI modes at early times. When the resolution is increased from \(N_{\theta}=128\) to \(N_{\theta}=192\), the diskaveraged profiles generally agree within their standard deviations, although we observe a continuing trend towards higher gas pressures and magnetic pressures in the outer regions \(r\in[30\mbox{ M},50\mbox{ M}]\). The overall computational cost quickly becomes significant: for the \(N_{\theta}=128\) simulation we spent 100 K CPU hours on the Iboga cluster equipped with Intel(R) Xeon(R) E52640 v4 processors. As the runtime scales with resolution according to \(N_{\theta}^{4}\), doubling resolution would cost a considerable 1.6 M CPU hours.
4.4 Effect of AMR
In order to investigate the effect of the AMR treatment, we have performed a 2D AMRGRMHD simulation of the torus setup. It is clear that whether a simulation can benefit from adaptive mesh refinement is very much dependent on the physical scenario under investigation. For example, in the hydrodynamic simulations of recoiling BHs due to (Meliani et al. 2017), refinement on the spiral shock was demonstrated to yield significant speedups at a comparable quality of solution. This is understandable as the numerical error is dominated by the shock hypersurface. In the turbulent accretion problem, whether automated mesh refinement yields any benefits is not clear.
The initial conditions for this test are the same as those used in Section 4.1. However, due to the limitation of current AMR treatment, we resort to the GLM divergence cleaning method. Three refinement levels are used and refinement is triggered by the error estimator due to (Löhner 1987) with the tolerance set to \(\epsilon_{\mathrm{t}} =0.1\) (see Section 2.11). The numerical resolution in the base level is set to \(N_{r} \times N_{\theta}= 128 \times 128\). To test the validity and efficiency, we also perform the same simulation in a uniform grid with resolution of \(N_{r} \times N_{\theta}= 512 \times512\) which corresponds to the resolution on the highest AMR level.
CPU hours (CPUH) spent by the simulations of the 2D magnetised torus at uniform resolution and fraction of that time spent by the equivalent AMR runs up to \(\pmb{t=2{,}000\mbox{ M}}\)
Grid size \(\boldsymbol{(N_{r} \times N_{\theta})}\)  CPU time uniform [CPUH]  Equiv. AMR time fraction [ \(\boldsymbol{\epsilon_{\mathrm{t}}=0.1}\) ] 

512 × 512  674.0  0.643 
5 Radiation postprocessing
In order to compute synthetic observable images of the BH shadow and surrounding accretion flow it is necessary to perform generalrelativistic raytracing and GRRT postprocessing [see, e.g., Fuerst and Wu (2004), Vincent et al. (2011), Younsi et al. (2012), Younsi and Wu (2015), Chan et al. (2015), Dexter (2016), Pu et al. (2016), Younsi et al. (2016)]. In this article the GRRT code BHOSS (Black Hole Observations in Stationary Spacetimes) (Younsi et al. 2017) is used to perform these calculations. From BHAC, GRMHD simulation data are produced which are subsequently used as input for BHOSS. Although BHAC has full AMR capabilities, for the GRRT it is most expedient to output GRMHD data that has been regridded to a uniform grid.
Since these calculations are performed in postprocessing, the effects of radiation forces acting on the plasma during its magnetohydrodynamical evolution are not included. Additionally, the fastlight approximation has also been adopted in this study, i.e., it is assumed that the lightcrossing timescale is shorter than the dynamical timescale of the GRMHD simulation and the dynamical evolution of the GRMHD simulation as light rays propagate through it is not considered. Such calculations are considered in an upcoming article (Younsi et al. 2017).
Several different coordinate representations of the Kerr metric are implemented in BHOSS, including BoyerLindquist (BL), Logarithmic BL, Cartesian BL, KerrSchild (KS), Logarithmic KS, Modified KS and Cartesian KS. All GRMHD simulation data used in this study are specified in Logarithmic KS coordinates. Although BHOSS can switch between all coordinate systems on the fly, it is most straightforward to perform the GRRT calculations in the same coordinate system as the GRMHD data, only adaptively switching to e.g., Cartesian KS when near the polar region. This avoids the need to transform between coordinate systems at every point along every ray in the GRMHD data interpolation, saving computational time.
5.1 Radiative transfer equation
Electromagnetic radiation is described by null geodesics of the background spacetime (in this case Kerr), and these are calculated in BHOSS using a RungeKuttaFehlberg integrator with fourth order adaptive step sizing and 5th order error control. Any spacetime metric may be considered in BHOSS, as long as the contravariant or covariant metric tensor components are specified, even if they are only tabulated on a grid. For the calculations presented in this article the Kerr spacetime is written algebraically and in closedform.
The observer is calculated by constructing a local orthonormal tetrad using trial basis vectors. These basis vectors are then orthonormalized using the metric tensor through a modified GramSchmidt procedure. The initial conditions of each ray for the coordinate system under consideration are then calculated and the geodesics are integrated backwards in time from the observer, until they either: (i) escape to infinity (exit the computational domain), (ii) are captured by the BH, or (iii) are effectively absorbed by the accretion flow.
5.2 BHOSSsimulated emission from Sgr A*
Having in mind the upcoming radio observations of the BH candidate Sgr A* at the Galactic Centre, the following discussion presents synthetic images of Sgr A*. The GRMHD simulations evolve a single fluid (of ions) and are scalefree in length and mass. Consequently a scaling must be applied before performing GRRT calculations. Within BHOSS this means specifying the BH mass, which sets the length and time scales, and specifying either the mass accretion rate or an electron density scale, which scales the gas density, temperature and magnetic field strength to that of a radiating electron.
Since the GRMHD simulation is of a single fluid, it is necessary to adopt a prescription for the local electron temperature and restmass density. Several such prescriptions exist, some which scale using the mass accretion rate [see, e.g., Mościbrodzka et al. (2009), Mościbrodzka et al. (2014), Dexter et al. (2010)], scale using density to determine the electron number density and physical accretion rate [see, e.g., Chan et al. (2015), Chan et al. (2015)], and some by employing a timedependent smoothing model of the mass accretion rate [see, e.g., Shiokawa et al. (2012)].
Each image is generated using a uniform grid of \(1{,}000\times1{,}000\) rays, sampling 60 uniformly logarithmically spaced frequency bins between 10^{9} Hz and 10^{15} Hz. All panels depict the observed image as seen at an observational frequency of 230 GHz, i.e. the frequency at which the EHT will image Sgr A*. This resolution is chosen because the integrated flux over the entire raytraced image is convergent: doubling the resolution from \(500\times500\) to \(1{,}000\times1{,}000\) yields an increase of 0.17%, and from \(1{,}000\times1{,}000\) to \(2{,}000\times2{,}000\) an increase of 0.09%.
In practical GRRT calculations only simulation data which has already reached a quasisteady state, typically \(t>2{,}000\mbox{ M}\), is used. In this study we focus on the observational appearance of the accretion flow and BH shadow image. The detailed discussion of the spectrum, variability and plasma models warrants a separate study.
5.3 Comparison of images
A natural and important question arises from GRRT calculations of BH shadows: do raytraced images of GRMHD simulation data converge as the resolution of the GRMHD simulation is increased? The existence of an optimal resolution, beyond which differences in images are small, implies that one can save additional computational time and expense by running the simulation at this optimal resolution. It would also imply that the GRMHD data satisfactorily capture the smallscale structure, turbulence and variations of the accretion flow. As such, it is informative to investigate the convergence of BH shadow images obtained from GRMHD simulation data of differing resolutions, both quantitatively and qualitatively.
A direct consequence of increasing the resolution of the GRMHD data is resolving the finescale turbulent structure of the accretion flow. The characteristic dark shadow delineating the BH shadow can be clearly seen in all images. As the resolution of the GRMHD data is increased, the images become less diffuse. It is difficult with the naked eye to draw firm physical conclusions, and so in the following we perform a quantitative pixelbypixel analysis.
The rightmost column of Eq. (110) denotes the crosscorrelation values, \(\mathcal{C}_{i,4}\), in descending order between images, i.e., the crosscorrelation of image 4 with images 1, 2, 3 and 4 respectively. Since \(\mathcal{C}_{i+1,4}>\mathcal{C}_{i,4}\) it is clear that the similarity between images increases as the resolution of the GRMHD simulation is increased. Similarly, for image 3 it is found that \(\mathcal{C}_{i+1,3}>\mathcal{C}_{i,3}\). Finally, it also follows that \(\mathcal{C}_{3,4}>\mathcal{C}_{2,3}>\mathcal{C}_{1,2}\), i.e., the correlation between successive pairs of images increases with increasing resolution, demonstrating the convergence of the GRMHD simulations with increasing grid resolution. Whilst the lowest resolution of \(128\times 64 \times64\) is certainly insufficient, both difference images and crosscorrelation measures indicate that a resolution of \(256\times128 \times128\) is sufficient and represents a good compromise.
6 Conclusions and outlook
We have described the capabilities of BHAC, a new multidimensional generalrelativistic magnetohydrodynamics code developed to perform hydrodynamical and MHD simulations of accretion flows onto compact objects in arbitrary stationary spacetimes exploiting the numerous advantages of AMR techniques. The code has been tested with several one, two and three dimensional scenarios in specialrelativistic and generalrelativistic MHD regimes. For validation, GRMHD simulations of MRI unstable tori have been compared with another wellknown and tested GRMHD code, the HARM3D code. BHAC shows very good agreement with the HARM3D results, both qualitatively and quantitatively. As a first demonstration of the AMR capabilities in multiscale simulations, we performed the magnetizedtorus accretion test with and without AMR. Despite the latter intrinsically implies an overhead of \(\sim10\%\), the AMR runtime amounted to 65% of that relative to the uniform grid, simply as a result of the more economical use of grid cells in the block based AMR. At the same time, the AMR results agree very well with the more expensive uniformgrid results. With increasing dynamic range, we expect the advantages of AMR to increase even more significantly, rendering it a useful tool for simulations involving structures of multiple physical scales.
Currently, two methods controlling the divergence of the magnetic field are available in BHAC and we compared them in three test problems. Although solutions obtained with the cellcentered fluxinterpolated constrained transport (FCT) algorithm and the divergence cleaning scheme (GLM) yield the same (correct) physical behaviour in the case of weak magnetic fields, FCT performs considerably better in the presence of strong magnetic fields. In particular, FCT is less diffusive than GLM, is able to preserve a stationary solution, and it creates less spurious structures in the magnetic field. For example, the use of GLM in the case of accretion scenarios with strong magnetic fields leads to worrisome artefacts in the highly magnetised funnel region. The development of a constrained transport scheme compatible with AMR is ongoing and will be presented in a separate work (Olivares et al. 2017).
The EHTC and its European contribution, the BlackHoleCam project (Goddi et al. 2016), aim at obtaining horizonscale radio images of the BH candidate at the Galactic Center. In anticipation of these results, we have used the 3D GRMHD simulations as input for GRRT calculations with the newly developed BHOSS code (Younsi et al. 2017). We found that the intensity maps resulting from different resolution GRMHD simulations agree very well, even when comparing snapshot data that was not time averaged. In particular, the normalised crosscorrelation between images achieves up to 94.8% similarity between the two highest resolution runs. Furthermore, the agreement between two images converges as the resolution of the GRMHD simulation is increased. Based on this comparison, we find that moderate grid resolutions of \(256\times128\times128\) (corresponding to physical resolutions of \(\Delta r_{\mathrm{KS}}\times\Delta\theta_{\mathrm{KS}}\times\Delta\phi _{\mathrm{KS}} = 0.04\mbox{ M} \times0.024 \mbox{ rad}\times0.05 \mbox{ rad}\) at the horizon) yield sufficiently converged intensity maps. Given the large and likely degenerate parameter space and the uncertainty in modelling of the electron distribution, this result is encouraging, as it demonstrates that the predicted synthetic image is quite robust against the everpresent time variability, but also against the impact that the grid resolution of the GRMHD simulations might have. In addition, independent information on the spatial orientation and magnitude of the spin, such as the one that could be deduced from the dynamics of a pulsar near Sgr A* (Psaltis et al. 2016), would greatly reduce the space of degenerate solutions and further increase the robustness of the predictions that BHAC will provide in terms of synthetic images.
Finally, we have demonstrated the excellent flexibility of BHAC with a variety of different astrophysical scenarios that are ongoing and will be published shortly. These include: oscillating hydrodynamical equilibrium tori for the modelling of quasiperiodic oscillations (de Avellar et al. 2017), episodic jet formation (Porth et al. 2016) and magnetised tori orbiting nonrotating dilaton BHs (Mizuno et al. 2017).
Using \(\tau=UD\) instead of U improves accuracy in regions of low energy and enables one to consistently recover the Newtonian limit.
In the generalrelativistic hydrodynamic WhiskyTHC code (Radice and Rezzolla 2012; Radice et al. 2014), this desirable property allows to set floors on density close to the limit of floating point precision \(\sim10^{16}\rho_{\mathrm{ref}}\).
We note that for the reference solution we have relied here on the extensive set of tests performed in flat spacetime within the MPIAMRVAC framework; however, we could also have employed as reference solution the ‘exact’ solution as derived in Ref. (Giacomazzo and Rezzolla 2006).
Note that the discrepancy in \(v^{r}\) appears less dramatic when viewed in terms of the fourvelocity \(u^{r}\).
Even when the dissipation length is well resolved, highReynolds number flows show indications for positive Lyapunov exponents and thus nonconvergent chaotic behaviour see, e.g., Lecoanet et al. (2016).
Since we use the same Courant limited timestep for all gridlevels, the speedup is entirely due to saving in computational cells. The additional speedup that would be gained from (Berger and Oliger 1984)type hierarchical timesteps can be estimated from the level population of the simulation: the expected additional gain is only \(\sim8\%\) for this setup.
Declarations
Acknowledgements
It is a pleasure to thank Christian Fromm, Mariafelicia de Laurentis, Thomas Bronzwaer, Jordy Davelaar, Elias Most and Federico Guercilena for discussions. We are grateful to Scott Noble for the ability to use the HARM3D code for comparison and to Zakaria Meliani for input on the construction of BHAC. The initial setup for the toroidalfield equilibrium torus was kindly provided by Chris Fragile. The simulations were performed on LOEWE at the CSCFrankfurt and Iboga at ITP Frankfurt. We acknowledge technical support from Thomas Coelho.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Abramowicz, MA, Chen, X, Kato, S, Lasota, JP, Regev, O: Thermal equilibria of accretion disks. Astrophys. J. Lett. 438, 3739 (1995). doi:10.1086/187709 arXiv:astroph/9409018 ADSView ArticleGoogle Scholar
 Alcubierre, M: Introduction to 3 + 1 Numerical Relativity. Oxford University Press, Oxford (2008). doi:10.1093/acprof:oso/9780199205677.001.0001 MATHView ArticleGoogle Scholar
 Aloy, MA, Rezzolla, L: A powerful hydrodynamic booster for relativistic jets. Astrophys. J. 640, 115118 (2006) ADSView ArticleGoogle Scholar
 Anile, AM: Relativistic Fluids and MagnetoFluids. Cambridge University Press, Cambridge (1990) MATHView ArticleGoogle Scholar
 Anninos, P, Fragile, PC, Salmonson, JD: Cosmos++: relativistic magnetohydrodynamics on unstructured grids with local adaptive refinement. Astrophys. J. 635, 723 (2005). doi:10.1086/497294 ADSView ArticleGoogle Scholar
 Antón, L, Zanotti, O, Miralles, JA, Martí, JM, Ibáñez, JM, Font, JA, Pons, JA: Numerical \(3+1\) general relativistic magnetohydrodynamics: a local characteristic approach. Astrophys. J. 637, 296 (2006) arXiv:astroph/0506063 ADSView ArticleGoogle Scholar
 Arnowitt, R, Deser, S, Misner, CW: Republication of: the dynamics of general relativity. Gen. Relativ. Gravit. 40, 19972027 (2008). doi:10.1007/s1071400806611 arXiv:grqc/0405109 ADSMATHView ArticleGoogle Scholar
 Asada, K, Nakamura, M: The structure of the M87 Jet: a transition from parabolic to conical streamlines. Astrophys. J. Lett. 745, 28 (2012). doi:10.1088/20418205/745/2/L28 arXiv:1110.1793 ADSView ArticleGoogle Scholar
 Avara, MJ, McKinney, JC, Reynolds, CS: Efficiency of thin magnetically arrested discs around black holes. Mon. Not. R. Astron. Soc. 462, 636648 (2016). doi:10.1093/mnras/stw1643 arXiv:1508.05323 ADSView ArticleGoogle Scholar
 Baiotti, L, Hawke, I, Montero, PJ, Löffler, F, Rezzolla, L, Stergioulas, N, Font, JA, Seidel, E: Threedimensional relativistic simulations of rotating neutronstar collapse to a Kerr black hole. Phys. Rev. D 71(2), 024035 (2005). doi:10.1103/PhysRevD.71.024035 arXiv:grqc/0403029 ADSView ArticleGoogle Scholar
 Baiotti, L, Rezzolla, L: Binary neutronstar mergers: a review of Einstein’s richest laboratory (2016). arXiv:1607.03540
 Balbus, SA, Hawley, JF: A powerful local shear instability in weakly magnetized disks. I  linear analysis. II  nonlinear evolution. Astrophys. J. 376, 214233 (1991). doi:10.1086/170270 ADSView ArticleGoogle Scholar
 Balbus, SA, Hawley, JF: Instability, turbulence, and enhanced transport in accretion disks. Rev. Mod. Phys. 70, 153 (1998). doi:10.1103/RevModPhys.70.1 ADSView ArticleGoogle Scholar
 Balsara, DS, Kim, J: A comparison between divergencecleaning and staggeredmesh formulations for numerical magnetohydrodynamics. Astrophys. J. 602, 10791090 (2004). doi:10.1086/381051 arXiv:astroph/0310728 ADSView ArticleGoogle Scholar
 Banyuls, F, Font, JA, Ibáñez, JM, Martí, JM, Miralles, JA: Numerical \(3+1\) generalrelativistic hydrodynamics: a local characteristic approach. Astrophys. J. 476, 221 (1997). doi:10.1086/303604 ADSView ArticleGoogle Scholar
 Barkov, MV, Aharonian, FA, BoschRamon, V: Gammaray flares from red giant/jet interactions in active galactic nuclei. Astrophys. J. 724, 15171523 (2010). doi:10.1088/0004637X/724/2/1517 arXiv:1005.5252 ADSView ArticleGoogle Scholar
 Barkov, MV, Komissarov, SS: Stellar explosions powered by the BlandfordZnajek mechanism. Mon. Not. R. Astron. Soc. 385, 2832 (2008). doi:10.1111/j.17453933.2008.00427.x ADSView ArticleGoogle Scholar
 Beckwith, K, Hawley, JF, Krolik, JH: The influence of magnetic field geometry on the evolution of black hole accretion flows: similar disks, drastically different jets. Astrophys. J. 678, 11801199 (2008). doi:10.1086/533492 ADSView ArticleGoogle Scholar
 Begelman, MC: Radiatively inefficient accretion: breezes, winds and hyperaccretion. Mon. Not. R. Astron. Soc. 420, 29122923 (2012). doi:10.1111/j.13652966.2011.20071.x arXiv:1110.5356 ADSView ArticleGoogle Scholar
 Berger, MJ, Oliger, J: Adaptive mesh refinement for hyperbolic partial differential equations. J. Comput. Phys. 53, 484512 (1984) ADSMathSciNetMATHView ArticleGoogle Scholar
 Biretta, JA, Owen, FN, Cornwell, TJ: A search for motion and flux variations in the M87 jet. Astrophys. J. 342, 128134 (1989). doi:10.1086/167581 ADSView ArticleGoogle Scholar
 Blandford, RD, Begelman, MC: On the fate of gas accreting at a low rate on to a black hole. Mon. Not. R. Astron. Soc. 303, 15 (1999). doi:10.1046/j.13658711.1999.02358.x arXiv:astroph/9809083 ADSView ArticleGoogle Scholar
 Blandford, RD, Znajek, RL: Electromagnetic extraction of energy from Kerr black holes. Mon. Not. R. Astron. Soc. 179, 433456 (1977) ADSView ArticleGoogle Scholar
 Bondi, H: On spherically symmetric accretion. Mon. Not. R. Astron. Soc. 112, 195 (1952) ADSView ArticleGoogle Scholar
 Čada, M, Torrilhon, M: Compact thirdorder limiter functions for finite volume methods. J. Comput. Phys. 228, 41184145 (2009). doi:10.1016/j.jcp.2009.02.020 ADSMathSciNetMATHView ArticleGoogle Scholar
 Calder, AC, Fryxell, B, Plewa, T, Rosner, R, Dursi, LJ, Weirs, VG, Dupont, T, Robey, HF, Kane, JO, Remington, BA, Drake, RP, Dimonte, G, Zingale, M, Timmes, FX, Olson, K, Ricker, P, MacNeice, P, Tufo, HM: On validating an astrophysical simulation code. Astrophys. J. Suppl. Ser. 143, 201229 (2002). doi:10.1086/342267 arXiv:astroph/0206251 ADSView ArticleGoogle Scholar
 Chan, CK, Psaltis, D, Özel, F, Medeiros, L, Marrone, D, Sądowski, A, Narayan, R: Fast variability and millimeter/IR flares in GRMHD models of Sgr A* from strongfield gravitational lensing. Astrophys. J. 812, 103 (2015). doi:10.1088/0004637X/812/2/103 arXiv:1505.01500 ADSView ArticleGoogle Scholar
 Chan, CK, Psaltis, D, Özel, F, Narayan, R, Sądowski, A: The power of imaging: constraining the plasma properties of GRMHD simulations using EHT observations of Sgr A*. Astrophys. J. 799, 1 (2015). doi:10.1088/0004637X/799/1/1 arXiv:1410.3492 ADSView ArticleGoogle Scholar
 Colella, P, Woodward, PR: The piecewise parabolic method (ppm) for gasdynamical simulations. J. Comput. Phys. 54(1), 174201 (1984). doi:10.1016/00219991(84)901438 ADSMATHView ArticleGoogle Scholar
 Cowling, TG: The magnetic field of sunspots. Mon. Not. R. Astron. Soc. 94, 3948 (1933). doi:10.1093/mnras/94.1.39 ADSMATHView ArticleGoogle Scholar
 Davis, SF: Simplified secondorder Godunovtype methods. SIAM J. Sci. Stat. Comput. 9, 445473 (1988) MathSciNetMATHView ArticleGoogle Scholar
 de Avellar, M, et al.: (2017) in prepratation Google Scholar
 De Villiers, JP, Hawley, JF: A numerical method for general relativistic magnetohydrodynamics. Astrophys. J. 589, 458480 (2003). doi:10.1086/373949 arXiv:astroph/0210518 ADSView ArticleGoogle Scholar
 Dedner, A, Kemm, F, Kröner, D, Munz, CD, Schnitzer, T, Wesenberg, M: Hyperbolic divergence cleaning for the MHD equations. J. Comput. Phys. 175, 645673 (2002). doi:10.1006/jcph.2001.6961 ADSMathSciNetMATHView ArticleGoogle Scholar
 Del Zanna, L, Zanotti, O, Bucciantini, N, Londrillo, P: ECHO: a Eulerian conservative highorder scheme for general relativistic magnetohydrodynamics and magnetodynamics. Astron. Astrophys. 473, 1130 (2007). doi:10.1051/00046361:20077093 arXiv:0704.3206 ADSView ArticleGoogle Scholar
 Dexter, J: A public code for general relativistic, polarised radiative transfer around spinning black holes. Mon. Not. R. Astron. Soc. 462, 115136 (2016). doi:10.1093/mnras/stw1526 arXiv:1602.03184 ADSView ArticleGoogle Scholar
 Dexter, J, Agol, E, Fragile, PC: Millimeter flares and VLBI visibilities from relativistic simulations of magnetized accretion onto the galactic center black hole. Astrophys. J. Lett. 703, 142146 (2009). doi:10.1088/0004637X/703/2/L142 arXiv:0909.0267 ADSView ArticleGoogle Scholar
 Dexter, J, Agol, E, Fragile, PC, McKinney, JC: The submillimeter bump in Sgr A* from relativistic MHD simulations. Astrophys. J. 717, 10921104 (2010). doi:10.1088/0004637X/717/2/1092 arXiv:1005.4062 ADSView ArticleGoogle Scholar
 Dexter, J, McKinney, JC, Agol, E: The size of the jet launching region in M87. Mon. Not. R. Astron. Soc. 421, 15171528 (2012). doi:10.1111/j.13652966.2012.20409.x arXiv:1109.6011 ADSView ArticleGoogle Scholar
 Dionysopoulou, K, Alic, D, Palenzuela, C, Rezzolla, L, Giacomazzo, B: Generalrelativistic resistive magnetohydrodynamics in three dimensions: formulation and tests. Phys. Rev. D 88(4), 044020 (2013). doi:10.1103/PhysRevD.88.044020 arXiv:1208.3487 ADSView ArticleGoogle Scholar
 Doeleman, S, Agol, E, Backer, D, Baganoff, F, Bower, GC, Broderick, A, Fabian, A, Fish, V, Gammie, C, Ho, P, Honman, M, Krichbaum, T, Loeb, A, Marrone, D, Reid, M, Rogers, A, Shapiro, I, Strittmatter, P, Tilanus, R, Weintroub, J, Whitney, A, Wright, M, Ziurys, L: Imaging an event horizon: submmVLBI of a super massive black hole. In: Astro2010: The Astronomy and Astrophysics Decadal Survey. ArXiv Astrophysics ePrints, vol. 2010 (2009) arXiv:0906.3899 Google Scholar
 Duez, MD, Liu, YT, Shapiro, SL, Stephens, BC: Relativistic magnetohydrodynamics in dynamical spacetimes: numerical methods and tests. Phys. Rev. D 72(2), 024028 (2005). doi:10.1103/PhysRevD.72.024028 arXiv:astroph/0503420 ADSView ArticleGoogle Scholar
 Eisenhauer, F, Perrin, G, Brandner, W, Straubmeier, C, Richichi, A, Gillessen, S, Berger, JP, Hippler, S, Eckart, A, Schöller, M, Rabien, S, Cassaing, F, Lenzen, R, Thiel, M, Clénet, Y, Ramos, JR, Kellner, S, Fédou, P, Baumeister, H, Hofmann, R, Gendron, E, Boehm, A, Bartko, H, Haubois, X, Klein, R, DoddsEden, K, Houairi, K, Hormuth, F, Gräter, A, Jocou, L, Naranjo, V, Genzel, R, Kervella, P, Henning, T, Hamaus, N, Lacour, S, Neumann, U, Haug, M, Malbet, F, Laun, W, Kolmeder, J, Paumard, T, Rohloff, RR, Pfuhl, O, Perraut, K, Ziegleder, J, Rouan, D, Rousset, G: GRAVITY: getting to the event horizon of Sgr A*. In: Optical and Infrared Interferometry. Proc. SPIE, vol. 7013, p. 70132 (2008). doi:10.1117/12.788407 arXiv:0808.0063 View ArticleGoogle Scholar
 Etienne, ZB, Liu, YT, Paschalidis, V, Shapiro, SL: General relativistic simulations of blackholeneutronstar mergers: effects of magnetic fields. Phys. Rev. D 85(6), 064029 (2012). doi:10.1103/PhysRevD.85.064029 arXiv:1112.0568 ADSView ArticleGoogle Scholar
 Etienne, ZB, Paschalidis, V, Haas, R, Mösta, P, Shapiro, SL: IllinoisGRMHD: an opensource, userfriendly GRMHD code for dynamical spacetimes. Class. Quantum Gravity 32(17), 175009 (2015). doi:10.1088/02649381/32/17/175009 arXiv:1501.07276 ADSMathSciNetMATHView ArticleGoogle Scholar
 Evans, CR, Hawley, JF: Simulation of magnetohydrodynamic flows  a constrained transport method. Astrophys. J. 332, 659677 (1988). doi:10.1086/166684 ADSView ArticleGoogle Scholar
 Faber, JA, Baumgarte, TW, Etienne, ZB, Shapiro, SL, Taniguchi, K: Relativistic hydrodynamics in the presence of puncture black holes. Phys. Rev. D 76(10), 104021 (2007). doi:10.1103/PhysRevD.76.104021 arXiv:0708.2436 ADSView ArticleGoogle Scholar
 Fender, RP, Belloni, TM, Gallo, E: Towards a unified model for black hole Xray binary jets. Mon. Not. R. Astron. Soc. 355, 11051118 (2004). doi:10.1111/j.13652966.2004.08384.x arXiv:astroph/0409360 ADSView ArticleGoogle Scholar
 Fishbone, LG, Moncrief, V: Relativistic fluid disks in orbit around Kerr black holes. Astrophys. J. 207, 962976 (1976) ADSMathSciNetView ArticleGoogle Scholar
 Font, JA: Numerical hydrodynamics in general relativity. Living Rev. Relativ. 6, 4 (2003) ADSMathSciNetMATHView ArticleGoogle Scholar
 Font, JA, Daigne, F: The runaway instability of thick discs around black holes  I. The constant angular momentum case. Mon. Not. R. Astron. Soc. 334, 383400 (2002) ADSView ArticleGoogle Scholar
 Font, JA, Ibanez, JM, Marquina, A, Marti, JM: Multidimensional relativistic hydrodynamics: characteristic fields and modern highresolution shockcapturing schemes. Astron. Astrophys. 282, 304314 (1994) ADSGoogle Scholar
 Foucart, F, Chandra, M, Gammie, CF, Quataert, E: Evolution of accretion discs around a Kerr black hole using extended magnetohydrodynamics. Mon. Not. R. Astron. Soc. 456, 13321345 (2016). doi:10.1093/mnras/stv2687 arXiv:1511.04445 ADSView ArticleGoogle Scholar
 Fragile, PC, Blaes, OM, Anninos, P, Salmonson, JD: Global general relativistic magnetohydrodynamic simulation of a tilted black hole accretion disk. Astrophys. J. 668, 417429 (2007). doi:10.1086/521092 arXiv:0706.4303 ADSView ArticleGoogle Scholar
 Fryxell, B, Olson, K, Ricker, P, Timmes, FX, Zingale, M, Lamb, DQ, MacNeice, P, Rosner, R, Truran, JW, Tufo, H: FLASH: an adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes. Astrophys. J. Suppl. Ser. 131, 273334 (2000). doi:10.1086/317361 ADSView ArticleGoogle Scholar
 Fuerst, SV, Wu, K: Radiation transfer of emission lines in curved spacetime. Astron. Astrophys. 424, 733746 (2004). doi:10.1051/00046361:20035814 arXiv:astroph/0406401 ADSMATHView ArticleGoogle Scholar
 Galeazzi, F, Kastaun, W, Rezzolla, L, Font, JA: Implementation of a simplified approach to radiative transfer in general relativity. Phys. Rev. D 88(6), 064009 (2013). doi:10.1103/PhysRevD.88.064009 arXiv:1306.4953 ADSView ArticleGoogle Scholar
 Gammie, CF, McKinney, JC, Tóth, G: Harm: a numerical scheme for general relativistic magnetohydrodynamics. Astrophys. J. 589, 458 (2003) arXiv:astroph/0301509 ADSView ArticleGoogle Scholar
 García, A, Galtsov, D, Kechkin, O: Class of stationary axisymmetric solutions of the EinsteinMaxwellDilatonAxion field equations. Phys. Rev. Lett. 74, 12761279 (1995). doi:10.1103/PhysRevLett.74.1276 ADSMathSciNetMATHView ArticleGoogle Scholar
 Giacomazzo, B, Rezzolla, L: The exact solution of the Riemann problem in relativistic MHD. J. Fluid Mech. 562, 223259 (2006) arXiv:grqc/0507102 ADSMathSciNetMATHView ArticleGoogle Scholar
 Giacomazzo, B, Rezzolla, L: WhiskyMHD: a new numerical code for general relativistic magnetohydrodynamics. Class. Quantum Gravity 24, 235 (2007). doi:10.1088/02649381/24/12/S16 arXiv:grqc/0701109 ADSMathSciNetMATHView ArticleGoogle Scholar
 Giannios, D, Sironi, L: The S2 star as a probe of the accretion disc of Sgr A*. Mon. Not. R. Astron. Soc. 433, 2529 (2013). doi:10.1093/mnrasl/slt051 arXiv:1303.2115 ADSView ArticleGoogle Scholar
 Goddi, C, Falcke, H, Kramer, M, Rezzolla, L, Brinkerink, C, Bronzwaer, T, Deane, R, De Laurentis, M, Desvignes, G, Davelaar, JRJ, Eisenhauer, F, Eatough, R, FragaEncinas, R, Fromm, CM, Gillessen, S, Grenzebach, A, Issaoun, S, Janßen, M, Konoplya, R, Krichbaum, TP, Laing, R, Liu, K, Lu, RS, Mizuno, Y, Moscibrodzka, M, Müller, C, Olivares, H, Porth, O, Pfuhl, O, Ros, E, Roelofs, F, Schuster, K, Tilanus, R, Torne, P, van Bemmel, I, van Langevelde, HJ, Wex, N, Younsi, Z, Zhidenko, A: BlackHoleCam: fundamental physics of the Galactic center. Int. J. Mod. Phys. D (2016) submitted. arXiv:1606.08879
 Gold, R, McKinney, JC, Johnson, MD, Doeleman, SS: Probing the magnetic field structure in Sgr A* on black hole horizon scales with polarized radiative transfer simulations. ArXiv eprints (2016). arXiv:1601.05550
 Gottlieb, S, Shu, C: Total Variation Diminishing RungeKutta schemes. Math. Comput. 67, 7385 (1998) ADSMathSciNetMATHView ArticleGoogle Scholar
 Gourgoulhon, E: \(3+1\) Formalism and bases of numerical relativity. ArXiv General Relativity and Quantum Cosmology eprints (2007). arXiv:grqc/0703035
 Gourgouliatos, KN, Fendt, C, ClausenBrown, E, Lyutikov, M: Magnetic field structure of relativistic jets without current sheets. Mon. Not. R. Astron. Soc. 419, 30483059 (2012). doi:10.1111/j.13652966.2011.19946.x arXiv:1110.0838 ADSView ArticleGoogle Scholar
 Hamlin, ND, Newman, WI: Role of the KelvinHelmholtz instability in the evolution of magnetized relativistic sheared plasma flows. Phys. Rev. 87(4), 043101 (2013). doi:10.1103/PhysRevE.87.043101 ADSGoogle Scholar
 Harten, A, Lax, PD, van Leer, B: On upstream differencing and Godunovtype schemes for hyperbolic conservation laws. SIAM Rev. 25, 35 (1983). doi:10.1137/1025002 MathSciNetMATHView ArticleGoogle Scholar
 Hartle, JB, Thorne, KS: Slowly rotating relativistic stars. II. Models for neutron stars and supermassive stars. Astrophys. J. 153, 807 (1968). doi:10.1086/149707 ADSView ArticleGoogle Scholar
 Hawley, JF, Krolik, JH: Magnetically driven jets in the Kerr metric. Astrophys. J. 641, 103116 (2006). doi:10.1086/500385 arXiv:astroph/0512227 ADSView ArticleGoogle Scholar
 Hawley, JF, Richers, SA, Guan, X, Krolik, JH: Testing convergence for global accretion disks. Astrophys. J. 772, 102 (2013). doi:10.1088/0004637X/772/2/102 arXiv:1306.0243 ADSView ArticleGoogle Scholar
 Hawley, JF, Smarr, LL, Wilson, JR: A numerical study of nonspherical black hole accretion. I equations and test problems. Astrophys. J. 277, 296311 (1984). doi:10.1086/161696 ADSView ArticleGoogle Scholar
 Ho, LC: Radiatively inefficient accretion in nearby galaxies. Astrophys. J. 699, 626637 (2009). doi:10.1088/0004637X/699/1/626 arXiv:0906.4104 ADSView ArticleGoogle Scholar
 Hu, XY, Adams, NA, Shu, CW: Positivitypreserving method for highorder conservative schemes solving compressible Euler equations. J. Comput. Phys. 242, 169180 (2013). doi:10.1016/j.jcp.2013.01.024 arXiv:1203.1540 ADSMathSciNetMATHView ArticleGoogle Scholar
 Johannsen, T, Psaltis, D: Metric for rapidly spinning black holes suitable for strongfield tests of the nohair theorem. Phys. Rev. D 83(12), 124015 (2011). doi:10.1103/PhysRevD.83.124015 arXiv:1105.3191 ADSView ArticleGoogle Scholar
 Keppens, R, Meliani, Z: Linear wave propagation in relativistic magnetohydrodynamics. Phys. Plasmas 15(10), 102103 (2008). doi:10.1063/1.2991408 arXiv:0810.2416 ADSView ArticleGoogle Scholar
 Keppens, R, Meliani, Z, van Marle, AJ, Delmont, P, Vlasis, A, van der Holst, B: Parallel, gridadaptive approaches for relativistic hydro and magnetohydrodynamics. J. Comput. Phys. 231, 718744 (2012). doi:10.1016/j.jcp.2011.01.020 ADSMathSciNetMATHView ArticleGoogle Scholar
 Koide, S, Meier, DL, Shibata, K, Kudoh, T: General relativistic simulation of early jet formation in a rapidly rotating black hole magnetosphere. Astrophys. J. 536, 668674 (2000) ADSView ArticleGoogle Scholar
 Komissarov, SS: A Godunovtype scheme for relativistic magnetohydrodynamics. Mon. Not. R. Astron. Soc. 303, 343366 (1999). doi:10.1046/j.13658711.1999.02244.x ADSView ArticleGoogle Scholar
 Komissarov, SS: Magnetized tori around Kerr black holes: analytic solutions with a toroidal magnetic field. Mon. Not. R. Astron. Soc. 368, 9931000 (2006). doi:10.1111/j.13652966.2006.10183.x arXiv:astroph/0601678 ADSView ArticleGoogle Scholar
 Komissarov, SS, Barkov, MV, Vlahakis, N, Königl, A: Magnetic acceleration of relativistic active galactic nucleus jets. Mon. Not. R. Astron. Soc. 380, 5170 (2007). doi:10.1111/j.13652966.2007.12050.x arXiv:astroph/0703146 ADSView ArticleGoogle Scholar
 Koren, B: Numerical Methods for AdvectionDiffusion Problems. Notes on Numerical Fluid Mechanics, vol. 45. Vieweg, Braunschweig (1993) MATHGoogle Scholar
 Kozlowski, M, Jaroszynski, M, Abramowicz, MA: The analytic theory of fluid disks orbiting the Kerr black hole. Astron. Astrophys. 63, 209220 (1978) ADSMathSciNetMATHGoogle Scholar
 Landau, LD, Lifshitz, EM: The Classical Theory of Fields. Course of Theoretical Physics, vol. 2. Elsevier, Oxford (2004) MATHGoogle Scholar
 Lecoanet, D, McCourt, M, Quataert, E, Burns, KJ, Vasil, GM, Oishi, JS, Brown, BP, Stone, JM, O’Leary, RM: A validated nonlinear KelvinHelmholtz benchmark for numerical hydrodynamics. Mon. Not. R. Astron. Soc. 455, 42744288 (2016). doi:10.1093/mnras/stv2564 arXiv:1509.03630 ADSView ArticleGoogle Scholar
 Leung, PK, Gammie, CF, Noble, SC: Numerical calculation of magnetobremsstrahlung emission and absorption coefficients. Astrophys. J. 737, 21 (2011). doi:10.1088/0004637X/737/1/21 ADSView ArticleGoogle Scholar
 Löhner, R: An adaptive finite element scheme for transient problems in CFD. Comput. Methods Appl. Mech. Eng. 61, 323338 (1987). doi:10.1016/00457825(87)900983 ADSMATHView ArticleGoogle Scholar
 MacNeice, P, Olson, KM, Mobarry, C, de Fainchtein, R, Packer, C: Paramesh: A parallel adaptive mesh refinement community toolkit. Comput. Phys. Commun. 126(3), 330354 (2000). doi:10.1016/S00104655(99)005019 ADSMATHView ArticleGoogle Scholar
 Markoff, S: Sagittarius A* in context: daily flares as a probe of the fundamental Xray emission process in accreting black holes. Astrophys. J. Lett. 618, 103106 (2005). doi:10.1086/427841 arXiv:astroph/0412140 ADSView ArticleGoogle Scholar
 Marrone, DP, Moran, JM, Zhao, JH, Rao, R: An unambiguous detection of Faraday rotation in Sagittarius A*. Astrophys. J. Lett. 654, 5760 (2007). doi:10.1086/510850 arXiv:astroph/0611791 ADSView ArticleGoogle Scholar
 Martí, JM, Müller, E: Gridbased methods in relativistic hydrodynamics and magnetohydrodynamics. Living Rev. Comput. Astrophys. 1 (2015). doi:10.1007/lrca20153
 Martins, JR, Sturdza, P, Alonso, JJ: The complexstep derivative approximation. ACM Trans. Math. Softw. 29(3), 245262 (2003). doi:10.1145/838250.838251 MathSciNetMATHView ArticleGoogle Scholar
 McKinney, JC: General relativistic magnetohydrodynamic simulations of the jet formation and largescale propagation from black hole accretion systems. Mon. Not. R. Astron. Soc. 368, 15611582 (2006). doi:10.1111/j.13652966.2006.10256.x ADSView ArticleGoogle Scholar
 McKinney, JC, Blandford, RD: Stability of relativistic jets from rotating, accreting black holes via fully threedimensional magnetohydrodynamic simulations. Mon. Not. R. Astron. Soc. 394, 126130 (2009). doi:10.1111/j.17453933.2009.00625.x ADSView ArticleGoogle Scholar
 McKinney, JC, Gammie, CF: A measurement of the electromagnetic luminosity of a Kerr black hole. Astrophys. J. 611, 977995 (2004). doi:10.1086/422244 arXiv:astroph/0404512 ADSView ArticleGoogle Scholar
 McKinney, JC, Tchekhovskoy, A, Blandford, RD: Alignment of magnetized accretion disks and relativistic jets with spinning black holes. Science 339, 49 (2013). doi:10.1126/science.1230811 arXiv:1211.3651 ADSView ArticleGoogle Scholar
 McKinney, JC, Tchekhovskoy, A, Sadowski, A, Narayan, R: Threedimensional general relativistic radiation magnetohydrodynamical simulation of superEddington accretion, using a new code HARMRAD with M1 closure. ArXiv eprints (2013). arXiv:1312.6127
 McKinney, JC, Tchekhovskoy, A, Sadowski, A, Narayan, R: Threedimensional general relativistic radiation magnetohydrodynamical simulation of superEddington accretion, using a new code HARMRAD with M1 closure. Mon. Not. R. Astron. Soc. 441, 31773208 (2014). doi:10.1093/mnras/stu762 arXiv:1312.6127 ADSView ArticleGoogle Scholar
 Meliani, Z, Mizuno, Y, Olivares, H, Porth, O, Rezzolla, L, Younsi, Z: Simulations of recoiling black holes: adaptive mesh refinement and radiative transfer. Astron. Astrophys. 598, 38 (2017). doi:10.1051/00046361/201629191 arXiv:1606.08192 ADSView ArticleGoogle Scholar
 Meliani, Z, Sauty, C, Tsinganos, K, Vlahakis, N: Relativistic Parker winds with variable effective polytropic index. Astron. Astrophys. 425, 773781 (2004). doi:10.1051/00046361:20035653 arXiv:astroph/0407100 ADSView ArticleGoogle Scholar
 Michel, FC: Accretion of matter by condensed objects. Astrophys. Space Sci. 15, 153 (1972) ADSView ArticleGoogle Scholar
 Mignone, A: Highorder conservative reconstruction schemes for finite volume methods in cylindrical and spherical coordinates. J. Comput. Phys. 270, 784814 (2014). doi:10.1016/j.jcp.2014.04.001 arXiv:1404.0537 ADSMathSciNetMATHView ArticleGoogle Scholar
 Mignone, A, Zanni, C, Tzeferacos, P, van Straalen, B, Colella, P, Bodo, G: The PLUTO code for adaptive mesh computations in astrophysical fluid dynamics. Astrophys. J. Suppl. Ser. 198, 7 (2012). doi:10.1088/00670049/198/1/7 arXiv:1110.0740 ADSView ArticleGoogle Scholar
 Misner, CW, Thorne, KS, Wheeler, JA: Gravitation. W. H. Freeman, San Francisco (1973) Google Scholar
 Mizuno, Y, et al.: General relativistic magnetohydrodynamic simulations of an accretion torus in a nonrotating dilaton black hole (2017) in prepratation Google Scholar
 Mizuno, Y, Gómez, JL, Nishikawa, KI, Meli, A, Hardee, PE, Rezzolla, L: Recollimation shocks in magnetized relativistic jets. Astrophys. J. 809, 38 (2015). doi:10.1088/0004637X/809/1/38 arXiv:1505.00933 ADSView ArticleGoogle Scholar
 Mizuno, Y, Nishikawa, KI, Koide, S, Hardee, P, Fishman, GJ: RAISHIN: A highresolution threedimensional general relativistic magnetohydrodynamics code. ArXiv Astrophysics eprints (2006). arXiv:astroph/0609004
 Mocz, P, Pakmor, R, Springel, V, Vogelsberger, M, Marinacci, F, Hernquist, L: A moving mesh unstaggered constrained transport scheme for magnetohydrodynamics. Mon. Not. R. Astron. Soc. 463, 477488 (2016). doi:10.1093/mnras/stw2004 arXiv:1606.02310 ADSView ArticleGoogle Scholar
 Mościbrodzka, M, Falcke, H, Shiokawa, H: General relativistic magnetohydrodynamical simulations of the jet in M 87. Astron. Astrophys. 586, 38 (2016). doi:10.1051/00046361/201526630 arXiv:1510.07243 ADSView ArticleGoogle Scholar
 Mościbrodzka, M, Falcke, H, Shiokawa, H, Gammie, CF: Observational appearance of inefficient accretion flows and jets in 3D GRMHD simulations: application to Sagittarius A*. Astron. Astrophys. 570, 7 (2014). doi:10.1051/00046361/201424358 arXiv:1408.4743 ADSView ArticleGoogle Scholar
 Mościbrodzka, M, Gammie, CF, Dolence, JC, Shiokawa, H, Leung, PK: Radiative models of SGR A* from GRMHD simulations. Astrophys. J. 706, 497507 (2009). doi:10.1088/0004637X/706/1/497 arXiv:0909.5431 ADSView ArticleGoogle Scholar
 Nagakura, H, Yamada, S: General relativistic hydrodynamic simulations and linear analysis of the standing accretion shock instability around a black hole. Astrophys. J. 689, 391406 (2008). doi:10.1086/590325 arXiv:0808.4141 ADSView ArticleGoogle Scholar
 Narayan, R, Igumenshchev, IV, Abramowicz, MA: Selfsimilar accretion flows with convection. Astrophys. J. 539, 798808 (2000). doi:10.1086/309268 arXiv:astroph/9912449 ADSView ArticleGoogle Scholar
 Narayan, R, Yi, I: Advectiondominated accretion: a selfsimilar solution. Astrophys. J. Lett. 428, 1316 (1994). doi:10.1086/187381 arXiv:astroph/9403052 ADSView ArticleGoogle Scholar
 Narayan, R, Yi, I: Advectiondominated accretion: underfed black holes and neutron stars. Astrophys. J. 452, 710 (1995). doi:10.1086/176343 arXiv:astroph/9411059 ADSView ArticleGoogle Scholar
 Noble, SC, Gammie, CF, McKinney, JC, Del Zanna, L: Primitive variable solvers for conservative general relativistic magnetohydrodynamics. Astrophys. J. 641, 626637 (2006). doi:10.1086/500349 arXiv:astroph/0512420 ADSView ArticleGoogle Scholar
 Noble, SC, Krolik, JH, Hawley, JF: Direct calculation of the radiative efficiency of an accretion disk around a black hole. Astrophys. J. 692, 411421 (2009). doi:10.1088/0004637X/692/1/411 arXiv:0808.3140 ADSView ArticleGoogle Scholar
 Olivares, H, et al.: (2017) in prepratation Google Scholar
 Orszag, SA, Tang, CM: Smallscale structure of twodimensional magnetohydrodynamic turbulence. J. Fluid Mech. 90, 129143 (1979). doi:10.1017/S002211207900210X ADSView ArticleGoogle Scholar
 Palenzuela, C, Lehner, L, Reula, O, Rezzolla, L: Beyond ideal MHD: towards a more realistic modelling of relativistic astrophysical plasmas. Mon. Not. R. Astron. Soc. 394, 17271740 (2009). doi:10.1111/j.13652966.2009.14454.x arXiv:0810.1838 ADSView ArticleGoogle Scholar
 Porth, O, Olivares, H, Mizuno, Y, Younsi, Z, Rezzolla, L, Moscibrodzka, M, Falcke, H, Kramer, M: The black hole accretion code. ArXiv eprints (2016). arXiv:1611.09720
 Porth, O, Xia, C, Hendrix, T, Moschou, SP, Keppens, R: MPIAMRVAC for solar and astrophysics. Astrophys. J. Suppl. Ser. 214, 4 (2014). doi:10.1088/00670049/214/1/4 arXiv:1407.2052 ADSView ArticleGoogle Scholar
 Powell, KG: Approximate Riemann solver for magnetohydrodynamics (that works in more than one dimension). Computer Applications in Science and Engineering (ICASE). Technical report (1994) Google Scholar
 Powell, KG, Roe, PL, Linde, TJ, Gombosi, TI, De Zeeuw, DL: A solutionadaptive upwind scheme for ideal magnetohydrodynamics. J. Comput. Phys. 154, 284309 (1999). doi:10.1006/jcph.1999.6299 ADSMathSciNetMATHView ArticleGoogle Scholar
 Press, WH, Teukolsky, SA, Vetterling, WT, Flannery, BP: Numerical Recipes 3rd Edition: The Art of Scientific Computing Cambridge University Press, Cambridge (2007) MATHGoogle Scholar
 Psaltis, D, Wex, N, Kramer, M: A quantitative test of the nohair theorem with Sgr A* using stars, pulsars, and the event horizon telescope. Astrophys. J. 818, 121 (2016). doi:10.3847/0004637X/818/2/121 arXiv:1510.00394 ADSView ArticleGoogle Scholar
 Pu, HY, Yun, K, Younsi, Z, Yoon, SJ: Odyssey: a public GPUbased code for general relativistic radiative transfer in Kerr spacetime. Astrophys. J. 820, 105 (2016). doi:10.3847/0004637X/820/2/105 arXiv:1601.02063 ADSView ArticleGoogle Scholar
 Quataert, E, Gruzinov, A: Convectiondominated accretion flows. Astrophys. J. 539, 809814 (2000). doi:10.1086/309267 arXiv:astroph/9912440 ADSView ArticleGoogle Scholar
 Radice, D, Rezzolla, L: THC: a new highorder finitedifference highresolution shockcapturing code for specialrelativistic hydrodynamics. Astron. Astrophys. 547, 26 (2012). doi:10.1051/00046361/201219735 arXiv:1206.6502 ADSView ArticleGoogle Scholar
 Radice, D, Rezzolla, L: Universality and intermittency in relativistic turbulent flows of a hot plasma. Astrophys. J. 766, 10 (2013). doi:10.1088/20418205/766/1/L10 arXiv:1209.2936 ADSView ArticleGoogle Scholar
 Radice, D, Rezzolla, L, Galeazzi, F: Beyond secondorder convergence in simulations of binary neutron stars in full generalrelativity. Mon. Not. R. Astron. Soc. Lett. 437, 4650 (2014). doi:10.1093/mnrasl/slt137 arXiv:1306.6052 ADSView ArticleGoogle Scholar
 Radice, D, Rezzolla, L, Galeazzi, F: Highorder fully generalrelativistic hydrodynamics: new approaches and tests. Class. Quantum Gravity 31(7), 075012 (2014). doi:10.1088/02649381/31/7/075012 arXiv:1312.5004 ADSMATHView ArticleGoogle Scholar
 Rezzolla, L, Zanotti, O: Relativistic Hydrodynamics. Oxford University Press, Oxford (2013). doi:10.1093/acprof:oso/9780198528906.001.0001 MATHView ArticleGoogle Scholar
 Rezzolla, L, Zhidenko, A: New parametrization for spherically symmetric black holes in metric theories of gravity. Phys. Rev. D 90(8), 084009 (2014). doi:10.1103/PhysRevD.90.084009 arXiv:1407.3086 ADSView ArticleGoogle Scholar
 Sądowski, A: Thin accretion discs are stabilized by a strong magnetic field. Mon. Not. R. Astron. Soc. 459, 43974407 (2016). doi:10.1093/mnras/stw913 ADSView ArticleGoogle Scholar
 Sądowski, A, Narayan, R, Tchekhovskoy, A, Zhu, Y: Semiimplicit scheme for treating radiation under M1 closure in general relativistic conservative fluid dynamics codes. Mon. Not. R. Astron. Soc. 429, 35333550 (2013). doi:10.1093/mnras/sts632 arXiv:1212.5050 ADSView ArticleGoogle Scholar
 Shiokawa, H, Dolence, JC, Gammie, CF, Noble, SC: Global general relativistic magnetohydrodynamic simulations of black hole accretion flows: a convergence study. Astrophys. J. 744, 187 (2012). doi:10.1088/0004637X/744/2/187 arXiv:1111.0396 ADSView ArticleGoogle Scholar
 Sorathia, KA, Reynolds, CS, Stone, JM, Beckwith, K: Global simulations of accretion disks. I. Convergence and comparisons with local models. Astrophys. J. 749, 189 (2012). doi:10.1088/0004637X/749/2/189 arXiv:1106.4019 ADSView ArticleGoogle Scholar
 Spiteri, RJ, Ruuth, SJ: A new class of optimal highorder strongstabilitypreserving time discretization methods. SIAM J. Numer. Anal. 40(2), 469491 (2002) MathSciNetMATHView ArticleGoogle Scholar
 Squire, W, Trapp, G: Using complex variables to estimate derivatives of real functions. SIAM Rev. 40, 110112 (1998). doi:10.1137/S003614459631241X ADSMathSciNetMATHView ArticleGoogle Scholar
 Stawarz, Ł, Aharonian, F, Kataoka, J, Ostrowski, M, Siemiginowska, A, Sikora, M: Dynamics and highenergy emission of the flaring HST1 knot in the M 87 jet. Mon. Not. R. Astron. Soc. 370, 981992 (2006). doi:10.1111/j.13652966.2006.10525.x arXiv:astroph/0602220 ADSView ArticleGoogle Scholar
 Suresh, A, Huynh, HT: Accurate monotonicitypreserving schemes with RungeKutta time stepping. J. Comput. Phys. 136(1), 8399 (1997). doi:10.1006/jcph.1997.5745 ADSMathSciNetMATHView ArticleGoogle Scholar
 Takahashi, HR, Ohsuga, K, Kawashima, T, Sekiguchi, Y: Formation of overheated regions and truncated disks around black holes: threedimensional general relativistic radiationmagnetohydrodynamics simulations. Astrophys. J. 826, 23 (2016). doi:10.3847/0004637X/826/1/23 arXiv:1605.04992 ADSView ArticleGoogle Scholar
 Tchekhovskoy, A, Metzger, BD, Giannios, D, Kelley, LZ: Swift J1644+57 gone MAD: the case for dynamically important magnetic flux threading the black hole in a jetted tidal disruption event. Mon. Not. R. Astron. Soc. 437, 27442760 (2014). doi:10.1093/mnras/stt2085 arXiv:1301.1982 ADSView ArticleGoogle Scholar
 Toro, EF: Riemann Solvers and Numerical Methods for Fluid Dynamics. Springer, Berlin (1999) MATHView ArticleGoogle Scholar
 Toth, G: The div b=0 constraint in shockcapturing magnetohydrodynamics codes. J. Comput. Phys. 161, 605652 (2000). doi:10.1006/jcph.2000.6519 ADSMathSciNetMATHView ArticleGoogle Scholar
 van der Holst, B, Keppens, R, Meliani, Z: A multidimensional gridadaptive relativistic magnetofluid code. Comput. Phys. Commun. 179, 617627 (2008). doi:10.1016/j.cpc.2008.05.005 arXiv:0807.0713 ADSMathSciNetMATHView ArticleGoogle Scholar
 Villiers, JPD, Hawley, JF: Global general relativistic magnetohydrodynamic simulations of accretion tori. Astrophys. J. 592, 1060 (2003) arXiv:astroph/0210518 ADSView ArticleGoogle Scholar
 Vincent, FH, Paumard, T, Gourgoulhon, E, Perrin, G: GYOTO: a new general relativistic raytracing code. Class. Quantum Gravity 28(22), 225011 (2011). doi:10.1088/02649381/28/22/225011 arXiv:1109.4769 ADSMathSciNetMATHView ArticleGoogle Scholar
 Weinberg, S: Gravitation and Cosmology: Principles and Applications of the General Theory of Relativity. Wiley, New York (1972) Google Scholar
 White, CJ, Stone, JM: GRMHD in Athena++ using advanced Riemann slvers and staggeredmesh constrained transport. ArXiv eprints (2015). arXiv:1511.00943
 White, CJ, Stone, JM, Gammie, CF: An extension of the Athena++ code framework for GRMHD based on advanced Riemann solvers and staggeredmesh constrained transport. Astrophys. J. Suppl. Ser. 225, 22 (2016). doi:10.3847/00670049/225/2/22 arXiv:1511.00943 ADSView ArticleGoogle Scholar
 York, JW: Kinematics and dynamics of general relativity. In: Smarr, LL (ed.) Sources of Gravitational Radiation, pp. 83126. Cambridge University Press, Cambridge (1979) Google Scholar
 Younsi, Z, et al.: (2017) in prepratation Google Scholar
 Younsi, Z, Wu, K: Variations in emission from episodic plasmoid ejecta around black holes. Mon. Not. R. Astron. Soc. 454, 32833298 (2015). doi:10.1093/mnras/stv2203 arXiv:1510.01700 ADSView ArticleGoogle Scholar
 Younsi, Z, Wu, K, Fuerst, SV: General relativistic radiative transfer: formulation and emission from structured tori around black holes. Astron. Astrophys. 545, 13 (2012). doi:10.1051/00046361/201219599 arXiv:1207.4234 ADSView ArticleGoogle Scholar
 Younsi, Z, Zhidenko, A, Rezzolla, L, Konoplya, R, Mizuno, Y: New method for shadow calculations: application to parametrized axisymmetric black holes. Phys. Rev. D 94(8), 084025 (2016). doi:10.1103/PhysRevD.94.084025 arXiv:1607.05767 ADSView ArticleGoogle Scholar
 Yuan, F, Narayan, R: Hot accretion flows around black holes. Annu. Rev. Astron. Astrophys. 52, 529588 (2014). doi:10.1146/annurevastro082812141003. arXiv:1401.0586 ADSView ArticleGoogle Scholar
 Yuan, F, Quataert, E, Narayan, R: Nonthermal electrons in radiatively inefficient accretion flow models of Sagittarius A*. Astrophys. J. 598, 301312 (2003). doi:10.1086/378716 arXiv:astroph/0304125 ADSView ArticleGoogle Scholar
 Zanotti, O, Dumbser, M: A high order special relativistic hydrodynamic and magnetohydrodynamic code with spacetime adaptive mesh refinement. Comput. Phys. Commun. 188, 110127 (2015). doi:10.1016/j.cpc.2014.11.015 arXiv:1312.7784 ADSMathSciNetMATHView ArticleGoogle Scholar
 Zanotti, O, Fambri, F, Dumbser, M: Solving the relativistic magnetohydrodynamics equations with ADER discontinuous Galerkin methods, a posteriori subcell limiting and adaptive mesh refinement. Mon. Not. R. Astron. Soc. 452, 30103029 (2015). doi:10.1093/mnras/stv1510 arXiv:1504.07458 ADSView ArticleGoogle Scholar
 Zanotti, O, Rezzolla, L, Font, JA: Quasiperiodic accretion and gravitational waves from oscillating ‘toroidal neutron stars’ around a Schwarzschild black hole. Mon. Not. R. Astron. Soc. 341, 832848 (2003) ADSView ArticleGoogle Scholar
 Zhang, W, MacFadyen, AIR: A relativistic adaptive mesh refinement hydrodynamics code. Astrophys. J. Suppl. Ser. 164, 255 (2006) ADSView ArticleGoogle Scholar
 Zrake, J, MacFadyen, AI: Spectral and intermittency properties of relativistic turbulence. Astrophys. J. 763, 12 (2013). doi:10.1088/20418205/763/1/L12 arXiv:1210.4066 ADSView ArticleGoogle Scholar