On the reliability of Nbody simulations
 Tjarda Boekholt^{1}Email author and
 Simon Portegies Zwart^{1}
DOI: 10.1186/s4066801400053
© Boekholt and Portegies Zwart; licensee Springer 2015
Received: 23 July 2014
Accepted: 19 November 2014
Published: 28 March 2015
Abstract
The general consensus in the Nbody community is that statistical results of an ensemble of collisional Nbody simulations are accurate, even though individual simulations are not. A way to test this hypothesis is to make a direct comparison of an ensemble of solutions obtained by conventional methods with an ensemble of true solutions. In order to make this possible, we wrote an Nbody code called Brutus, that uses arbitraryprecision arithmetic. In combination with the BulirschStoer method, Brutus is able to obtain converged solutions, which are true up to a specified number of digits.
We perform simulations of democratic 3body systems, where after a sequence of resonances and ejections, a final configuration is reached consisting of a permanent binary and an escaping star. We do this with conventional doubleprecision methods, and with Brutus; both have the same set of initial conditions and initial realisations. The ensemble of solutions from the conventional simulations is compared directly to that of the converged simulations, both as an ensemble and on an individual basis to determine the distribution of the errors.
We find that on average at least half of the conventional simulations diverge from the converged solution, such that the two solutions are microscopically incomparable. For the solutions which have not diverged significantly, we observe that if the integrator has a bias in energy and angular momentum, this propagates to a bias in the statistical properties of the binaries. In the case when the conventional solution has diverged onto an entirely different trajectory in phasespace, we find that the errors are centred around zero and symmetric; the error due to divergence is unbiased, as long as the timestep parameter, \(\eta\le2^{5}\) and when simulations which violate energy conservation by more than 10% are excluded. For resonant 3body interactions, we conclude that the statistical results of an ensemble of conventional solutions are indeed accurate.
Keywords
methods: numerical methods: Nbody simulations stars: dynamics binaries: formation1 Introduction
Analytical solutions to the Nbody problem are known for \(N=2\), which are the familiar conic sections. Also, for several systems possessing symmetries, analytical solutions have been found, for example the equilateral triangle (Lagrange 1772). For a more general initial configuration, solutions have to be obtained by means of numerical integration. Given an initial Nbody realisation, one can calculate all mutual forces and subsequently the net acceleration of each particle. Different integration methods exist which take the accelerations, and update the positions and velocities to a time \(t+\Delta t\), with Δt the timestep size. This process is repeated until the end time is reached.
Miller (1964) recognised that obtaining the solution to an Nbody problem by numerical integration is difficult. This is caused by exponential divergence. Consider a certain Nbody problem, i.e. N pointparticles, each with a given mass, position and velocity. This system evolves with time in a definite and unique way. If one goes back to the initial state and slightly perturbs only one coordinate of a single particle, the perturbed Nbody problem will also have a definite and unique but different solution than the original one. When the two solutions are compared as a function of time, it is observed that differences can grow exponentially (Miller 1964; Dejonghe and Hut 1986; Goodman et al. 1993; Hut and Heggie 2002). If the initial perturbation is due to a numerical error, the calculated solution will also diverge away from the true solution.
Several authors have estimated the timescale of this divergence (Goodman et al. 1993; Hut and Heggie 2002), and arrived at an efolding timescale of the order a dynamical, crossing time. Simulation times of interest are typically much longer than a crossing time and therefore staying close to the true solution is numerically challenging.
If the result of a direct Nbody simulation of for example a star cluster, has diverged away from the true solution, the result may well be meaningless (Goodman et al. 1993). The general consensus however, is that statistically the results are representative for the true solution to the Nbody problem (Smith 1979; Heggie 1991; Goodman et al. 1993). The underlying idea is that the statistics of an ensemble of Nbody simulations are representative for the true statistics, obtained by an ensemble of true solutions, with the same set of initial conditions. We regard this the hypothesis we want to test.
One way to test this hypothesis is to directly compare statistics obtained by conventional methods, with the statistics obtained from an ensemble of true solutions. To obtain true solutions, we wrote an Nbody code which can solve the Nbody problem to arbitrary precision.
Such a code can be realised if the different sources of error are controlled. The error has contributions from the time discretisation of the integrator and the roundoff due to the limited precision of the computer (Zadunaisky 1979). Another possible source of error is in the initial conditions, for example the configuration of the Solar System is only approximately known (Ito and Tanikawa 2002). However, if the initial condition is a random realisation of a distribution function, this is less often a problem. Using the BulirschStoer method (Bulirsch and Stoer 1964; Gragg 1965), the discretisation error can be controlled to stay within a specified tolerance. Using arbitraryprecision arithmetic instead of conventional doubleprecision or singleprecision, the roundoff error can be reduced by increasing the number of digits.
We obtain converged solutions to the Nbody problem by decreasing the BulirschStoer tolerance and increasing the number of digits systematically. We define a converged solution in our experiments as a solution for which the first specified number of decimal places of every phasespace coordinate in our final configuration in the Nbody experiment becomes independent of the length of the mantissa and the BulirschStoer tolerance. We explain the method of convergence in Section 2 and we give examples of the procedure in Section 3.
Using this new, brute force Nbody code which we call Brutus, we test the reliability of Nbody simulations by a controlled numerical experiment which we describe in Section 4. In this experiment we perform a series of resonant 3body simulations, where the term resonant implies a phase or multiple phases during the interaction where the stars are more or less equidistant (Hut and Bahcall 1983). These phases are intermingled by ejections, where a binary and single star are clearly separated. We perform the simulations with conventional doubleprecision, and with arbitraryprecision to reach the converged solution. In Section 5, the solutions are compared individually to investigate the distribution of the errors. We also compare the global statistical distributions using twosample KolmogorovSmirnov tests (Kolmogorov 1933; Smirnov 1948).
2 Methods
2.1 The benchmark integrator
To test how inaccurate we are allowed to integrate while still obtaining accurate statistics (Smith 1979; Quinlan and Tremaine 1992) we vary the timestep parameter η, to obtain statistics from conventional simulations with different precision.
2.2 The Brutus Nbody code
The results obtained with the benchmark integrator are compared to those obtained with Brutus, which uses an arbitraryprecision library.^{1} With this library we can specify the number of bits, \(L_{\mathrm{w}}\), used to store the mantissa, while the exponent has a fixed wordlength. The length of the mantissa can be specified and increased, with the aim of controlling the roundoff error.
The integration of the equations of motion is realised using the Verletleapfrog scheme (Verlet 1967). The timestep is shared among all particles, but varies for every step according to equation (1).
To control the discretisation error, we implemented the BulirschStoer (BS) method, which uses iterative integration and polynomial extrapolation to infinitesimal timestep size (Bulirsch and Stoer 1964; Gragg 1965). An integration step is accepted, when two subsequent BS iterations have converged to below the BS tolerance level, ϵ.
2.3 Method of convergence
We consider the solutions A and B to be converged when \(\delta _{\mathrm{A},\mathrm{B}}<10^{p}\) at all times during the simulation. Note that converged in this case means convergence of the total solution, contrary to convergence per integration step as in the previous section. This criterion for convergence is roughly equivalent to comparing the first p decimal places of the positions and velocities for all N stars, in two subsequent calculations A, B. In most of our experiments we adopt \(p=3\), i.e. all coordinates have to converge to about three decimal places or more. We perform a subset of simulations with \(p=15\) to investigate the effect of small errors (see Section 5.4.3).
Each simulation starts by specifying the initial positions and velocities of N stars in doubleprecision (see Section 4). The simulation is carried out with the parameter set \((\epsilon, L_{\mathrm{w}})\). We start each simulation with \(\epsilon=10^{6} \) and \(L_{\mathrm{w}}=56\mbox{ bits}\). This corresponds to a level of accuracy similar to what we reach with the conventional Hermite integrator. After this simulation, we increase \(L_{\mathrm{w}}\), for example to 72 bits (∼22 decimal places), redo the simulation and calculate \(\delta^{2}\). We repeat this procedure until \(\delta< 10^{p}\). When this is achieved, we have obtained a solution in which the roundoff error is below a specified number of decimal places for this particular value of ϵ.
We now reduce the tolerance parameter ϵ, for example by a factor of 100, and repeat the procedure of increasing \(L_{\mathrm{w}}\). This series will again lead to a converged solution, but this time it is obtained using a smaller ϵ, and is likely to be different than the previous converged solution. We continue decreasing the value of ϵ by factors of 100 and repeat the procedure, until two subsequent iterations in ϵ lead to a converged solution with a value of \(\delta< 10^{p}\). By this time we are assured of having a solution to the gravitational Nbody problem, that is accurate up to at least p decimal places.
3 Validation and performance
3.1 The Pythagorean problem
To show that our method works, we adopt the Pythagorean 3body system (Burrau 1913). Previous numerical studies have shown that this system dissolves into a binary and an escaper (Szebehely and Peters 1967; Aarseth et al. 1994). After many complex, close encounters the dissolution happens at about \(t=60\) time units (Dejonghe and Hut 1986), or about 16 crossing times.
In the following step, we repeat the calculation with a precision of (\(\epsilon=10^{6}\), \(L_{\mathrm{w}}=56\mbox{ bits}\)), and compare the result with the calculation using (\(\epsilon=10^{4}\), \(L_{\mathrm{w}}=48\mbox{ bits}\)). The comparison is represented by the orange curves (second from above) in Figure 1. The overall behaviour of δ is similar, but the system diverges at a later time due to a higher initial precision.
We continue the iterative procedure until a converged solution has been obtained. In the first three panels of Figure 1, it can be seen that subsequent simulations with higher precision shift the curve to lower values of δ. Superposed on the steady growth of the error are sharp spikes, where the error grows by several orders of magnitude, after which the error restores again (Miller 1964). These spikes are dominated by errors in the velocity, as can be deduced by comparing the magnitude of the spikes in position and velocityspace. Eccentric binaries which are out of phase when comparing two solutions cause large, periodic errors in the velocity. We finish the procedure when a solution is obtained for which the criterion for convergence is fulfilled, considering the magnitude of the error between the sharp spikes (bottom, black curves).
In the bottom right panel of Figure 1, we compare solutions obtained by the Hermite integrator to the converged solution. The different curves belong to different timestep parameters; η = 2^{−3}, 2^{−5}, 2^{−7} up to 2^{−13}. Note that for a timestep parameter \(\eta< 2^{9}\), the curve is not shifted to lower values of δ, but even increases again. At this point roundoff error becomes important, making the solution less accurate. The final close encounter in the Pythagorean problem occurs around 60 time units, after which a permanent binary and an escaper are formed. The Hermite integrator is able to accurately reproduce the evolution up to this point, but not subsequently, because δ has increased to values of order unity or higher. This can be explained by a small error in the final close encounter between all three stars, such that the direction of the escaper is slightly different.
To obtain the converged solution up to the first three decimal places, a tolerance of 10^{−14} and a wordlength of 88 bits were needed. The simulation was about twice as slow compared to the Hermite simulation with \(\eta=2^{9}\). The Hermite simulation, however, had a slightly different solution and a final, relative energy conservation of 10^{−8}. Decreasing the value of η will improve the level of energy conservation, but due to roundoff error δ will not decrease.
3.2 The equilateral triangle
As a second test case, we adopt the 3body equilateral triangle as an initial condition (Lagrange 1772). In the exact solution this configuration remains intact, but small perturbations, such as produced by numerical errors, quickly cause the triangle to fall apart. For this problem, we also have a source of error in the initial conditions. Whereas the Pythagorean problem can be set up using integers, the initial condition for the equilateral triangle contains irrational numbers. To control the error in the initial condition, we calculate the initial coordinates with the same wordlength as used for the simulation.
The red dashed curves in the same diagram in Figure 2 give the results of the fourthorder Hermite, which are compared with the most precise Brutus simulation (with \(\epsilon=10^{80}\), \(L_{\mathrm{w}}=352\mbox{ bits}\)). The timestep parameter η = 10^{−1}, 10^{−2}, 10^{−3} and 10^{−4} for subsequent curves. The Hermite integrations show a similar behaviour as the Brutus results, which could imply that the rate of divergence is a physical property of the configuration, rather than a property of the integrator.
In the right panel of Figure 2 we show the duration for which the triangular configuration remains intact as a function of BS tolerance. For this experiment we halt the simulation when the distance between any two particles has increased or decreased by 10%, after which the triangle falls apart quickly. This diagram also illustrates the linear relation between accuracy and time in this system, which is caused by the constant number of digits being lost during every unit of time. The small scatter is due to the discrete times at which we check the triangular configuration. The solid, blue line is a fit to the data and its slope is \(0.52(3)\), which is equivalent to a loss of \(1.9(1)\) digits per cycle.
3.3 A Plummer distribution with \(N=16\)
The right panel in Figure 3 shows the energy conservation (black bullets, solid line) and the normalized phase space distance (red triangles, dashed line) versus ϵ. Energy conservation is proportional to ϵ, but the solutions only start to converge for \(\epsilon<10^{34}\). More generally, even if conserved quantities like total energy are conserved to machineprecision or better, it is not guaranteed that the solution itself has converged.
The highest precision Brutus simulation in this example (\(\epsilon=10^{50}\), \(L_{\mathrm{w}}=232\mbox{ bits}\)), took about a day of wallclock time, which is about 7,000 times slower than a simulation with Hermite using \(\eta=2^{9}\).
3.4 Scaling of the wallclock time
The use of arbitraryprecision arithmetic dramatically increases the CPU time of Nbody simulations. Also the BS method, which performs integration steps iteratively, makes an integration scheme more expensive by at least a factor two or more. To investigate for example how feasible it would be to run a converged Nbody simulation for 10^{3} stars through core collapse, we perform a scaling test in which we vary the number of particles and the precision, ϵ and \(L_{\mathrm{w}}\).
For \(N>32\), it becomes efficient to parallellise the code. Our version implements iparallellisation (Portegies Zwart et al. 2008) in the calculation of the accelerations. In the top right panel of Figure 4, we plot the speedup, S, against the number of cores. For \(N = 1\mbox{,}024\), we obtain a speed up of a factor 30 using 64 cores.
In the lower panels of Figure 4 we present the scaling of the wallclock time with BS tolerance and wordlength. To measure the dependence on BS tolerance, we simulated a 16body cluster for 1 Hénon time unit. We varied the BS tolerance while keeping the wordlength fixed at \(L_{\mathrm{w}} = 1\mbox{,}024\mbox{ bits}\). The relation obtained converges to \(t_{\mathrm{CPU}} \propto\epsilon^{0.032}\). A similar experiment was performed to measure the dependence on wordlength. This time we fixed the BS tolerance at \(\epsilon=10^{10}\) and varied the wordlength. For \(L_{\mathrm{w}} < 1\mbox{,}024\), the relation can be estimated as \(t_{\mathrm{CPU}} \propto L_{\mathrm{w}}^{0.33}\), while for \(L_{\mathrm{w}} > 1\mbox{,}024\), \(t_{\mathrm{CPU}} \propto L_{\mathrm{w}}\). This transition depends on the internal workings of the arbitraryprecision library which we will not discuss here.
Using a very long wordlength of 4,096 bits, i.e. ∼10^{3} digits, results in a slowdown of a factor \(f_{\mathrm{s}} \sim16\) compared to 64 bits. But for some simulations a BS tolerance smaller than 10^{−50} can easily be required to reach convergence, and this will result in a slowdown of a factor \(f_{\mathrm{s}} > 100\). The very small BS tolerance is often the main cause for the slowdown of the simulations, instead of the increased wordlength.
For direct Nbody codes, the time for integrating up to core collapse usually scales as \(\mathcal{O}(N^{3})\). Using the analysis above, we estimate that the time for converged core collapse simulations scales approximately exponentially. This is effectively caused by the exponential divergence.
4 Precision of statistical results: experimental setup
In the previous section we demonstrated that it is possible to obtain a converged solution for a particular initial condition. We have also shown that a solution obtained by Hermite diverges from the converged solution, even up to the point that the microscopic solution given by Hermite is beyond recognition. We now perform a statistical study, to examine the hypothesis that doubleprecision Nbody simulations produce statistically indistinguishable results, from those obtained from an ensemble of converged solutions with the same set of initial conditions. Because it is computationally expensive to reach convergence, we start investigating the hypothesis above by exploring the accuracy of 3body statistics.
 1.
Plummer distribution equal mass.
 2.
Plummer distribution with masses 1:2:4.
 3.
Plummer distribution equal mass with zero velocities.
 4.
Plummer distribution with masses 1:2:4 and zero velocities.
In the case of the cold initial conditions, the systems start democratically, i.e. the minimal distance between each pair of particles is greater than \(N^{1}\). We reject initial conditions in which this criterion is not satisfied. This is to prevent initial realisations where two stars which are very near, fall to each other radially causing very long wallclock times for the integration. When starting with a democratic configuration, there will also be an initial close triple encounter (Aarseth et al. 1994), which is hard to integrate accurately and is therefore a good test. A total of 10,000 random realisations are generated for each set of initial conditions and can be found in Additional files 1, 2, 3 and 4.
 1.
escaper has a positive energy, \(E >0\),
 2.
is a certain distance away from the center of mass, \(r > 2 r_{\mathrm{virial}}\),
 3.
is moving away from the center of mass, \(r \cdot v > 0\).
There may be situations in which a star is ejected without actually escaping from the binary. After a long excursion the star turns around and once again engages the binary in a 3body resonance (Hut and Bahcall 1983). Because these systems need to be integrated for a longer time, they also require higher precision to reach convergence, which takes a long time to integrate [see also Hut (1993)]. To deal with this issue, we perform the simulations iteratively by increasing the final integration time \(t_{\mathrm{end}}\). Starting with \(t_{\mathrm{end}}=50\) Hénon time units, we evolve every system and detect those that are dissolved. Then we increase \(t_{\mathrm{end}}\) to 100, 150, 200, etc., but only for those systems which have not yet dissolved. A complete ensemble of solutions is obtained up to \(t_{\mathrm{end}} \sim 500\), or equivalently ∼180 crossing times where the crossing time has a value of \(2\sqrt{2}\) in Hénon units (Hénon 1971; Heggie and Mathieu 1986). Systems which take a longer time to integrate are not taken into account in this research. The fraction of longlived systems is however a statistic we measure. We gathered the final, converged configurations in Additional files 1, 2, 3 and 4.
Each initial realisation is run with the Hermite code, using standard doubleprecision, and with Brutus, using arbitraryprecision until a converged solution is obtained. At the end of each simulation, we investigate the nature of the binary and the escaper. In addition to the BS tolerance, wordlength, CPU time and dissolution time, we record the mass, speed and escape direction of the escaping single star, and the semimajor axis, binding energy and eccentricity of the binary. In this way, we obtain statistics for \(N=3\) generated by a conventional Nbody solver and by Brutus.
5 Results
Before we perform a detailed comparison between results obtained by Hermite and Brutus, we first compare the Brutus results with analytical distributions from the literature in order to relate to previous studies. We compare Hermite and Brutus on a global level by performing twosample KolmogorovSmirnov tests (Kolmogorov 1933; Smirnov 1948) to see whether global distributions are statistically indistinguishable. We also compare the distribution of lifetimes of triples to see whether precision influences the stability and we measure the typical CPU time and BS tolerance needed to obtain a converged solution. After this, we compare Hermite and Brutus per individual system, with the aim of investigating the nature of the differences of every individual outcome. Finally, we define categories which classify a conventional simulation as a preservation or exchange, depending on whether the identity of the escaping star is consistent between Hermite and Brutus.
5.1 Brutus versus analytical distributions
Fitted powerlaw indices for the velocity and kinetic energy distributions of the escaping stars and for the binding energy distribution of the binary stars
α  β  

Velocity  
Plummer equal mass  2.5 ± 0.09  6.7 ± 1.02 
Plummer mass ratio  3.8 ± 0.16  4.4 ± 0.43 
Cold Plummer equal mass  2.6 ± 0.19  3.8 ± 0.28 
Cold Plummer mass ratio  3.4 ± 0.45  3.4 ± 0.19 
Kinetic energy  
Plummer equal mass  0.9 ± 0.02  1.8 ± 0.04 
Plummer mass ratio  0.8 ± 0.02  1.6 ± 0.04 
Cold Plummer equal mass  0.99 ± 0.02  1.3 ± 0.03 
Cold Plummer mass ratio  0.98 ± 0.03  1.2 ± 0.02 
Binding energy  
Plummer equal mass  4.31 ± 0.13  
Plummer mass ratio  5.12 ± 0.32  
Cold Plummer equal mass  2.37 ± 0.11  
Cold Plummer mass ratio  2.38 ± 0.12 
The empirical distributions obtained by Brutus are in qualitative agreement with the analytical estimates present in the literature (Heggie 1975; Monaghan 1976; Valtonen and Karttunen 2006). Slight variations are present due to the dependence on total angular momentum, a limited statistical sampling and assumptions made in the derivation of the analytical distributions. Nevertheless, a similar qualitative agreement has been obtained between the analytical distributions discussed above and empirical distributions from an ensemble of conventional numerical solutions, e.g. not converged (Valtonen and Karttunen 2006, Chapters 78 and references therein). The question remains to what extend conventional and converged solutions agree quantitatively.
5.2 Brutus versus Hermite: global comparison
A quantitative way to compare global distributions is by performing twosample KolmogorovSmirnov tests (KS tests) (Kolmogorov 1933; Smirnov 1948). The KS test gives the likelihood that two samples are drawn from the same distribution, quantified by the value called p. When the pvalue is below five percent, the distributions are considered to be significantly different.
5.3 Lifetime of triple systems
The initially cold systems dissolve faster than the initially virialised systems. This is somewhat expected due to the close triple encounter resulting from the initial cold collapse: the rate of energy exchange can be very high for these encounters (Johnstone and Rucinski 1991). After ∼180 crossing times, about 40% of the systems which started with an equalmass Plummer initial configuration, are undissolved, compared to about 10% for the cold Plummer with different masses. Systems which include stars with different masses dissolve faster than their equal mass counterparts. Energy equipartition tends to cause the lightest particle to quickly reach the escape velocity.
In Figure 7, the grey curves through the data points represent the interpolated Hermite results. Even though Hermite and Brutus use different algorithms and precisions to solve the equations of motion, we find that the lifetime of an unstable triple is statistically indistinguishable between converged Brutus and nonconverged Hermite solutions (but see also Section 6.3).
We were able to obtain a complete ensemble of systems dissolving within ∼180 crossing times. Simulations which take longer than this are not taken into account in this experiment. The fraction of longlived systems as obtained by Hermite and Brutus are consistent. For our purpose of comparing results from conventional integrators with the converged solution, integrating up to ∼180 crossing times is sufficient, in the sense that there is enough time for conventional solutions to diverge from the true solution (see Section 5.4.1). Including the longlived triple systems may however influence the statistical distributions and biases on the long term.
5.4 Brutus versus Hermite: individual comparison
Data points on the diagonal represent accurate solutions, whereas the scatter around it represents inaccurate Hermite solutions. The diagonal is present in each panel and extends throughout the range of possible outcomes. The width of the diagonal is very narrow. When the normalized phasespace distance between the Hermite and Brutus solution \(\delta< 10^{1}\), then the coordinates are accurate enough to produce derived quantities accurate to at least one decimal place and Hermite and Brutus will give similar results. Once \(\delta> 10^{1}\), the solution has diverged to a different trajectory in phasespace leading to a different outcome. This outcome could in principle be any of the possible outcomes as can be derived from the amount of scatter in the Hermite solutions at a fixed Brutus solution.
In the scatter plot of the dissolution time, we observe that for small times (\(t < 10\)), Hermite and Brutus agree on the solution in the sense that the data points lie on the diagonal. Systems which dissolve after a short time don’t have sufficient time to accumulate enough error to diverge to another trajectory in phasespace. Once however this level of divergence is reached, the scatter immediately covers the entire, available outcome space. This randomisation is also observed in the other panels.
5.4.1 The fraction of accurate solutions
5.4.2 The error distribution
The width of the error distributions converge to a nonzero value. This can be understood because with decreasing timestep, roundoff errors will become more important so that the standard deviation of the errors will never reach zero. For the dissolution time, the width of the error distribution for the smallest timestep parameter adopted, varies from 60 to 100 crossing times. For the eccentricities the width is on average ∼0.2. For the semimajor axis the width approaches ∼0.05 (in Hénon units). In the case of the semimajor axis, the data representing the cold collapse simulations behave differently, because the width is much larger than the width for the data representing the initially virialised systems.
5.4.3 Symmetry of the error distribution
We observe that the majority of errors are larger than ∼10^{−3} and within the statistical error, the positive and negative errors have a similar distribution. For the smallest errors however, we observe an asymmetry in the sense that there are more negative, small errors. The magnitude of the error where this excess occurs is determined by the precision of the integration. For the smallest η, the excess is below doubleprecision and thus not observable anymore (see Section 6.2 for more explanation).
5.5 Escaper identity

Accurate: The coordinates are accurate, up to at least two digits.

Preservation: The coordinates are inaccurate, but same star escapes.

Exchange: Different star escapes.
The cold collapse with equalmass stars is the hardest problem to integrate as the accurate fraction is of comparable magnitude as the preservation and exchange fractions. The accurate fraction generally remains dominant, with a final fraction varying from about 0.4 for the equalmass cold Plummer to about 0.7 for the Plummer with different masses. For the lesser precision (\(\eta= 2^{3}\), bottom row in the figure), the accurate fractions decrease to below 0.2.
In the panels in Figure 13, which include the data for the systems with different masses, preservation is more common than exchange. This can be understood, because due to energy equipartition, the lightest particle will be more likely to escape and therefore the identity is more often correct than in the equal mass case. For the equal mass case, the fraction of preservation and exchange is comparable, except in the case of the equalmass cold Plummer with the low precision (\(\eta= 2^{3}\), the bottom row). If we regard the identity of the escaping star to be completely random once the solution has become inaccurate, we would expect the fraction of exchange to be twice the fraction of preservation. This is roughly what we observe in the equal mass cold collapse case with low precision. Because of the low precision and the initial close encounter, solutions will diverge very quickly. In the panel with the higher precision this trend is not observed because the solutions are less randomised. The preservation category includes solutions which slightly differ from the converged solution only in the escape angle of the escaper. Also the longlived triples are not taken into account here, which will alter these fractions.
6 Discussion
6.1 Energy conservation
Timereversible, symplectic integrators should in principle conserve energy to a better level than nonsymplectic integrators, since there is no drift present in the energy error. Therefore, by using a symplectic integrator, the number of simulations with large energy error could be reduced. Using a Leapfrog integrator with constant timesteps, we tested this assumption and we find that for resonant 3body interactions, it is challenging to obtain accurate solutions. The main reason is that, contrary to regular systems like, for example, the Solar System, resonant 3body interactions often include very close encounters, which need a very small timestep size to be resolved accurately. This is especially the case for the initially cold systems. Adopting such a small timestep size for the whole simulation, will increase the wallclock time to that of Brutus or beyond.
6.2 Asymmetry at small errors
We also vary the integration method because different methods produce different (biased) error distributions in energy and angular momentum. We use a standard Leapfrog integrator, a standard Hermite integrator and a Hermite integrator which uses the \(P(EC)^{n}\) method (we adopted \(n=3\)) (Kokubo et al. 1998). This last method adds an iterative procedure to the algorithm to improve the predictions and corrections, which improves the timesymmetry. For each method we implement a shared, adaptive timestep criterion as in equation (1), with a timestep parameter \(\eta= 2^{7}\). As a consequence they will not be timesymmetric nor symplectic.
We first look at the error distributions in the total energy and angular momentum. We observe that none of them are symmetric, in the sense that the positive and negative errors have identical distributions, except for the angular momentum in the Leapfrog simulations. The Leapfrog solutions tend to gain energy, whereas the standard Hermite loses energy. The Hermite with the \(P(EC)^{n}\) method produces both positive and negative errors in the energy, but not in a symmetric manner.
If we compare the resulting error distributions to the actual error distributions, we find that the approximated error distribution is positioned at the asymmetry in the empirical error distribution. This is most clearly seen for the semimajor axis and eccentricity (see Figure 15).
Upon inspection of the velocity data, we observe no asymmetry in the Hermite results. When we measure which fraction of the energy error is reserved for the binary and which fraction for the escaper, we find that in most cases the error propagates to the binary. For the Leapfrog however, the asymmetry is still present.
6.3 Preservation of the macroscopic properties
Valtonen et al. (2004) state that the final statistical distributions forget the specific initial conditions and only depend on globally conserved quantities. This assumption makes predictions which are verified by our experiment. The results show that for a timestep parameter \(\eta< 2^{5}\), the distributions are statistically indistinguishable, even though at least half of the solutions diverged from the converged solution. If however, energy conservation is grossly violated, biases are introduced in the statistics. In our experiment, a maximum level of relative energy conservation of \(\vert \Delta E/E \vert = 0.1\) was sufficient to remove the biases. This is a much milder constraint than the \(\vert \Delta E/E \vert \sim10^{6}\) usually adopted in collisional simulations. Whether 0.1 is also sufficient for systems with more stars, should be verified experimentally. Heggie (1991) for example, finds that the energy of escaping stars in higherN systems, depends sensitively on integration accuracy. The maximum required level of energy conservation should be such that it is below the energy taken away from the cluster by the escaping stars.
The chaoticity of the 3body problem is illustrated by the scatter diagrams in Figure 9. For a certain value of a statistic obtained by Brutus, any other value in the allowed outcome space is reachable for the Hermite integrator. For example, if the converged solution gives an eccentricity for the binary of 0.6, a diverged solution can produce any eccentricity between 0 and 1. Once the solution has diverged from the true solution, it will start a random walk through or near the allowed phasespace until the 3body system has dissolved. We observed that this randomisation happens in such a way that the available outcome space is still completely sampled and that it preserves global statistical distributions.
In Section 4, we discussed that the lifetime of an unstable triple does not depend on the integrator used nor on the accuracy of that integrator. This last point should be interpreted in the sense that when more effort is put into performing simulations with higher precision, that this does not change the global statistics, even though individual solutions will change with precision (see for example the Hermite results in Figure 1). If instead we continue to decrease the precision, there will be a point where biases start to appear. Urminsky (Urminsky 2008) analysed the 3body Sitnikov problem and showed that the precision of the integration influences the average lifetime of triple systems, contrary to our results. The integration times in our experiment however, are much shorter. Obtaining a converged solution for a resonant 3body system for longer than 200 crossing times, is still computationally challenging. Therefore any statistical difference on the long term will not be visible in our experiment.
7 Conclusion
Brutus is an Nbody code that uses the BulirschStoer method to control discretisation errors, and arbitraryprecision arithmetic to control roundoff errors. By using the method of convergence, where we systematically vary the BulirschStoer tolerance parameter and the wordlength, we can obtain a solution for a particular Nbody problem, for which the first p digits in the mantissa are independent of the timestep size and wordlength. We call this solution converged to p decimal places.
Obtaining the converged solution is computationally expensive, mainly because of the exponential divergence of the solution. In some cases, BulirschStoer tolerances of 10^{−100} are needed to reach convergence. We estimate that the time for simulating a star cluster up to core collapse, until convergence, scales approximately exponentially with the number of stars. Simulations with 256 stars however, may be performed within a year of computing time.
The motivation to obtain expensive, converged solutions is to test the assumption that the statistics of an ensemble of approximate solutions, are indistinguishable from the statistics of an ensemble of true solutions. To put this assumption to the test, we have investigated the statistics on the breakup of 3body systems. In our experiment, a bound triple system will eventually dissolve into a binary and an escaping star. Solutions to every initial realisation were obtained using the standard Hermite integrator and using Brutus.
For systems with a long lifetime it is challenging to obtain the converged solution. Due to repeated ejections and resonances, many accurate digits will be lost and so a very small BulirschStoer tolerance is required. Therefore, we have set an integration limit at ∼180 crossing times. For equalmass, virialised systems, ∼40% of the random initial realisations were not dissolved by this time. For the initially cold systems with different masses this was ∼10%. Hermite and Brutus are consistent on the average lifetime of an unstable triple system. However, possible differences on the long term are not visible in this experiment.
When we compare the results on an individual basis, we find that on average about half of the Hermite solutions give accurate results, i.e. at most a 1% relative difference compared to Brutus. For the inaccurate results, the error distribution becomes unbiased and symmetric for a timestep parameter \(\eta\le2^{5}\) and implementing a maximum level of relative energy conservation of \(\vert \Delta E/E\vert < 0.1\).
Once the conventional solution has diverged from the converged solution, it will start a random walk through or near the allowed region in phase space. such that any allowed outcome of a statistic is reachable. This randomisation process completely samples the available outcome space of a statistic and it also preserves the global statistical distributions.
KolmogorovSmirnov tests were performed to compare the global distributions produced by Hermite and Brutus. No significant differences were detected when using the criteria mentioned above for the timestep parameter η and relative energy conservation. This research for the 3body problem supports the assumption that results from conventional Nbody simulations are valid in a statistical sense. We observed however that a bias is introduced for the smallest errors, if the algorithm used to solve the equations of motion, is biased in the conservation of energy and angular momentum. In this research however, this bias did not have an appreciable effect. It is important to see whether this remains true for statistics of higherN systems or systems with a dominant mass. An example of a higherN system where precision might play a role is a young star cluster (without gas) going through the process of cold collapse (Caputo et al. 2014). At the moment of deepest collapse, a fraction of stars will obtain large accelerations, so that a small error in the acceleration can cause large errors in the position and velocity. The rate of divergence can increase up to about 5 digits per Hénon time unit for 128 particles and it increases with N.
Declarations
Acknowledgements
We thank Douglas Heggie, Piet Hut, Michiko Fujii and Guilherme Gonçalves Ferrari for useful discussions and comments on the manuscript. TB would also like to thank Ann Young and Lucie Jíková for carefully reading the manuscript and improving the presentation. The authors also thank the referees for providing useful improvements to our manuscript. This work was supported by the Netherlands Research Council NWO (grants #643.200.503, #639.073.803 and #614.061.608) and by the Netherlands Research School for Astronomy (NOVA). Part of the numerical computations were carried out on the Little Green Machine at Leiden University and on the Lisa cluster at SURFSara in Amsterdam.
Open Access This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.
Authors’ Affiliations
References
 Aarseth, SJ, Anosova, JP, Orlov, VV, Szebehely, VG: Global chaoticity in the Pythagorean threebody problem. Celest. Mech. Dyn. Astron. 58, 116 (1994) View ArticleADSMathSciNetGoogle Scholar
 Aarseth, SJ, Anosova, JP, Orlov, VV, Szebehely, VG: Close triple approaches and escape in the threebody problem. Celest. Mech. Dyn. Astron. 60, 131137 (1994) View ArticleADSMATHMathSciNetGoogle Scholar
 Aarseth, SJ, Henon, M, Wielen, R: A comparison of numerical methods for the study of star cluster dynamics. Astron. Astrophys. 37, 183187 (1974) ADSGoogle Scholar
 Bulirsch, R, Stoer, J: Fehlerabschätzungen und extrapolation mit rationalen funktionen bei verfahren vom richardsontypus. Numer. Math. 6, 413427 (1964) View ArticleMATHMathSciNetGoogle Scholar
 Burrau, C: Numerische Berechnung eines Spezialfalles des Dreikörperproblems. Astron. Nachr. 195, 113 (1913) View ArticleADSGoogle Scholar
 Caputo, DP, de Vries, N, Portegies Zwart, S: On the effects of subvirial initial conditions and the birth temperature of R136. Mon. Not. R. Astron. Soc. 445, 674685 (2014) View ArticleADSGoogle Scholar
 Dejonghe, H, Hut, P: Roundoff sensitivity in the Nbody problem. In: Hut, P, McMillan, SLW (eds.) The Use of Supercomputers in Stellar Dynamics. Lecture Notes in Physics, vol. 267, p. 212. Springer, Berlin (1986) View ArticleGoogle Scholar
 Goodman, J, Heggie, DC, Hut, P: On the exponential instability of Nbody systems. Astrophys. J. 415, 715 (1993) View ArticleADSGoogle Scholar
 Gragg, WB: On extrapolation algorithms for ordinary initial value problems. SIAM J. Numer. Anal. 2, 384403 (1965) ADSMATHMathSciNetGoogle Scholar
 Heggie, DC: Binary evolution in stellar dynamics. Mon. Not. R. Astron. Soc. 173, 729787 (1975) View ArticleADSGoogle Scholar
 Heggie, DC: Chaos in the Nbody problem of stellar dynamics. In: Roeser, S, Bastian, U (eds.) Predictability, Stability, and Chaos in NBody Dynamical Systems, pp. 4762 (1991) View ArticleGoogle Scholar
 Heggie, DC, Mathieu, RD: Standardised units and time scales. In: Hut, P, McMillan, SLW (eds.) The Use of Supercomputers in Stellar Dynamics. Lecture Notes in Physics, vol. 267, p. 233. Springer, Berlin (1986) View ArticleGoogle Scholar
 Hénon, MH: The Monte Carlo method (Papers appear in the Proceedings of IAU Colloquium No. 10 Gravitational NBody Problem (ed. by Myron Lecar), R. Reidel Publ. Co., DordrechtHolland.) Astrophys. Space Sci. 14, 151167 (1971) View ArticleADSGoogle Scholar
 Hut, P: Binarysinglestar scattering. III  Numerical experiments for equalmass hard binaries. Astrophys. J. 403, 256270 (1993) View ArticleADSGoogle Scholar
 Hut, P, Bahcall, JN: Binarysingle star scattering. I  Numerical experiments for equal masses. Astrophys. J. 268, 319341 (1983) View ArticleADSGoogle Scholar
 Hut, P, Heggie, DC: Orbital divergence and relaxation in the gravitational Nbody problem. J. Stat. Phys. 109, 10171025 (2002) View ArticleADSMATHMathSciNetGoogle Scholar
 Ito, T, Tanikawa, K: Longterm integrations and stability of planetary orbits in our Solar system. Mon. Not. R. Astron. Soc. 336, 483500 (2002) View ArticleADSGoogle Scholar
 Johnstone, D, Rucinski, SM: Statistical properties of planar zeroangularmomentum equalmass triple systems. Publ. Astron. Soc. Pac. 103, 359367 (1991) View ArticleADSGoogle Scholar
 Kokubo, E, Yoshinaga, K, Makino, J: On a timesymmetric Hermite integrator for planetary Nbody simulation. Mon. Not. R. Astron. Soc. 297, 10671072 (1998) View ArticleADSGoogle Scholar
 Kolmogorov, A: Sulla determinazione empirica di una legge di distribuzionc. 1st. Ital. Attuari. G. 4, 111 (1933) MATHGoogle Scholar
 Lagrange, JL: Essai sur le Problème des Trois Corps. Prix de l’Académie Royale des Sciences de Paris 6, 292 (1772)
 Makino, J, Aarseth, SJ: On a Hermite integrator with AhmadCohen scheme for gravitational manybody problems. Publ. Astron. Soc. Jpn. 44, 141151 (1992) ADSGoogle Scholar
 Miller, RH: Irreversibility in small stellar dynamical systems. Astrophys. J. 140, 250 (1964) View ArticleADSGoogle Scholar
 Monaghan, JJ: A statistical theory of the disruption of threebody systems. I  Low angular momentum. Mon. Not. R. Astron. Soc. 176, 6372 (1976) View ArticleADSGoogle Scholar
 Newton, I: Philosophiae Naturalis Principia Mathematica (1687)
 Plummer, HC: On the problem of distribution in globular star clusters. Mon. Not. R. Astron. Soc. 71, 460470 (1911) View ArticleADSGoogle Scholar
 Portegies Zwart, S, McMillan, S, Groen, D, Gualandris, A, Sipior, M, Vermin, W: A parallel gravitational Nbody kernel. New Astron. 13, 285295 (2008) View ArticleADSGoogle Scholar
 Portegies Zwart, S, McMillan, S, Pelupessy, I, van Elteren, A: Multiphysics simulations using a hierarchical interchangeable software interface. In: CapuzzoDolcetta, R, Limongi, M, Tornambè, A (eds.) Advances in Computational Astrophysics: Methods, Tools, and Outcome. Astronomical Society of the Pacific Conference Series, vol. 453, p. 317 (2012) Google Scholar
 Quinlan, GD, Tremaine, S: On the reliability of gravitational Nbody integrations. Mon. Not. R. Astron. Soc. 259, 505518 (1992) View ArticleADSGoogle Scholar
 Smirnov, N: Table for estimating the goodness of fit of empirical distributions. Ann. Math. Stat. 19(2), 279281 (1948) View ArticleMATHGoogle Scholar
 Smith, H Jr.: The dependence of statistical results from Nbody calculations on N. Astron. Astrophys. 76, 192199 (1979) ADSGoogle Scholar
 Szebehely, V, Peters, CF: Complete solution of a general problem of three bodies. Astron. J. 72, 876 (1967) View ArticleADSGoogle Scholar
 Urminsky, D: On the calculation of average lifetimes for the 3body problem. In: Vesperini, E, Giersz, M, Sills, A (eds.) IAU Symposium, vol. 246, pp. 235236 (2008) Google Scholar
 Valtonen, M, Karttunen, H: The ThreeBody Problem. Cambridge University Press, Cambridge (2006) View ArticleMATHGoogle Scholar
 Valtonen, M, Mylläri, A, Orlov, V, Rubinov, A: Statistical approach to the threebody problem. In: Byrd, GG, Kholshevnikov, KV, Myllri, AA, Nikiforov, II, Orlov, VV (eds.) Order and Chaos in Stellar and Planetary Systems. Astronomical Society of the Pacific Conference Series, vol. 316, p. 45 (2004) Google Scholar
 Verlet, L: Computer ‘Experiments’ on classical fluids. I. Thermodynamical properties of LennardJones molecules. Phys. Rev. 159, 98103 (1967) View ArticleADSGoogle Scholar
 Zadunaisky, PE: On the accuracy in the numerical solution of the Nbody problem. Celest. Mech. 20, 209230 (1979) View ArticleADSMATHMathSciNetGoogle Scholar