HMC: Reducing the number of rejections by not using leapfrog and some results on the acceptance rate

Highlights: • Numerical evidence shows that leapfrog may not be best integrator to use within HMC. • Alternative integrators give fewer rejections with the same computational cost. • We establish a monotonic relationship between acceptance rate and energy error. • Novel theory validates the derivation of alternative integrators to leapfrog. The leapfrog integrator is routinely used within the Hamiltonian Monte Carlo method and its variants. We give strong numerical evidence that alternative, easy to implement algorithms yield fewer rejections with a given computational effort. When the dimensionality of the target distribution is high, the number of accepted proposals may be multiplied by a factor of three or more. This increase in the number of accepted proposals is not achieved by impairing any positive features of the sampling. We also establish new non-asymptotic and asymptotic results on the monotonic relationship between the expected acceptance rate and the expected energy error. These results further validate the derivation of one of the integrators we consider and are of independent interest.