HMC and NUTS (very briefly)

This is a very brief overview. For more details see the Stan manual and Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo.

Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) method that uses the derivatives of the density function being sampled to generate efficient transitions spanning the posterior. It uses an approximate Hamiltonian dynamics simulation based on numerical integration which is then corrected by performing a Metropolis acceptance step.

Algorithm summary The Hamiltonian Monte Carlo algorithm starts at a specified initial set of parameters; in Stan, this value is either user-specified or generated randomly. Then, for a given number of iterations, a new momentum vector is sampled and the current value of the parameters is updated using the leapfrog integrator with discretization time stepsize and number of steps n_leapfrog according to the Hamiltonian dynamics. Then a Metropolis acceptance step is applied, and a decision is made whether to update to the new state or keep the existing state.

No-U-Turn Sampler

The no-U-turn sampler (NUTS) automatically selects an appropriate n_leapfrog in each iteration in order to allow the proposals to traverse the posterior without doing unnecessary work. The motivation is to maximize the expected squared jump distance (see, e.g., Roberts et al. (1997)) at each step and avoid the random-walk behavior that arises in random-walk Metropolis or Gibbs samplers when there is correlation in the posterior. For a precise definition of the NUTS algorithm see Hoffman and Gelman (2011, 2014)

Algorithm summary NUTS generates a proposal by starting at an initial position determined by the parameters drawn in the last iteration. It then generates an independent unit-normal random momentum vector. It then evolves the initial system both forwards and backwards in time to form a balanced binary tree. At each iteration of the NUTS algorithm the treedepth is increased by one, doubling n_leapfrog and effectively doubling the computation time. The algorithm terminates in one of two ways, either Rather than using a standard Metropolis step, the final parameter value is selected via multinomial sampling among the Hamiltonian trajectories.

Configuring the no-U-turn sampler involves putting a cap on the treedepth that it evaluates during each iteration. This is controlled through a maximum depth parameter. The number of leapfrog steps taken is then bounded by 2 to the power of the maximum depth minus 1.

For more details see Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo.