Abstract
Diffusion generative modelling (DGM) based on stochastic differentialequations (SDEs) with score matching has achieved unprecedented results in datageneration. In this paper, we propose a novel fast high-quality generativemodelling method based on high-order Langevin dynamics (HOLD) with scorematching. This motive is proved by third-order Langevin dynamics. By augmentingthe previous SDEs, e.g. variance exploding or variance preserving SDEs forsingle-data variable processes, HOLD can simultaneously model position,velocity, and acceleration, thereby improving the quality and speed of the datageneration at the same time. HOLD is composed of one Ornstein-Uhlenbeck processand two Hamiltonians, which reduce the mixing time by two orders of magnitude.Empirical experiments for unconditional image generation on the public data setCIFAR-10 and CelebA-HQ show that the effect is significant in both Frechetinception distance (FID) and negative log-likelihood, and achieves thestate-of-the-art FID of 1.85 on CIFAR-10.