pipette
ESEspañol

Theoretical guarantees for stochastic gradient Langevin dynamics

Daniel Paulin, Peter A. Whalley

Preprint

In the authors' words

We prove asymptotic bias bounds for stochastic gradient Langevin dynamics in Wasserstein distance of order two. We assume that the negative log-density is strongly convex with a Lipschitz gradient, and that the stochastic gradient estimator is unbiased with an error satisfying a mean-square Lipschitz condition. The bounds are of order under a fourth moment assumption on the stochastic gradient error and of order under only a second moment assumption, where is the stepsize. A spiked-noise example shows that a second moment assumption alone is insufficient for a bound of order that is uniform over noise distributions with a fixed variance.

Main resultLimitation the authors admit

Appeared: Friday, October 2. arXiv. Preprint, not yet peer-reviewed.

Authors' comment: 8 pages, 1 figure