Geometric Moment Contraction for Stochastic Nesterov Acceleration
In the authors' words
We study geometric moment contraction (GMC) of the constant-parameter stochastic Nesterov recursion \[ Y_k=\Theta_k+\beta(\Theta_k-\Theta_{k-1}),\qquad \Theta_{k+1}=Y_k-\gamma G(Y_k,X_{k+1}). \] Under mean strong monotonicity and stochastic Lipschitz continuity, an explicit Perron comparison proves synchronous contraction when . This direct criterion includes infinite-variance gradients for , but its small-step regime requires . A complementary power-Lyapunov argument establishes a positive, generally much smaller, step-size interval for every fixed and every , using only a finite th gradient moment. At , a simpler explicit certificate gives \[ 0<\gamma<\frac{2\mu(1-\beta)^2}{L_2^2(1-\beta+2\beta^2)}. \] Its quadratic high-momentum scaling is a limitation of the chosen metric, not a sharp stability boundary. We quantify this loss, provide a general mean-only quadratic -procedure, and exploit endpoint Lyapunov inequalities under stronger samplewise sector information. Verified endpoint certificates can be orders of magnitude less conservative than the explicit metric.
Appeared: Monday, September 28. arXiv. Preprint, not yet peer-reviewed.