pipette
ENEnglish

Variance Reduction Note 2: The Convergence of SPIDER

Alina Lodhi

Preprint

En palabras de los autores

The stochastic path-integrated differential estimator (SPIDER) tracks a gradient by accumulating small, shared-sample gradient differences and periodically refreshing the estimate. This second expository note on recursive variance reduction explains SPIDER-SFO and gives a self-contained proof of its expected first-order stationarity guarantee. The argument combines an epochwise mean-square error bound with descent under a clipped, normalized update. It yields stochastic gradient evaluations in the online setting and component gradient evaluations for a finite sum, with the remaining problem parameters fixed. All refresh costs are included. The presentation is pedagogical and makes no claim of a new algorithm or convergence rate.

Resultado principalLimitación que admiten los autores

Apareció: martes, 20 de octubre. arXiv. Preprint, todavía sin revisión por pares.