Precise Convergence Speed of Clipped SGD
En palabras de los autores
We present a tightened convergence analysis of clipped gradient descent on -smooth functions, with quantitative constants. Building on the ideas of Koloskova et al (2023), we refactor several case disjunctions to reveal the central role of a control of the bias derived from fundamental properties of -projection, simplifying proofs. We also extend the domain of validity from to where for clipping constant , which matches the more traditional analysis of smooth functions. We strengthen the convergence criterion from to with matching speed, and lower the final achievable loss from to the more precise .
Resultado principalEl resumen no menciona limitaciones.
Apareció: viernes, 25 de septiembre. arXiv. Preprint, todavía sin revisión por pares.