pipette
ENEnglish

CAGE-NAS: Certified Functional Descent for Efficient Model Growth

Santiago Florido Gomez, St\'ephane Rivaud

PreprintAfirmaciones fuertes, leer con cuidado

En palabras de los autores

The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be expanded. CAGE-NAS formulates this decision in function space through an admissibility criterion on approximations of the functional gradient. As long as a representation enables a certified Functional Gradient Descent step, the architecture remains fixed; when the criterion fails, a function-preserving expansion is applied and the resulting representation is evaluated again. As the main instance, we study the family induced by the tangent space, using a regularized projection of the functional gradient. In a controlled setting with exact certification, CAGE-NAS produces architectures positioned above the 99.8th performance percentile by held-out RMSE among all admissible alternatives within the same parameter budget, without enumerating them during the growth trajectory.

Resultado principalLimitación que admiten los autores

Apareció: viernes, 2 de octubre. arXiv. Preprint, todavía sin revisión por pares.

Comentario de los autores: 18 pages, 4 figures, 4 tables. Accepted at AXIOM 2026: Foundations of Efficient Deep Learning (NeurIPS 2026 Workshop)