Optimal approximation of smooth functions by ReLU neural networks
En palabras de los autores
We study uniform approximation of the H\"older ball by fully connected ReLU networks. Except in the one-dimensional Lipschitz case, existing constructions remain separated from the information-theoretic efficiency limit. This article closes the gap for every and : networks with width , depth , and parameter magnitude attain uniform approximation error of order . No relation between width and depth is required. The construction uses new encoding and decoding mechanisms that exploit higher-order finite-difference bounds for local Taylor coefficients of the target function. As a statistical consequence, empirical risk minimizers over the corresponding clipped network classes attain the minimax regression rate without additional logarithmic factors. The paper also gives a detailed exposition of the key approximation techniques, covering both established and new mechanisms.
Apareció: lunes, 19 de octubre. arXiv. Preprint, todavía sin revisión por pares.