pipette
ESEspañol

Dimension-Free PAC-Bayes Generalization for Deep Neural Networks via Dirichlet Processes

Sumit Vashishtha

PreprintBold claims, read criticallyClaims a big step

In the authors' words

Classical PAC-Bayesian generalization theory places prior distributions over Euclidean parameter space , causing relative entropy to scale with network dimension . For modern overparameterized neural networks where , this parameter-counting dependency yields vacuous generalization bounds (). In this work, we resolve this barrier by shifting prior specification from internal parameters to the non-parametric space of data distributions via a Dirichlet Process (DP), , inducing parameter distributions through push-forward measures under the learning algorithm. By the Data Processing Inequality, parameter-space relative entropy is strictly bounded by the data-space DP divergence, which we evaluate in exact closed form using Digamma functions. The resulting PAC-Bayesian certificate is strictly invariant to network parameter count , width, and depth. On intrinsic -dimensional submanifolds, optimal partition scaling achieves the minimax non-parametric rate . Across vision architectures scaling up to 4.2 million parameters ( to ) on the full MNIST benchmark, while classical weight-space bounds scale past (becoming vacuous), our data-space DP-PAC-Bayes certificate remains stable, non-vacuous, and certified at --.

Main resultThe abstract does not state a limitation.

Appeared: Friday, October 16. arXiv. Preprint, not yet peer-reviewed.