pipette
ESEspañol

Sieve-free ReLU Bayesian priors with near-minimax posterior contraction

Hien Duy Nguyen, Jacob Westerhout, Julyan Arbel

Preprint

In the authors' words

We establish fractional-posterior contraction for shallow ReLU networks in Gaussian random-design regression with a -H\"older target, using priors independent of the sample size and smoothness parameters. For any fixed design law on , we obtain contraction in R\'enyi divergence and, after clipping posterior draws at a bound on the target range, in squared loss at rate . This matches the classical H\"older minimax rate up to a logarithmic factor. These priors have random finite width and use width-dependent uniform or Gaussian weight distributions, or width-independent Student distributions, including Cauchy distributions.

Main resultLimitation the authors admit

Appeared: Monday, October 19. arXiv. Preprint, not yet peer-reviewed.