Double descent without digital computation
In the authors' words
The success of large-scale AI relies on the phenomenon of double descent, where overfitting first increases and then counterintuitively decreases as the network grows relative to the amount of training data and becomes overparameterized. In this overparameterized regime it is easy to find good solutions that do not overfit. However, double descent has not previously been observed in physical neural networks, which are attracting increasing attention but remain small-scale. Here we demonstrate double descent in a decentralized analog network of self-adjusting resistive elements. This system trains itself and performs tasks without a digital processor, offering potential gains in energy efficiency and speed-but must inevitably confront component nonidealities. We find that standard training protocols fail to yield double descent, but a modified protocol that accommodates generic imperfections succeeds, qualitatively matching digital neural network simulations. Our findings show that analog physical neural networks, if appropriately trained, can reproduce important facets of overparameterized digital learning. This result is critical if such networks are to be scaled up to useful sizes.
In plain words AI summary
Double descent is a pattern where overfitting first rises then falls as AI networks grow larger than their training data. The authors built a physical network of self-adjusting resistors that learns without any digital processor. Standard training failed to show double descent, but a modified protocol that handles imperfections succeeded. Properly trained analog networks can replicate key features of large digital AI systems. This matters for scaling up physical neural networks to practical sizes.
Written by Kimi K2.5 using only the abstract above. Jev checked that every sentence is supported by the abstract sentences it cites, and removed any that were not. Point at a sentence to highlight its source.
Appeared: Saturday, September 26. Proceedings of the National Academy of Sciences. Peer-reviewed journal.