A Statistical-Physics-Inspired Framework of Machine Learning

While the statistical and probabilistic aspects of neural networks (NNs) have been extensively studied, conventional NN training typically seeks a single optimal configuration of internal parameters, which is fundamentally underdetermined when the number of parameters exceeds the number of training samples. In this paper, we investigate a framework of statistical-physics-inspired machine learning (SML). Predictions are obtained as expectation values by integrating over all possible parameter configurations, rather than relying on the optimal parameter configuration. Such an algorithm allows regular system-level behavior to emerge naturally, without being overwhelmed by the underlying irregular parameter-space landscape. The number of unknowns scales with the training-data size, not the NN size, making the approach attractive to the high-dimensional low-sample-size regime. Realized in closed form on a power-law network (PN), SML ranks first on standardized function benchmarks against 19 high-performance reference model variants, and outperforms most of them on 848 real-world datasets. SML-PN remains quite robust to label noise, irrelevant features, and contamination.