Freedom Causes Neural Generalisation

Can increasing freedom improve neural generalisation? In theory, maximising freedom is necessary to maximise generality. Furthermore, freedom of function measures behaviour, whereas complexity- or gradient-based alternatives are measures of form. This means freedom is reparameterisation invariant, and should yield more consistent performance improvements across architectures, embodiments and datasets. Yet in practice, freedom has proven difficult or intractable to operationalise. Here I present a fully neural approximation of freedom of function, estimating how many further input–output commitments remain possible while preserving correctness on training data. In each experiment, training data is used to calibrate an instrument with which freedom is approximated. Then, neural networks are trained and undergo evolutionary selection for freedom measured using that instrument. On all five benchmarks tested, KMNIST, binary Fashion-MNIST, Rotten Tomatoes, binary AG News and binary 20 Newsgroups, this intervention achieves higher mean test accuracy than evolution using either approximate marginal likelihood or gradient-based GdScore. It wins 86 of 120 pairwise comparisons, by up to 14.8%. As predicted, freedom is more consistent. Freedom’s final pick beats a random on every benchmark, by as much as 9.6%. On the text benchmarks in particular, a single generation of freedom selection improves test accuracy by 4.48–6.09%. These results offer causal evidence that selecting for freedom improves neural generalisation, connecting freedom of function to learned representations and adaptive behaviour.