False Learning in Sparse Parity: From Reversed Updates to Resonant Barriers
Abstract
We identify false learning in sparse parity: task-defined data subsets generate systematic updates away from the correct selector, and these reversed updates can be coherently amplified into population-loss barriers. In the factorized selector–evaluator architecture we study, training success forms strikingly non-monotone bands across the input size and task sparsity . We trace this behavior to population atoms that lie deterministically on the wrong slopes of the frozen evaluator's periodic loss, even though the labels and population gradients are exact and the evaluator is correct on its discrete support. During training, many such atoms can cross loss vertices within the same narrow interval, aligning their false-to-true transitions and creating a barrier. A Fourier analysis describes this collective effect through resonances of the atom-distribution characteristic function, coupled to evaluator-loss harmonics. The resonance condition reduces to simple integer relations , and the resulting barriers account for the observed bands of trainability. False learning is therefore not merely ordinary wrong-way motion inside an isolated spurious basin: sparse-parity structure makes the reversed updates and their collective amplification systematic and predictable.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.