acceptodds
Under review as a conference paper at ICLR 2027

Donor-Derived Token Organization Changes What a Neural Network Can Reach

Abstract

Learned neural representations can alter what a new learner subsequently reaches, but identifying which properties of a transferred representation carry that effect requires interventions that preserve some properties while replacing others. We study this problem by transplanting a rank-4 carrier learned on modular addition into independently initialized recipients. In a prospectively specified experiment, donor carriers reach a fixed generalization criterion in 45 of 64 donor–recipient cells, whereas matched substitutes preserving the donor spectrum, right basis, norm, rank, and recipient coupling reach it in 0 of 64. The effect replicates on a fresh recipient cohort (47/64 versus 0/64). Severe projection onto only the donor’s dominant Fourier modes retains substantial reachability (35/64). Moreover, synthetic carriers matched to these projections in rank, Fourier-support complexity, dominant-mode concentration, and singular spectrum are themselves active (36/64 versus 0/64 for norm-matched random carriers), although the experiment does not identify which matched property carries the effect. Conversely, randomly permuting a donor carrier across token identities reduces reachability from 47/64 to 0/64 while leaving it in exactly the same row-permutation orbit. Thus, descriptors invariant to row-to-token assignment cannot determine learning reachability at this operating point. Together, these interventions show that compact representational organization can causally alter finite-horizon learning reachability, while progressively narrowing—but not yet identifying—the structural properties sufficient to carry the effect.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.