acceptodds
Under review as a conference paper at ICLR 2027

Where Does Augmentation Repair Compositional Generalization? The Locus Shifts with Compression

Abstract

Augmentation is known to repair compositional generalization failures in neural decoders, but what it repairs, and where, is less clear. In a controlled toy regime we establish a full mechanism including a sharp repair-locus transition; in more realistic settings we find a weaker but consistent regularity. We study this in a controlled setting: a compressed (superposed) representation is decoded by a serialized language readout, and augmentation is applied under pre-registered, budget-matched protocols (1000+ trained models, all judgments fixed before execution). Four results emerge. (R1) Failure is a content-addressing mismatch. Probes recover the information (separation ratio ) while slot-level diagnostics locate the collapse in the object slots (0.49–0.74), with the relation slot substantially more robust (0.77–0.96); the pattern holds in both a toy regime and a semi-synthetic bridge. (R2) Repair is unit-local and thresholded. Augmenting a combination rebuilds that combination's readout pathway: 300 examples per unit raise covered units to 0.70–1.00 (median 0.93) while 75 per unit leave accuracy near chance (), and no transfer to unaugmented units occurs. Coverage, not augmentation content, is therefore the driver of repair: mechanism-guided selection is never better than random in the toy regime, and uniform coverage dominates mechanism-guided selection in the bridge (both pre-registered, 5 seeds). (R3) Compression deepens failure (0.000 at versus 0.195 with no compression) and shapes repair efficiency non-monotonically: mid-compression (–) repairs slowest. (R4) In the toy regime the repair locus depends on compression. At the readout pathway alone is sufficient (freezing the encoder still repairs covered units, , 5/5 seeds); at encoder reorganization is required (freezing costs 0.52, 0/5). A seven-point frozen-encoder scan shows the transition is continuous (crossover between and ), and R3's slow-repair plateau disappears when the encoder is frozen: the plateau is a property of encoder reorganization, not of the readout. A first-order account based on Cover's linear-separability capacity predicts the transition band from first principles, with no fitted parameters. On continuous features the locus does not transfer naively (a random frozen code fails by 0.56), while freezing a trained encoder with decoder-only augmentation recovers most of the repair (0.62 vs. 0.75) at lower cost. Under end-to-end training of a small CNN the failure pattern and the weak form both reproduce: unseen accuracy is 0.020 at baseline with object slots at 0.39 against a relation slot at 1.00, and freezing the trained encoder recovers 88.5% of the repair gain (0.539 vs. 0.606). Throughout, geometry changes globally under augmentation while behavior repairs locally: geometric reorganization is incidental at mild compression and is the repair itself at strong compression, a locality that slot-level measurement reveals and geometric analysis cannot.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.