When Does Adaptive Capacity Help? A Component Audit of Continual Learning
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Continual-learning systems combine trainable modules with allocation, routing, and prototype logit correction. We audit these components separately and find that capacity's value depends on the surrounding system. In a corrected frozen-feature adapter system, AURA, bypassing the pool preserves final accuracy within one percentage point on four feature sets when prototype logit correction remains present. Factorial and inference interventions show that prototype scores account for much of the text-system performance; the small gated pool effect also persists across four BGE task orders. This is not general capacity redundancy: a native CODA-Prompt positive control gains 1.54 points from a larger prompt budget, while a native SEMA diagnostic changes the sign of the expansion effect with metric aggregation. Under the selected routing/optimization protocols, DINOv2-B pools with a non-oracle class-centroid router gain 1.37 points post-encoder and 3.07 points in four internal blocks over no adapters; an internal task-mean configuration gains 0.93 points. These comparisons do not isolate routing from optimization. The corresponding post-encoder controls do not improve ViT-B/16. A stronger ridge classifier also exceeds AURA on all four frozen feature sets. These results support a practical evaluation principle: distinguish the value of a module subsystem from the value of additional allocation, and test both under explicit logit-correction, routing, and optimization controls.
- Primary area:
- transfer learning, meta learning, and lifelong learning