When Does Adaptive Capacity Help? A Component Audit of Continual Learning
Abstract
Continual-learning systems combine trainable modules with allocation, routing, and prototype logit correction. We audit these components separately and find that capacity's value depends on the surrounding system. In a corrected frozen-feature adapter system, AURA, bypassing the pool preserves final accuracy within one percentage point on four feature sets when prototype logit correction remains present. Factorial and inference interventions show that prototype scores account for much of the text-system performance; the small gated pool effect also persists across four BGE task orders. This is not general capacity redundancy: a native CODA-Prompt positive control gains 1.54 points from a larger prompt budget, while a native SEMA diagnostic changes the sign of the expansion effect with metric aggregation. Under the selected routing/optimization protocols, DINOv2-B pools with a non-oracle class-centroid router gain 1.37 points post-encoder and 3.07 points in four internal blocks over no adapters; an internal task-mean configuration gains 0.93 points. These comparisons do not isolate routing from optimization. The corresponding post-encoder controls do not improve ViT-B/16. A stronger ridge classifier also exceeds AURA on all four frozen feature sets. These results support a practical evaluation principle: distinguish the value of a module subsystem from the value of additional allocation, and test both under explicit logit-correction, routing, and optimization controls.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.