acceptodds
Under review as a conference paper at ICLR 2027

Topology-Guided Adapter Lifecycles for Continual Test-Time Training

Abstract

Continual adaptation of a frozen language model commonly updates one low-rank adapter on an arriving stream. This design becomes fragile when domains recur: current updates overwrite earlier specialization, while periodic resets discard useful state. We treat adaptation as an adapter lifecycle problem and use the shape of the response cloud to decide when to route, spawn, or reuse an adapter. Two ingredients make this signal usable. First, a canonical response coordinate is invariant to the GL(r) gauge freedom of a low-rank factorization and preserves pairwise response distances; across our gauge sweep the resulting barcode drift stays below 2 × 10⁻¹², whereas the naive internal coordinate changes by more than 10². Second, a block comparison around an excluded transition band cancels any transient contained in that band while retaining the full separation between sustained regimes. On the three public continual learning protocols for language models, and without ever being told where one task ends, the resulting adapter bank reaches 80.0 on the Standard CL benchmark, matching the multi-task bound and exceeding every sequential published method, 78.6 on the fifteen-task Long Sequence benchmark against 73.6 for the strongest published replay-free method and 78.1 for per-task models, and 43.2 on TRACE against 40.0 and 32.9 for the strongest replay-free and boundary-free published results. Our pipeline reproduces every quoted baseline to within 0.5 points. Traversing each protocol a second time on disjoint shards separates the methods further: our score falls by 0.7 on TRACE where the strongest baseline falls by 5.5, and the bank holds nine adapters over sixteen stages of eight recurring tasks. A detection study on the same streams locates the cause, showing that the second-order signal used by the strongest boundary-free baseline collapses from 0.788 to 0.602 ROC-AUC on transitions whose moments agree, where the topological signal holds at 0.871. We also report an unlabeled test-time setting and a homogeneous stream on which topology offers no advantage.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.