- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CATL, a plug-and-play method for sequential test-time adaptation on long contexts. It updates selected MLP down-projection matrices and maintains a memory of previous parameter writes, including their subspace, accumulated directions, and importance. Incoming update components that oppose this memory are attenuated, while compatible and orthogonal components are retained. Experiments on BABILong and selected RULER tasks cover three Qwen3 base models and two instruction-tuned backbones. The paper reports improvements at 32K context length, together with component ablations and runtime and memory comparisons.
Critical strengths
- The method provides a concrete, replay-free way to distinguish compatible from conflicting historical writes. Preserving compatible components addresses a meaningful limitation of indiscriminate subspace projection.
- Improvements at 32K are consistent across the three base-model sizes, with additional gains on instruction-tuned models. This supports the usefulness of the adaptation recipe across several backbones.
Critical weaknesses
- The incremental benefit of the proposed mechanism is insufficiently established. The only matched naive-TTL comparison is Table 3, where CATL improves overall accuracy from 40.71 to 41.39. Without this baseline across the main settings or uncertainty estimates, the larger comparisons do not isolate the contribution of conflict handling.
- Historical parameter-write alignment is only a proxy for retaining earlier information. Non-expansiveness does not establish preservation of previous losses, and a norm-matched shrinkage control is missing. Moreover, the motivating naive-TTL visualization uses 4K contexts, whereas Appendix F evaluates CATL at 32K.
- Comparison fairness is unclear: CATL uses configuration-specific learning rates, while the competing methods use fixed recipes (Appendix C). The selection data, per-setting choices, and comparable tuning budgets are not sufficiently documented.
Questions for authors
-
Table 3 shows a relatively small gain over matched naive TTL. Can you report this comparison across the main model and benchmark settings, with paired uncertainty estimates, to clarify how consistently conflict handling improves over the underlying TTL procedure?
-
Does CATL outperform a control that uniformly rescales each naive-TTL update to match CATL's update norm? This would help distinguish the benefit of selectively handling conflicting directions from the benefit of simply taking smaller updates.
-
Can you compare CATL and naive TTL under the same 32K setup and measure how later updates affect losses on earlier chunks? The current write-alignment analysis does not directly establish that earlier information is better preserved.
-
How were CATL's configuration-specific learning rates selected, and which development data and search ranges were used? Do the comparative gains persist when competing methods receive comparable tuning budgets under the same selection protocol?