CATL: Conflict-Aware Test-Time Learning
Abstract
This paper studies how to rapidly acquire useful information from test-time contexts. Existing test-time learning (TTL) methods either rely on dedicated offline pretraining, or adapt lightweight parameters online without explicitly controlling the interactions among successive updates, making information encoded from earlier context portions vulnerable to interference and overwriting in long contexts. To overcome this challenge, we propose *Conflict-Aware Test-Time Learning* (CATL), a plug-and-play TTL method that casts sequential context adaptation as an online continual-learning process and introduces a conflict-aware update mechanism guided by a compact historical write memory. The memory is constructed from the optimizer update, capturing the subspace, orientation, and importance of previous writes. Guided by the memory, CATL selectively attenuates only update components that conflict with important historical writes, while preserving compatible and orthogonal directions. It thus can mitigate destructive cross-chunk interference without replaying prior context or requiring additional offline training. Experiments on long-context retrieval and reasoning tasks show that CATL outperforms all evaluated baselines at the maximum supported context length across Qwen3 base models ranging from 1.7B to 8B parameters. CATL also improves performance on instruction-tuned backbones with different attention architectures while maintaining favorable inference-time efficiency.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.