acceptodds
Withdrawn from ICLR 2027

TTShift : Learning to Adapt at Test Time with Shared Shifts

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Unlabeled test-time learning adapts LLMs using objectives available on target inputs, although these objectives only indirectly specify the reasoning behavior we ultimately care about. We first show that this distinction matters in practice: input next-token cross-entropy can continue to improve even as target accuracy, previously correct answers, and off-target behavior deteriorate. This motivates a complementary question: under such indirect supervision, how much update freedom is actually needed for useful adaptation? We introduce \method (Test-Time Shift), which freezes pretrained transformations and learns lightweight, dataset-shared additive shifts from unlabeled target inputs. Using Q/V projection-output shifts with a single fixed learning rate, \method improves mean accuracy over the corresponding Base model across seven reasoning tasks by percentage points on Qwen3-8B-Base and points on Qwen3.5-4B-Base. Controlled Q/V comparisons with rank-8 LoRA show that useful adaptation can persist after removing the learned input-dependent mapping correction: shared shifts exhibit a broader useful operating region and reach favorable gain–preservation operating points across the measured adaptation strengths. Further analysis shows that the shared direction can emerge from only partial cross-example gradient agreement and becomes directionally organized before its magnitude is fully accumulated, while downstream behavior continues to evolve. Together, these results suggest that effective test-time adaptation need not rely on highly expressive updates, and that restricting update freedom can provide a simple and useful way to adapt LLM reasoning behavior under indirect supervision.Anonymous code for reproducing the main experiments is available at https://anonymous.4open.science/r/review-code-FDE3/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Test-time activation steering, directional activation intervention, reasoning trajectory agreement, steering weight search, mathematical reasoning, large language models, inference-time adaptation.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LHijOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies unlabeled test-time adaptation for reasoning models and argues that reducing input next-token cross-entropy does not necessarily improve answer accuracy or preserve other capabilities. It proposes TTShift, which freezes the pretrained model and learns dataset-shared additive shifts at the Q/V projection outputs using only unlabeled target inputs. Experiments on Qwen3-8B-Base and Qwen3.5-4B-Base across seven reasoning tasks show that TTShift improves average accuracy over the base models and is more stable than Q/V LoRA across a wide learning-rate sweep. The paper also analyzes gradient agreement, shift formation, and how the learned Q/V translations affect attention computation.

Critical strengths
  1. The paper identifies a useful failure mode of test-time adaptation. The experiments clearly show that lower input CE can coincide with worse target accuracy and larger off-target degradation, which motivates restricting the adaptation space.

  2. The shared-shift result is simple and empirically interesting. TTShift uses far fewer trainable parameters than Q/V LoRA and shows a substantially broader positive operating region across learning rates, while often causing less damage to previously correct predictions.

Critical weaknesses
  1. The evaluation is fully transductive. The same target questions are used for both adaptation and evaluation. It is therefore unclear whether TTShift learns a shift that generalizes within the target distribution, or mainly adapts to the specific evaluation inputs. A held-out target split would make the generalization claim much stronger.

  2. The novelty relative to simpler shift/bias adaptation is not fully established. TTShift learns a dataset-shared additive vector at Q/V outputs, which is closely related in form to bias-only or representation-shift adaptation. The paper compares against LoRA, TLM, SLOT, and EATA, but lacks a tightly matched bias/shift baseline under the same objective and intervention locations. This makes it harder to isolate what is new about TTShift itself.

Questions for authors
  1. Can you evaluate TTShift in a non-transductive setting, where the shift is learned on one unlabeled subset of the target distribution and evaluated on held-out target examples?

  2. How does TTShift compare with a matched bias-only or simple representation-shift baseline using the same Q/V locations, optimization objective, and training budget?

  3. Can you report multiple seeds for the main results? The Qwen3.5-4B gain is only +0.94 pp, and most reported results are single-seed point estimates.

Reviewer_eG3MOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TTShift, which freezes a language model and learns dataset-shared additive shifts at query and value projection outputs using unlabeled input next-token cross-entropy. It argues that this restricted adaptation space can improve reasoning while limiting destructive behavioral changes. Across seven tasks, the reported mean gains are 3.38 percentage points on Qwen3-8B-Base and 0.94 points on Qwen3.5-4B-Base. Learning-rate sweeps compare TTShift with rank-8 Q/V LoRA, and additional analyses characterize gradient agreement, shift formation, and changes to attention computation.

Critical strengths
  • TTShift is simple and parameter-efficient: it trains 184,320 parameters on Qwen3-8B and reports improvements on all seven main tasks with one fixed learning rate. This makes the adaptation recipe practically interesting.
  • The learning-rate sweeps and off-target probes examine the mismatch between input-likelihood optimization and downstream behavior. Using GPQA for development separately from the main tasks also improves the interpretability of the evaluation.
Critical weaknesses
  • The methodological contribution is insufficiently distinguished from existing bias and representation adaptation. Without matched bias-only or scale-and-shift baselines, the results do not establish a substantive advantage of the proposed shared Q/V shifts beyond a compact parameterization; this is central to assessing the contribution.
  • The main explanation is not isolated experimentally. TTShift and rank-8 LoRA differ in capacity and effective update magnitude, so the learning-rate sweeps cannot establish that removing input-dependent updates causes the favorable trade-off. Controls matching functional change are needed to substantiate this central claim.
  • The empirical evidence is insufficiently robust to resolve these contribution concerns. Results are single-seed, the Qwen3.5 advantage over EATA is 0.04 percentage points, and the AIME25 gain is one answer. Preservation is also not established for each main-task adaptation under the final recipe, leaving the breadth of the claimed advantage uncertain.
Questions for authors
  1. How does TTShift compare with bias-only or scale-and-shift adaptation under the same unlabeled objective, adaptation sites where applicable, and tuning budget? This comparison is important for establishing what the proposed shared Q/V shifts contribute beyond existing lightweight adaptation parameterizations.

  2. Does the advantage over LoRA persist when capacity and functional change, such as output-distribution divergence, are controlled? Please also clarify the GPQA criterion and search budgets used to select the two learning rates, so that the comparison reflects comparable accuracy-preservation preferences.

  3. Can you provide paired confidence intervals and repetitions over shuffled adaptation orders for the main results? This is particularly important for the 0.04-percentage-point Qwen3.5 macro gap to EATA and the Qwen3-8B AIME25 gain of one correct answer, where the comparative conclusions may be sensitive to variability.

  4. Since the adaptation loss includes answer-format instructions and invalid outputs count as incorrect, how much do invalid-answer rates change? Does the gain persist on examples with parseable answers both before and after adaptation, or when instruction tokens are excluded from the loss?

  5. What off-target degradation occurs after adapting each of the seven main tasks with the final fixed learning rate, on both backbones? The existing preservation probes are useful, but this comparison would establish whether the gain-preservation trade-off holds for the configuration used in the main table.

Reviewer_pnwXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TTShift, a test-time adaptation method that learns dataset-shared additive shifts at the query and value projections while keeping the pretrained LLM frozen. The shifts are optimized using next-token cross-entropy on unlabeled test questions, without requiring gold answers or generated solutions. Experiments on two Qwen models and seven reasoning benchmarks show improved average accuracy and greater stability across learning rates compared with LoRA-based adaptation.

Critical strengths
  1. The method is simple and parameter-efficient. It updates only a small number of shared Q/V shift parameters without modifying the pretrained weights.
  2. The experiments cover seven reasoning benchmarks and two model architectures. The comparisons with LoRA, TLM, SLOT, and EATA provide a useful evaluation of the proposed method.
Critical weaknesses
  1. The improvements are modest, especially on Qwen3.5-4B-Base, where TTShift only outperforms EATA by 0.04 percentage points. The results are also based on a single seed and without sampling. I suggest pass@K or multiple seed and average.
  2. The methodological novelty is limited. The proposed method closely resembles existing bias-tuning and representation-shifting approaches that the authors acknowledge in related work. However, the paper does not establish whether TTShift outperforms simpler alternatives, such as bias-only tuning or parameter-matched low-rank updates (e.g., rank-1 LoRA). Without these comparisons, it is unclear whether the improvements come from the proposed update form or simply from reduced update capacity.
  3. In Sec. 4.1, the same questions are used for adaptation and evaluation. It is unclear whether the learned shifts generalize to unseen questions.
Questions for authors
  1. How were the baseline hyperparameters selected, and would TTShift still outperform LoRA and TLM under comparable tuning budgets or matched input-CE improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.