acceptodds
Withdrawn from ICLR 2027

Adapt Only When It Pays: Selective, Verifiable Test-Time Self-Training for LLM Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

12 views

Abstract

Test-time optimization adapts LLM parameters during inference and can correct misconceptions that test-time scaling cannot; recent query-conditioned approaches remove the dependence on external data by synthesizing structurally related problem–solution pairs from the input query itself and fine-tuning on them before answering. This recipe is effective but uniform in two respects: adaptation is **unconditional** — every query pays the full synthesis-and-update cost, including the majority the base model already answers correctly — and every synthesized pair is weighted **equally as supervision**, although it originates from the same model whose competence is in question, a reliability gap prior work identifies as a limitation but does not instrument. We propose **SelVer-TTT**, which makes test-time adaptation selective and its supervision verifiable: a cheap pre-adaptation probe gates the expensive update to queries that are both uncertain under the base model and resolvable by adaptation; for gated queries we bias synthesis toward variants with automatically checkable structure and weight each pair by whether its solution survives verification, so unverified samples never act as positive supervision; and following the persistent test-time optimization direction outlined as future work by prior query-conditioned self-training, we accumulate LoRA increments into a **bounded, decaying residual slot** under an explicit norm budget, reporting the resulting drift directly. Across 7 mathematical reasoning benchmarks and GPQA-Diamond, SelVer-TTT matches or exceeds unconditional query-conditioned self-training while adapting on only 38% of queries and reducing average wall-clock inference overhead by 2.4×, with verification-weighted supervision contributing 1.9 points over equally-reliable weighting and widening on the hardest split; the bounded residual memory retains 71% of per-query adaptation benefit across a 500-query stream while keeping degradation on a held-out general benchmark below 1.2 points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time training, test-time optimization, self-training, LLM reasoning, verifiable rewards, parameter-efficient adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_UcFkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Authors propose SelVer-TTT that selectively performs test-time self-training. The method estimates uncertainty from multiple sampled solutions and synthesizes training examples + solutions of the same schema. If a problem is sufficiently uncertain and if synthetic supervision passes checks, then a LoRA adapter is trained with residuals discarded between tasks or decayed in persistent mode. Authors report 51.6% unconditional -> 52.4% selective mean accuracy across several reasoning benchmarks with Qwen2.5-7b-instruct. The conditional setup reduces queries adapted and overhead adaptation time by ~60%.

Critical strengths

The synthesized supervision is checked before used for weight updates-- even the rationale is checked apparently. This contributes to the selectivity that mitigates noise from self-supervised TTT. Overall seems somewhat effective in reducing adaptation time while preserving and slightly improving overall accuracy.

Critical weaknesses

The experimental details around verification are underspecified. Checkers and structural-validators are mentioned and support the selectivity of this approach but are not described or given concrete examples per benchmark. Moreover, the results do not sufficiently clarify each component's contribution or reliability. Tables lack uncertainty estimates and ablation experiments, limiting conclusions about specific interventions.

Questions for authors
  • What structural validators and checkers produced tables 1-3?
  • Can you share representative accepted/rejected synthetic task samples (especially for GPQA)
  • How is retention/degradation computed for the multi-query/stream settings?
Reviewer_4N7jOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes SelVer-TTT, a query-conditioned test-time self-training framework for reasoning. It uses a pre-adaptation gate based on answer disagreement and the availability of verifiable synthetic examples, trains on checker-accepted targets, and manages LoRA updates through bounded residuals with optional cross-query retention. Experiments with Qwen2.5-7B-Instruct on seven mathematical benchmarks and GPQA-Diamond report a mean accuracy of 52.4% versus 51.6% for unconditional self-training, while adapting on 38% of queries and reducing average overhead from 96.8 to 40.3 seconds.

Critical strengths

Relevant practical motivation.

The paper addresses two practical challenges in test-time self-training: unnecessary adaptation cost and unreliable synthetic supervision. Combining selective updates, verification, and bounded residual memory is a reasonable design direction.

Critical weaknesses

W1. Limited novelty and missing comparison with the closest prior work.

QueST [1] already proposes query-conditioned problem–solution synthesis followed by LoRA-based test-time self-training, and evaluates on six overlapping mathematical benchmarks and GPQA-Diamond. SelVer-TTT adds gating, verified supervision, and residual memory, but does not discuss or directly compare with QueST (Sections 2 and 5). Its incremental contribution is therefore insufficiently established.

[1] Chaehee Song, Minseok Seo, Yeeun Seong, Doyi Kim, and Changick Kim. Query-Conditioned Test-Time Self-Training for Large Language Models. arXiv:2605.13369, 2026. https://arxiv.org/abs/2605.13369

W2. The gating strategy lacks convincing justification and ablations.

The gate measures answer disagreement and the availability of checkable synthetic examples, neither of which directly estimates adaptation gain (Section 4.1). With three samples and a threshold of 0.62, its uncertainty condition is met whenever no parsed answer occurs twice, potentially including parsing failures (Equation 1; Appendix A). The paper lacks matched-rate comparisons with uncertainty-only, checkability-only, and random gating, leaving the value of the combined policy unclear.

W3. Verified supervision is underspecified and its utility unproven.

The concrete checker implementations, structural validators, and reliability weights are inadequately documented (Sections 4.1–4.2; Appendix A), particularly how synthetic GPQA-Diamond targets can be independently checked without evaluation answers. Structural similarity and answer verification also do not establish that synthetic examples teach skills needed for the original query; no qualitative examples or direct analysis substantiate this assumption. Moreover, Table 3 does not clearly define the "equally-reliable weights" baseline, weakening the causal interpretation of the reported 1.9-point gain (Section 5.3).

W4. The empirical evidence does not establish a robust accuracy–efficiency advantage.

The improvement over unconditional self-training is only 0.8 points, without reported variability across runs (Table 1). The claimed 2.4× speedup lacks hardware details, a component-wise latency breakdown, and compute-matched inference-only baselines (Table 2). Persistent-mode results are also insufficiently documented: the 71% retention metric lacks a precise operational definition, the claimed 0.9-point MMLU drop cannot be reconstructed from Table 3 without the frozen-model reference, and no multiple-stream-order evaluation is reported despite the requirement stated in Section 4.4 (Section 5.4; Appendix C).

W5. Internal inconsistencies and unclear algorithmic justification.

Appendix A describes the gate statistic as "self-consistency entropy," whereas Equation (1) defines answer disagreement. The introduction refers to a subsequent "STV reference-paper section" that does not exist (page 2). Furthermore, rank truncation after each fixed-rank LoRA factor update (Equation 5) appears redundant because the product already has rank at most r; unlike compression when committing combined residuals (Equation 6), its necessity and cost are not justified.

Questions for authors

See weakness.

Reviewer_qnoKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies test-time self-training. A model creates practice problems related to a test question, trains a small LoRA update on them, and then answers the question. The method adds three controls to this process. First, a cheap check on the frozen model decides whether to adapt at all. The check passes only when the model's sampled answers disagree enough and when enough practice problems pass an automatic checker. Second, only practice problems that pass the checker receive training weight. Third, the LoRA update is kept within a fixed rank and size limit. In a persistent mode, this update is also carried to the next question with decay. With Qwen2.5-7B-Instruct on eight benchmarks, the method is 0.8 points better on average than adapting on every question.

Critical strengths
  1. The motivation for the work: doing test-time adaptation when it's worth the cost is interesting and could have good practical value.
  2. The method is efficient. It matches or beats adapting on every question while adapting on only 38% of questions and cutting the extra time per question by some fraction.
  3. The design is careful about label leakage. The gate and the checker are not allowed to see evaluation answers, and rejected or unverified practice problems get zero training weight.
  4. The update is kept safe by construction. It has a fixed rank and size limit. It never overwrites the base model, and the method falls back to the base model when something fails.
Critical weaknesses
  1. It’s unclear what the paper adds beyond existing work. Two of the method's main ideas follow common practice. Adapting only when the model is uncertain is a common practice in existing work on test-time compute allocation. Training on self-generated data that passes a correctness check is standard in self-training methods such as STaR and ReST-EM. The new idea is the resolvability score in the gate. However, the paper never compares the proposed gate with one that uses only the uncertainty score, so we do not know whether this score adds any value. The experiments (Section 5) contain three small tables, one figure, and less than one page of text. A reader cannot learn when or why test-time self-training helps a reasoning model, which is the question that the paper should focus on. The current version does not show a clear finding that can be valuable for future work on TTT.
  2. One major concern is the weak baseline quality. The existing baselines do not use the same compute. The method still adds about 40 seconds per question, but its only adaptation baseline is its own unconditional version. It should be compared at the same time budget with majority voting, best-of-N using the same checker, and in-context learning over the checked practice problems. The last comparison is needed to show that the weight update is necessary at all. The paper does not cite or compare with existing TTT methods that also update the model at test time. Examples include fine-tuning on retrieved neighbors (Hardt & Sun 2023), test-time training for few-shot reasoning (Akyürek et al. 2024), active data selection for test-time fine-tuning (SIFT), test-time reinforcement learning with majority-vote rewards (TTRL), and self-adapting models (SEAL). The paper does not cite any of them or use them as baselines. Without this comparison, it’s not clear what the paper adds to existing test-time training work.
  3. The experiments do not show which part of the method contributes to the gain. The gain over the baseline is pretty small, and the reported comparisons do not clearly separate its sources among the gate, checker, weight update, or the persistent mode (see Questions A1–A4). The paper also does not explain which checker the experiments use or how it confirms that a synthetic answer is correct, or report how often accepted answers are actually correct (see Questions A5–A6).
  4. The writing is often unclear, and several key terms are not explained. The paper is hard to read because much of the text is spent on rules and disclaimers, while several terms needed to understand the results are not defined (see Questions B1–B2).
  5. The setup is described differently in different places, and several results seem hard to reconcile. The same part of the method is sometimes described in two different ways, and some numbers do not fit together easily (see Questions C1–C3).
  6. The evaluation is too narrow to support the paper’s general claims. First, every task has a short final answer (math benchmarks and MCQA GPQA-Diamond), which is the easiest case for automatic checking, and the code and proof checkers that the paper lists are never tested. Second, the benchmarks lack diversity and difficulty. All but one are math; MATH-500 may overlap with MATH, half are rated easy or medium by the paper itself, and more recent sets such as AIME 2026 or HMMT 2026 are missing. Third, no experiment targets the settings where test-time training should matter most, such as clear distribution shift or streams of related questions for the persistent mode. Finally, only one model is tested, and it's not the most common reasoning model that can produce high-quality long thinking.
Questions for authors

A. Sources of the gain (Weakness 3)

  1. Does the unconditional baseline in Table 1 use the checker? Section 5.2 suggests that it trains on unverified data.
  2. What is the accuracy of a gate that uses only the uncertainty score?
  3. How does accuracy change if the checked practice problems are used in the prompt without a weight update?
  4. Does persistent mode give higher benchmark accuracy than per-query mode?
  5. Which checker do the experiments use, including for practice problems created from GPQA-Diamond questions?
  6. What fraction of practice problems are accepted, rejected, and marked unknown, and how often are accepted answers actually correct?

B. Clarity (Weakness 4)

  1. What are the "STV reference-paper section" (line 107) and the "STV verification" material (lines 135 to 136), and do all experiments come from SelVer-TTT?
  2. What is the "monitoring split," and does it overlap with any test benchmark?

C. Setup and results (Weakness 5)

  1. Does the 100% point in Figure 3b use the checker? If so, why does it equal the unconditional baseline (51.6) when verification weighting adds 1.9 points (Section 5.3)?
  2. A perfect gate would add 1.4 points, which is more than the method's 0.8-point gain over the baseline. Why does Section 5.4 conclude that gate accuracy matters less than verified supervision?
  3. How is the "71% retained" figure computed, given that a decay of 0.995 leaves only about 8% of the first question's update after 500 questions?

D. Other questions

  1. How does the method compare to majority voting with the same time budget?
  2. How were the gate thresholds chosen, and why use a 38% adaptation rate when 50% to 70% gives higher accuracy?
  3. Why are the base model's MATH and GSM8K scores much lower than the published results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.