acceptodds
Under review as a conference paper at ICLR 2027

Search Into Weights: Turning Test-Time Search into Capability per Parameter

15 views

Abstract

Test-time search produces both candidate solutions and feedback that can train a small model during inference. We propose a memory-conditioned draft model that serves as a speculative proposer and a within-problem learner. An append-only memory stores representations of previously sampled, target-verified rollouts and their outcome scores; gated cross-attention exposes these records to the draft. Verification distributions provide a distillation objective, while outcome-weighted replay provides an auxiliary learning signal. We distinguish two mechanisms that are often conflated: under exact speculative sampling, adapting the draft changes sampling cost but not the frozen target's conditional output distribution; direct changes to search quality require an explicit draft-dependent selection rule. We give sufficient conditions for exact sampling with adaptive memory, describe replay bookkeeping, and derive a compute ledger and a conditional break-even criterion. We specify a matched-budget evaluation on mathematical reasoning, including a decoder-only control given the same rollout information. This paper presents a method and analysis with an evaluation protocol; empirical improvements in accuracy, acceptance length, and architectural efficiency remain untested.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time training, speculative decoding, tree search, LLM reasoning, test-time compute

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_QPpWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes using a small draft model in speculative search and train itself during a single problem. It remembers previously finished attempts through an add-on attention layer and learns from two signals: matching the large model's predictions, and imitating attempts that scored well. The paper shows that this can keep the large model’s outputs unchanged, which also means the extra training can't directly improve answer quality, only speed. It then lays out an experiment plan but reports no experiments.

Critical strengths

None

Critical weaknesses
  1. Despite being a method paper, no empirical evidence is provided and none of the central claims is tested.
  2. The method simply assembles known components like LoRA test-time training, verification-feedback loss, etc. The novelty is limited.
Questions for authors
  1. A single MATH or AIME problem yields relatively few completed rollouts. Can they meaningfully improve the draft model?
  2. Section 6 makes accuracy at a matched FLOP budget the primary metric, with wall-clock time only secondary. However, the known benefit of speculative decoding is lower latency, not fewer FLOPs. Why choose FLOPs rather than latency as the primary budget?
Reviewer_DBUBOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes training a small draft model during tree search on each problem. The draft reads a memory of completed rollouts and learns from target-model verification scores and outcome-weighted replay. The target model stays frozen and verifies the proposed tokens. The paper gives arguments for preserving the target model's sampling distribution and recording the information needed to replay draft proposals. It also gives a compute-cost calculation and an evaluation plan. The abstract and Section 7 explicitly state that the proposed performance improvements have not been tested.

Critical strengths
  1. The theoretical claims are limited in a useful way. Keeping the draft and memory fixed within each verification burst makes the target-distribution argument clear. The paper also correctly separates the probability of the draft's raw proposal from the probability of tokens corrected by the target model. These points help explain what the method can guarantee, and the paper does not treat outcome-weighted replay as an unbiased policy-gradient update.

  2. The cost discussion includes training and memory-update costs, which are necessary for deciding whether the method helps at a fixed budget. The planned comparisons and ablations cover several important parts of the method. The manuscript is readable, and I did not see obvious overlap, cut-off equations, or other major layout problems. These planned experiments are useful, although they are not experimental evidence yet.

Critical weaknesses
  1. The main problem is the lack of experimental evidence. The Introduction gives a plausible reason to learn from previous search rollouts, but the paper does not show that its proposed memory and updates actually improve the draft on a problem. There are no accuracy results, acceptance-rate results, repeated runs, or completed component ablations. Thus, it is not possible to judge whether the full method solves the stated problem or whether any improvement is significant. In particular, the effects of memory cross-attention, memory contents, the two training losses, and the update schedule remain untested. Listing these ablations in an evaluation plan does not establish that the components work.

  2. The mathematical results do not close this evidence gap. Preserving the target model's sampling distribution ensures correct sampling, but does not by itself improve answer quality. The paper acknowledges this. Under the conventional verification cost used in Equation 4, even perfect acceptance still has a higher arithmetic cost per token than target-only sampling, because the draft also costs compute. Faster parallel verification could help latency, but that is a different measurement. The break-even condition is useful accounting, yet its required savings have not been measured. Without results showing where the savings come from and whether they cover training and memory costs, the claimed route to better accuracy at a fixed compute budget remains uncertain.

Questions for authors
  1. Are any completed experiments available for the full method and its main components? If not, please make clear throughout the paper that the proposed accuracy and efficiency benefits remain hypotheses. Results for the memory module and each loss would be especially important to my assessment.

  2. In the exact-sampling setting, what measured source of savings allows better accuracy at a fixed FLOP budget? Please relate this to Equation 4 and include the costs of training and updating memory. If the intended gain is wall-clock speed instead, please distinguish that claim.

  3. Does the intended search controller remain fixed when the draft is updated? If a changed controller or a non-exact proposal rule is needed for quality gains, please separate that setting from the target-distribution guarantee and evaluate it separately.

Reviewer_wC3gOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a test-time training paradigm in which the same model performs reasoning and acts as a speculative verifier, enabling an MCTS-like inference pipeline. The model is then trained at test time on each problem. The paper further provides mathematical discussions of correctness properties and a theoretical cost analysis, but offers no empirical results or supporting evidence.

Critical strengths

The combination of MCTS and test-time training is interesting.

Critical weaknesses
  1. The submission template appears non-standard, raising concerns about whether this is a valid ICLR submission.
  2. The paper provides no empirical results or other evidence to support the proposed method.
  3. The writing and formatting are very unclear.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.