- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes Input-Conditioned Search (ICS), which improves recursive reasoning by adding tentative assignments (Hypothesis injection) or applying equivalent transformations to the input (Validity-preserving restatements), then restarting reasoning without changing model weights. This helps recover from incorrect reasoning trajectories that longer computation or random perturbations may not resolve. Across six reasoning tasks, ICS reports higher accuracy with substantially less inference computation, with hypothesis injection providing most of the improvement.
Critical strengths
- ICS substantially improves accuracy with less inference computation and no additional training.
- Analyses are done thoroughly. The controlled experiments offer a useful design insight that finding a correct solution depends on both the input condition and where reasoning starts.
- The main paper is well organized with a clear motivation.
Critical weaknesses
- The two ICS operations (“hypothesis injection” & “validity-preserving restatements”) rely on task-specific structures and hand-designed rules, making it unclear how well they generalize beyond grid-based problems to more unstructured domains.
- The sensitivity to individual search settings (such as: beam width, search levels, proposals per parent, ...) remains unclear, making it difficult to judge how much tuning is needed for new tasks.
Questions for authors
See weaknesses.
2 comments
Responsed Part 2
W2. How sensitive is ICS to its search settings, and how much tuning does a new task need?
ICS is not quite sensitive to its search settings: what matters is how many hypotheses are evaluated, and this trades compute for accuracy monotonically. We swept every setting on Sudoku-Extreme (on the first 3,072 puzzles of the test set over three seeds; accuracy in %):
| Setting varied | Certificate selection | Model-based selection |
|---|---|---|
| Node budget | 12 to 384: 99.18 to 99.97 | 6 to 96: 98.59 to 99.80 |
| Proposals per parent | 3 to 48: 98.95 to 99.91 | 3 to 24: 98.35 to 99.61 |
| Search levels | 1 to 6: 98.62 to 99.41 | 1 to 8: 98.23 to 98.35 |
| Beam width | 1 to 8: 98.87 to 99.69 | 1 to 6: 97.30 to 98.45 |
| Setting of the paper | 99.91 | 99.61 |
The test-time budget (including node budget, proposals per parent, search levels, and beam width) sets the number of hypotheses, and accuracy rises monotonically with the budget and saturates. A consistent conclusion can be drawn from the similar observations on the other five tasks (Figure 33 in Appendix G). This is what one expects if the gain comes from searching on this axis at all and not from a tuned search strategy: even random positions and values work (W1).
A new task therefore needs little tuning: given a compute budget, an acceptance rule can be chosen on the training or validation set under model-based selection. Under certificate selection, we used one configuration for all six tasks (Table 5). Under model-based selection, search levels and beam width are also the same on all six tasks, and the acceptance rule is chosen by minimizing the error on the training set (Appendix A.6).
More advanced and general search on this axis removes these settings altogether: a learned search policy can decide them itself. Beam width, search levels and proposals per parent belong to the hand-written schedule of our simple heuristic search. A policy that learns to search over has no such schedule: within a compute budget, it decides at every step which hypothesis to probe or to commit next. Recently, we successfully trained a model that learns to search the input conditions without any task-specific knowledge. This model reaches competitive performance against the heuristics in ICS, requiring about only 64% FLOPs of ICS and even demonstrating out-of-distribution generalization capability in our experiments.
Response Part 1
We thank the reviewer for the precise reading and for the positive assessment. In our global response, we state the paper’s main contribution explicitly. We will also state it in the final version, so that it is clearer for future readers. We answer the two concerns raised by the reviewer below.
W1. Do the two operations rely on task-specific structures and hand-designed rules, and do they generalize beyond grid-based problems?
The contribution of the paper is a third test-time scaling axis for recursive reasoners. The two operations are only simple heuristics on this axis, and the axis does not depend on them. We study recursive reasoners as discrete dynamical systems: their latent state evolves as zt+1=F(zt) under an update function F. Depth and width scaling spend compute inside this one system, by iterating F longer or rerunning F from perturbed states, and neither can reliably leave a wrong fixed point of F (Section 5). The third axis scales by changing the system itself: from the same state, the reasoner continues with different update functions F1,F2,…,Fn. A trained reasoner already holds a family F of them, since one set of weights is trained to solve many different problems. However, which Fi∈F leads to a correct solution most efficiently from a given state is not known in advance, so we spend test-time compute on a search over F. In TRM, we can use the input conditions to select the update function (Fx=Fθ(⋅;x)), so hypothesis injection and restatements are two simple ways of running this search through the input.
What the operations take from a task does not need to be designed by hand: which positions to change and which transformations to apply are choices of the search itself, and this search can be learned without task-specific knowledge. Neither operation uses the rules of a task. The replaceable positions and their admissible symbols can be read off the training pairs automatically, and they coincide with our hand-written lists on all six tasks. Choosing among them needs no knowledge either: random positions with random values still reach 98.81% on Sudoku at 59 recursion steps, above depth scaling and PTRM at 6,400 steps (Table 12). Only restatements use a symmetry that we specify (Appendix A.6). Recently, we successfully trained a model that learns to search the input conditions without any task-specific knowledge. This model reaches competitive performance against the heuristics in ICS, requiring about only 64% FLOPs of ICS and even demonstrating out-of-distribution generalization capability in our experiments. We believe learning which representations of the input to try is the natural next step.
Neither the axis nor its mechanism is specific to grids. The mechanism, the removal of a wrong attractor by a change of the update function, is established for a general family of update functions with no assumption on the task (Appendix F.2 of the revised PDF). A hypothesis needs only input positions whose tokens can be replaced by tentative values. In masked diffusion language models (Sahoo et al., 2024; Nie et al., 2025), these are the masked positions. A looped language model (Geiping et al., 2025; Zhu et al., 2026) predicts each next token by iterating the same block of transformer layers on a latent state under a fixed context, which is the discrete dynamical system we study with the context as the input. The search is then over contexts. Going beyond puzzles is mainly a matter of scale ( larger token vocabularies and much longer contexts) but not of principle. We leave these experiments to future work, and the final version will state this.
References
[1] Geiping J et al., Scaling up test-time compute with latent reasoning: A recurrent depth approach, NeurIPS 2025.
[2] Nie S et al., Large language diffusion models, NeurIPS 2025.
[3] Sahoo SS et al. Simple and effective masked diffusion language models, NeurIPS 2024.
[4] Zhu RJ et al., Scaling latent reasoning via looped language models, arXiv 2025.