acceptodds
Under review as a conference paper at ICLR 2027

Input-Conditioned Search for Efficient Test-Time Scaling of Recursive Reasoners

Abstract

People often tackle difficult problems by thinking more deeply or exploring more possibilities. Similarly, recursive reasoners, which reason by repeatedly applying a learned update rule to latent states, can also scale inference through more updates (depth) or additional trajectories generated by perturbing latent states (width). When people hit a dead end in their thinking, they may step back and approach the problem from a fresh angle. However, neither depth scaling nor perturbation-based width scaling of recursive reasoners can reliably achieve this: both may converge to incorrect attractors. To address this, we propose to explore diverse reasoning traces by applying different update maps to the same latent state, without any additional training. We realize this idea through Input-Conditioned Search (ICS), which searches over input conditions to induce alternative update maps from the pretrained reasoner. Across six reasoning tasks, ICS achieves the highest mean accuracy, while requiring up to two orders of magnitude fewer model FLOPs than the evaluated scaling baselines. Controlled interventions confirm that changing the input condition can redirect trajectories from the same state. These findings establish input conditioning as an effective dimension of test-time scaling and open new opportunities for efficient test-time scaling by shaping the update rules of recursive reasoning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Recursive reasoning, Test-time scaling, Reasoning dynamics, Neural algorithmic reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KoVxOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Input-Conditioned Search (ICS), which improves recursive reasoning by adding tentative assignments (Hypothesis injection) or applying equivalent transformations to the input (Validity-preserving restatements), then restarting reasoning without changing model weights. This helps recover from incorrect reasoning trajectories that longer computation or random perturbations may not resolve. Across six reasoning tasks, ICS reports higher accuracy with substantially less inference computation, with hypothesis injection providing most of the improvement.

Critical strengths
  1. ICS substantially improves accuracy with less inference computation and no additional training.
  2. Analyses are done thoroughly. The controlled experiments offer a useful design insight that finding a correct solution depends on both the input condition and where reasoning starts.
  3. The main paper is well organized with a clear motivation.
Critical weaknesses
  1. The two ICS operations (“hypothesis injection” & “validity-preserving restatements”) rely on task-specific structures and hand-designed rules, making it unclear how well they generalize beyond grid-based problems to more unstructured domains.
  2. The sensitivity to individual search settings (such as: beam width, search levels, proposals per parent, ...) remains unclear, making it difficult to judge how much tuning is needed for new tasks.
Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Response Part 1

We thank the reviewer for the precise reading and for the positive assessment. In our global response, we state the paper’s main contribution explicitly. We will also state it in the final version, so that it is clearer for future readers. We answer the two concerns raised by the reviewer below.

W1. Do the two operations rely on task-specific structures and hand-designed rules, and do they generalize beyond grid-based problems?

The contribution of the paper is a third test-time scaling axis for recursive reasoners. The two operations are only simple heuristics on this axis, and the axis does not depend on them. We study recursive reasoners as discrete dynamical systems: their latent state evolves as zt+1=F(zt)z_{t+1} = F(z_t) under an update function FF. Depth and width scaling spend compute inside this one system, by iterating FF longer or rerunning FF from perturbed states, and neither can reliably leave a wrong fixed point of FF (Section 5). The third axis scales by changing the system itself: from the same state, the reasoner continues with different update functions F1,F2,…,FnF_1, F_2, \dots, F_n. A trained reasoner already holds a family F of them, since one set of weights is trained to solve many different problems. However, which Fi∈FF_i \in \mathcal{F} leads to a correct solution most efficiently from a given state is not known in advance, so we spend test-time compute on a search over F\mathcal{F}. In TRM, we can use the input conditions to select the update function (Fx=Fθ(⋅;x)F_x = F_{\theta}(\cdot; x)), so hypothesis injection and restatements are two simple ways of running this search through the input.

What the operations take from a task does not need to be designed by hand: which positions to change and which transformations to apply are choices of the search itself, and this search can be learned without task-specific knowledge. Neither operation uses the rules of a task. The replaceable positions and their admissible symbols can be read off the training pairs automatically, and they coincide with our hand-written lists on all six tasks. Choosing among them needs no knowledge either: random positions with random values still reach 98.81% on Sudoku at 59 recursion steps, above depth scaling and PTRM at 6,400 steps (Table 12). Only restatements use a symmetry that we specify (Appendix A.6). Recently, we successfully trained a model that learns to search the input conditions without any task-specific knowledge. This model reaches competitive performance against the heuristics in ICS, requiring about only 64% FLOPs of ICS and even demonstrating out-of-distribution generalization capability in our experiments. We believe learning which representations of the input to try is the natural next step.

Neither the axis nor its mechanism is specific to grids. The mechanism, the removal of a wrong attractor by a change of the update function, is established for a general family of update functions with no assumption on the task (Appendix F.2 of the revised PDF). A hypothesis needs only input positions whose tokens can be replaced by tentative values. In masked diffusion language models (Sahoo et al., 2024; Nie et al., 2025), these are the masked positions. A looped language model (Geiping et al., 2025; Zhu et al., 2026) predicts each next token by iterating the same block of transformer layers on a latent state under a fixed context, which is the discrete dynamical system we study with the context as the input. The search is then over contexts. Going beyond puzzles is mainly a matter of scale ( larger token vocabularies and much longer contexts) but not of principle. We leave these experiments to future work, and the final version will state this.

References

[1] Geiping J et al., Scaling up test-time compute with latent reasoning: A recurrent depth approach, NeurIPS 2025.

[2] Nie S et al., Large language diffusion models, NeurIPS 2025.

[3] Sahoo SS et al. Simple and effective masked diffusion language models, NeurIPS 2024.

[4] Zhu RJ et al., Scaling latent reasoning via looped language models, arXiv 2025.

AuthorsOpenReview ↗

Responsed Part 2

W2. How sensitive is ICS to its search settings, and how much tuning does a new task need?

ICS is not quite sensitive to its search settings: what matters is how many hypotheses are evaluated, and this trades compute for accuracy monotonically. We swept every setting on Sudoku-Extreme (on the first 3,072 puzzles of the test set over three seeds; accuracy in %):

Setting variedCertificate selectionModel-based selection
Node budget12 to 384: 99.18 to 99.976 to 96: 98.59 to 99.80
Proposals per parent3 to 48: 98.95 to 99.913 to 24: 98.35 to 99.61
Search levels1 to 6: 98.62 to 99.411 to 8: 98.23 to 98.35
Beam width1 to 8: 98.87 to 99.691 to 6: 97.30 to 98.45
Setting of the paper99.9199.61

The test-time budget (including node budget, proposals per parent, search levels, and beam width) sets the number of hypotheses, and accuracy rises monotonically with the budget and saturates. A consistent conclusion can be drawn from the similar observations on the other five tasks (Figure 33 in Appendix G). This is what one expects if the gain comes from searching on this axis at all and not from a tuned search strategy: even random positions and values work (W1).

A new task therefore needs little tuning: given a compute budget, an acceptance rule can be chosen on the training or validation set under model-based selection. Under certificate selection, we used one configuration for all six tasks (Table 5). Under model-based selection, search levels and beam width are also the same on all six tasks, and the acceptance rule is chosen by minimizing the error on the training set (Appendix A.6).

More advanced and general search on this axis removes these settings altogether: a learned search policy can decide them itself. Beam width, search levels and proposals per parent belong to the hand-written schedule of our simple heuristic search. A policy that learns to search over F\mathcal{F} has no such schedule: within a compute budget, it decides at every step which hypothesis to probe or to commit next. Recently, we successfully trained a model that learns to search the input conditions without any task-specific knowledge. This model reaches competitive performance against the heuristics in ICS, requiring about only 64% FLOPs of ICS and even demonstrating out-of-distribution generalization capability in our experiments.

Reviewer_MJj4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies test-time scaling for recursive reasoners such as TRM, which solve puzzles by repeatedly applying a learned update to a latent state. The authors argue that both existing scaling axes, running the same update for longer and perturbing the latent state to obtain several trajectories, keep the input-conditioned update map fixed and therefore tend to fall back into the same incorrect attractors. They propose Input-Conditioned Search, which changes the input instead. Hypothesis injection writes tentative cell assignments, chosen where the decoder margin is smallest, into the input and reruns the frozen model from a fresh initialization, organized as a beam search over partial assignments. Experiments show that perturbing a failed state rarely recovers it, that recovery becomes difficult after only a few recursion steps, and that an injected hypothesis can turn a wrong near-fixed point into a non-stationary state.

Critical strengths
  1. The empirical results are strong and consistent. ICS outperforms every recursive-reasoning baseline on all six tasks under both selection protocols, and it does so at a fraction of their compute. Using the same three TRM checkpoints for ICS, standard TRM, the depth-scaling variants and PTRM makes this part of the comparison clean, and the cost accounting charges every launched rollout, including unsuccessful hypotheses.
  2. Warm-start perturbations of failed states recover essentially nothing at any scale, recovery drops sharply once a trajectory has run for a few steps, and the residual landscapes in Figure 6 illustrate how a hypothesis makes a previously stationary wrong state unstable. These observations are useful beyond the specific method.
Critical weaknesses
  1. The method is closer to classical search than the framing suggests, and the novelty is correspondingly limited. Hypothesis injection is branching on variable assignments with the neural reasoner acting as the propagator, in the spirit of DPLL, and restatements are test-time augmentation over task symmetries, which TRM already uses on ARC. The related work acknowledges both lineages, yet the abstract and conclusion present input conditioning as a new dimension of test-time scaling.
  2. Several aspects of the comparison need clarification before the efficiency claims can be fully trusted. The baselines are evaluated at their large published budgets, whereas ICS stops as soon as a candidate is accepted. Model-based selection in ICS orders candidates by contradictions with the givens and with the injected assumptions, which is a partial form of rule checking that the baselines do not appear to receive. Some retrained baselines also look under-tuned on PPBench.
Questions for authors
  1. Can the authors report the baselines at budgets matched to the average compute actually used by ICS, with the same early stopping rule, so that the efficiency gain attributable to search can be separated from the effect of stopping early?
  2. Do the baselines receive the same restoration of givens and the same consistency ordering that ICS uses under model-based selection? If not, how do their accuracies change when these checks are applied?
  3. How were EqR and GRAM tuned on the PPBench tasks, and can the authors explain why EqR falls far below standard TRM on Tapa and Heyawake?
  4. Since random branching locations perform almost as well as margin-based ones, what in the results specifically supports the view that ICS works by inducing alternative update maps, rather than simply by exhaustive branching with a strong propagator?
  5. Does the approach extend to settings where mutable positions and validity-preserving transformations are not known in advance, for example a looped language model on text reasoning?
5 comments
AuthorsOpenReview ↗

Response Part 1

We thank the reviewer for the precise reading of the paper and the insightful questions. Our global response states the main contribution of this work. We also answer each point raised by the reviewer below.

W1. Novelty relative to DPLL-style branching and test-time augmentation.

The novelty we claim is a third test-time scaling axis for recursive reasoners and its theoretical basis, not the simple operations (hypotheses, restatements, and the random variants) that instantiate the search algorithm on this axis. We study recursive reasoners as discrete dynamical systems: their latent state evolves as zt+1=F(zt)z_{t+1} = F(z_t) under an update function FF. Depth and width scaling spend compute inside this one system by iterating FF longer or rerunning FF from perturbed states, and neither can reliably leave a wrong fixed point of FF (Section 5). The third axis scales by changing the system itself: from the same state, the reasoner continues with various update functions F1,F2,…,FnF_1, F_2, \dots, F_n. A trained reasoner already holds a family F\mathcal{F} of them, since one set of weights is trained to solve many different problems. However, which Fi∈FF_i \in \mathcal{F} leads to a correct solution most efficiently from a given state is not known in advance, so we spend test-time computation on a search over F\mathcal{F}. We also provide the theoretical basis in the revised PDF: changing the update functions can annihilate a wrong attracting fixed point through saddle-node folds. We prove sufficient conditions for this (Appendix F.2), verify them with 51 computer-assisted certificates in trained 4x4 Sudoku reasoners (Appendix D.3), and also find such saddle-node folds by numerical continuation in the Maze-Hard and Sudoku-Extreme models (Appendix D.2).

ICS therefore works for a reason that is essentially different from DPLL-style search: a hypothesis helps by making the reasoner reason differently, not by simply pruning a search tree. The clearest evidence is that wrong hypotheses can succeed. On Sudoku puzzles where the reasoner has converged to an incorrect near-fixed point, 10 to 15% of the hypotheses that contradict the solution still lead to a valid answer, because the injected value changes the update function without being forced into the answer (Tables 9 and 10). In DPLL, a wrong branch can only end in a conflict. The observations that you describe as useful beyond the specific method say the same thing: perturbing a failed state recovers essentially nothing because the update function stays the same, while a hypothesis makes the wrong state unstable because it changes the update function.

The operations themselves are just simple heuristics for a search over F\mathcal{F}, and their simplicity and efficiency demonstrate the effectiveness and the potential of scaling on this axis. Since which update function F∈FF \in \mathcal{F} gives a correct and efficient direction from a state is not known in advance, we formulate it as a search problem. In TRM, the input can select the update function (Fx=Fθ(⋅;x)F_x = F_\theta(\cdot; x)), so hypothesis injection, restatements, and the random variants of Table 12 are all just simple search heuristics over F\mathcal{F} through input conditions. Even the variant with random locations and values reaches 98.81% at 59 recursion steps, above depth and width scaling at 6,400 steps (98.1 to 98.3%). These observations provide promising evidence for developing more advanced searching algorithm/policies on this axis (e.g., models that learn to search over F\mathcal{F} during inference), architectures that explicitly learn various recursive reasoning functions (e.g., zt+1=F(zt;mi),mi∈Mz_{t+1} = F(z_t; m_i), m_i \in M controls “ways of thinking”$), and training objectives considering this axis and desired properties in the discrete dynamical system.

We will revise the paper to state the contributions in these terms in the final version to make them clearer to readers, and will discuss the detailed relations and distinctions with respect to DPLL-style branching and test-time augmentation in the related work.

AuthorsOpenReview ↗

Response Part 2

W2.Part1. and Q1. Baselines at the compute ICS uses, with the same early-stopping rules.

The gain is not an effect of early stopping: at matched compute, and with the same stopping rule, ICS stays significantly outperforming all the baselines across all the tasks (apart from Light-Up, where several methods achieve about 100% acc.) and under both answer selection settings (Show in Table below or Table 19 in Appendix G of the revised PDF). We run the baselines with the same early-stopping rules as ICS and report their best performance on Maze-Hard, Nurikabe, Tape, and Heyawake (TRM + greedy depth on Sudoku-Extreme already uses the same early-stopping rules as ICS, and most methods almost solve the Light-Up already, so these two tasks are not reported). The observation is that the adaptive stopping reduces the average puzzle cost of the baseline at the cost of lower performance, while ICS consistently and significantly outperforms all other scaling axes with the same trained backbones.

TaskSelectionICSTRM depth, same stopPTRM adaptive, same stopPTRM fixed, K × 64
Mazecertificate99.8 @ 16791.8 @ 8591.9 @ 12591.6 @ 128
Mazemodel-based99.1 @ 17891.1 @ 1891.1 @ 14191.6 @ 128
Nurikabecertificate95.7 @ 25084.6 @ 15985.8 @ 21484.3 @ 128
Nurikabemodel-based88.1 @ 13580.9 @ 10981.4 @ 8184.3 @ 128
Tapacertificate95.9 @ 40875.4 @ 25476.9 @ 32674.9 @ 320
Tapamodel-based85.1 @ 20266.7 @ 19867.2 @ 13274.9 @ 128
Heyawakecertificate99.1 @ 18587.8 @ 12788.7 @ 18188.1 @ 128
Heyawakemodel-based93.9 @ 12283.8 @ 10484.6 @ 7687.8 @ 64

The gain is attributable to the search: puzzles that the reasoner does not solve in 2.4 million steps with depth and width scaling can be solved by ICS in about 2 to 4 steps under the “right” update function FF. On Sudoku-Extreme, several puzzles remain unsolved under depth and width scaling after more than 2.4 million recursion steps, while about 94% of them can be solved by ICS within 4 recursion steps under a ``correct’’ update function Fx~F_{\tilde{x}} (most $\tilde{x} involves only a single hypothesis injection). More than 98% of the compute that ICS spends on these puzzles goes into finding this update function ( which is not known in advance) but not into reasoning with it. We will also add this evidence to the final version.

W2.Part2 and Q2. Are the restoration of givens and the consistency ordering a partial form of rule checking that the baselines do not receive?

The restoration of givens and consistency ordering check a candidate against its own input, not against any task rules. It ranks candidates by contradictions with the hypothesis that the search itself injected and with the givens, and then by halting score (Eq. 7). No rule of the task is evaluated (e.g., it never asks whether a digit repeats in a Sudoku row), so a candidate that passes can still be invalid.

The baselines also receive the restoration of givens but not the consistency ordering. Applying consistency ordering to baselines changes no outcome. Since the baselines already receive the givens restoration, we additionally applied consistency ordering to them. The baselines use depth and width scaling with no hypothesis injection, so only the comparison with the givens remains. The consistency ordering does not change any baseline’s performance across all the tasks.

Even the complete rule check does not close the gap: using the same backbone(TRM), depth scaling and width scaling under certificate answer selection are still below ICS with model-based selection. The certificate is the complete rule check for one task, so for the same candidates no selection rule can return more valid answers. With the certificates and with 6,400 recursion steps on every puzzle, TRM+depth scaling and PTRM remain 1 to 10 points below ICS under model-based selection on all five tasks that the backbone does not already solve (Table 1).

AuthorsOpenReview ↗

Response Part 3

W2 and Q3. Are EqR and GRAM under-tuned on PPBench? Why does EqR fall far below standard TRM on Tapa and Heyawake?

The gaps are real and not an effect of under-tuning: they demonstrate that several baselines failed to generalize across the six tasks, which is further evidence for the generality of ICS. We have put substantial effort and resources into getting the best performance out of every baseline across all six tasks. Without changing their architectures, we worked on two fronts: training recipes and test-time scaling strategies. For training recipes, we have tried extending training epochs up to 26 times the TRM backbone training epochs and tuning hyperparameters (learning-rate schedule, weight decay, etc.). For test-time scaling strategies, we tried cold-start, warm-start and hybrid perturbation and tuned the relative perturbation scales. These improved the baselines considerably over our first runs with the official recipes (e.g., on Heyawake, Eqr is improved from 9.9% to 18.8%; On Tapa, GRAM is improved from 21.5% to 43.1%). The baselines’ performance reported in the main table is already the best performance in all the recipes we have tried.

Given the evidence we have, we believe the remaining gaps come from the methods themselves. For EqR, the same implementation is competitive on Sudoku and Maze while falling far below TRM on Tapa and Heyawake, so we attribute the gaps to its architecture but not to the training and test-time scaling recipes. GRAM injects model-generated perturbations during training and has to fit many more perturbed states, which naturally requires more training data than standard TRM, and Tapa has the smallest training set of the four PPBench tasks. Consistently, GRAM is above standard TRM on Nurikabe and Heyawake, the two tasks with the most training data. We would be glad to work with the reviewers on the deeper reasons, and we will release the code and trained weights of all methods upon acceptance so that the community can study related problems.

Q4. What supports alternative update maps, rather than exhaustive branching with a strong propagator?

A propagator can only complete a correct assumption, while ICS also succeeds with wrong assumptions and with no assumption at all. As noted under W1, 10 to 15% of the hypotheses that contradict the solution still lead to a valid answer (Tables 9 and 10), while propagating that value could never produce this. Restatements involve no branching and assume nothing about the solution, yet from a fresh start they recover 35% of the Sudoku failures at an incorrect near-fixed point and 72% of those on Maze, against 6% and 0% for the original input (Table 14). Both results are expected if what matters is that the reasoner runs under a different update function.

The change of the update function is visible directly in the dynamics of the trained models. At the wrong fixed point of Figure 6, injecting the hypothesis raises the relative one-step update residual from 2.410-7 to 0.225: the state where the original update function is stuck is not a fixed point of the new one. Numerical continuation in the Maze-Hard and Sudoku-Extreme models shows how this happens: as the input embedding moves from the original input to the hypothesis, the wrong attracting fixed point collides with an unstable one and both disappear (Saddle-node bifurcation or fold, Appendix D.2).

The theory and the certified experiments added in the revised PDF establish this mechanism rigorously. We prove sufficient conditions for this annihilation of a wrong attracting fixed point (Appendix F.2) and verify them with computer-assisted proofs in 51 cases on separately trained 4×4 Sudoku reasoners (Appendix D.3). Just past the certified fold, the reasoner reaches the correct solution from the very state where it was stuck in 31 of the 42 cases. Nothing in this mechanism involves branching or propagation: the wrong attractor does not exist under the new update function.

The search is also far from exhaustive, and random locations doing almost as well is what this view predicts. Model-based ICS spends 61 recursion steps per Sudoku puzzle on average, fewer than four standard 16-step runs, and on the puzzles that the original update function never solves (we run more than 2.4M recursion steps with the original update function with depth and width scaling), a single level of hypotheses almost always suffices (W2 and Q1). Any location changes the update function, so where the hypothesis is placed matters much less than that the update function changes.

Reviewer_Zb9tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Input-Conditioned Search (ICS). It's a training-free method to improve reasoning performance for recursive reasoning model. The idea is that when a recursive reasoning model is stuck at an incorrect answer, depth scaling or perturbation-based with scaling cannot help much,. Instead, the model should explore diverse training traces by applying different update maps to the latent. The author implement this through hypothesis injection and validity-preserving restatements. Using Tiny Recursive Models, ICS achieves 99.70% accuracy on Sudoku-Extreme at 1.14 TFLOPs per puzzle.

Critical strengths
  1. ICS shows strong empirical improvement with substantially lower inference compute. ICS outperforms baseline across six reasoning benchmarks, under both certificate-based and model-based answer selection. In Sudoku-Extreme, the model achieves 99.70% accuracy at 1.14 TFLOPs per puzzle, a 92% reduction in errors relative to the baseline at comparable compute.
  2. The paper is clearly written, and the graphs are organized cleanly.
Critical weaknesses
  1. Limited evidence of method generalization. All six benchmarks are grid-based constraint-satisfaction problems with explicitly defined variables and available task-specific transformations. Also hypothesis injection and validity-preserving restatements depends on human specified knowledge of the problem itself.
  2. The Sudoku ablations in Appendix C.4 show that randomizing both hypothesis locations and values still achieves 98.81% accuracy. This suggests that simpler search over modified inputs might be cleaner than model-guided approach. It weakens the claim of advantages from the paper's proposed search strategy.
Questions for authors
  1. How important is training-time access to partial puzzle states for the effectiveness of hypothesis injection? In particular, can the authors evaluate ICS on backbone models trained without partial-state augmentation?
  2. Could the authors provide accuracy-versus-FLOPs curves for all 6 tasks?
4 comments
AuthorsOpenReview ↗

Response Part 1

We thank the reviewer for the clear summary and the two questions, both of them led to new experiments discussed below.

W1. Evidence of generalization and dependence on human-specified knowledge.

In principle, neither the problems ICS solves nor its mechanism is specific to grid puzzles, and the mechanism has a theoretical basis. ICS addresses a recursive reasoner captured by a wrong attractor of its update function, by changing that function through the input conditions. This mechanism is established for a general family of update functions, with no assumption on the task (Appendix F.2 of the revised PDF). A hypothesis needs only input positions whose tokens can be replaced by tentative values (Section 3). In our puzzles these are the blank cells. In masked diffusion language models (Sahoo et al., 2024; Nie et al., 2025) they are the masked positions, which are filled step by step to generate text. More broadly, a looped language model (Geiping et al., 2025; Zhu et al., 2026) predicts each next token by iterating the same block of transformer layers on a latent state under a fixed context, which is the same discrete dynamical system we study, with the context as the input. On such architectures this work may therefore open a direction that is not yet fully explored: search over contexts during next-token prediction. Going beyond puzzles is mainly a matter of scale (i.e., larger token vocabularies and longer contexts) and not of principle. We leave these large-scale experiments to future work, and the final version will state this.

In practice, the six tasks already give strong empirical evidence that ICS generalizes across tasks. Although all six are grid puzzles, the evidence is strong because they differ enough that the gains of the other methods do not generalize across them. The evaluated baselines with different backbones failed to generalize across the six tasks: GRAM, EqR and Attractor each fall more than 10 points below standard on at least one task (Table 1). With the same backbone, both depth scaling and width scaling on TRM gain at most 3.6 points with large model FLOPs. ICS is the only one method that keeps its gain significant (and achieving the best acc.) on every task at up to two orders of magnitude less compute than other baselines (Table 1, Figure 3, Appendix G).

Only restatements require the knowledge we specify, and both operations are simple heuristics that motivate future search algorithms requiring no task knowledge along this proposed scaling axis. Neither hypothesis injection nor restatements uses the rules of a task. For hypotheses, which positions are blank and which symbols may fill them need not be specified: read off the training pairs automatically, they coincide with our hand-written lists on all six tasks. For restatements, we do specify a symmetry: rotations and reflections of the grid, and the standard symmetries of Sudoku. The hypothesis injection, restatements, and the random variant are all deliberately simple heuristics on the proposed scaling axis. Their simplicity and effectiveness do not weaken the proposed scaling axis but are the strongest evidence for this efficient test-time scaling axis. They demonstrate the large potential of developing advanced general search algorithms/policies on this axis and motivate promising architecture and training objective design considering our scientific discoveries.

References

[1] Geiping J et al., Scaling up test-time compute with latent reasoning: A recurrent depth approach, NeurIPS 2025.

[2] Nie S et al., Large language diffusion models, NeurIPS 2025.

[3] Sahoo SS et al. Simple and effective masked diffusion language models, NeurIPS 2024.

[4] Zhu RJ et al., Scaling latent reasoning via looped language models, arXiv 2025.

AuthorsOpenReview ↗

Response Part 2

W2. Random locations and values still reach 98.81%. Does this weaken the advantage of the proposed method?

This result supports the paper's main claim instead of weakening it: the claim is the scaling axis, not a particular search strategy. The main contribution of this work is a third test-time scaling axis of recursive reasoners and its theoretical basis.(please refer to the golbal response and Appendix F.2, D.2, D.3 in the revised PDF for more details and the theory) We study a recursive reasoner as a discrete dynamical system: its latent state evolves as zt+1=F(zt)z_{t+1} = F(z_t) under an update function FF. Depth and width scaling spend compute inside this one system by iterating FF longer or rerunning FF from perturbed states, and neither can reliably leave a wrong fixed point of FF (Please refer to Section 5). This work introduces the third scaling axis that changes the system itself: from the same latent state ztz_t, continue with different update functions F1(zt),F2(zt),…,Fn(zt)F_1(z_t), F_2(z_t), \dots, F_n(z_t), so that the reasoning proceeds in different directions. A trained recursive reasoner already contains a family F\mathcal{F} of such update functions: a single set of weights is trained to solve different problems, thus a lot of update functions coexist in it and ordinary inference uses only one F∈FF \in \mathcal{F}. However, which Fi∈FF_i \in \mathcal{F} leads to a correct solution most efficiently from a given state is not known in advance, so we spend test-time computation on a search over F\mathcal{F}.

ICS and its variants are just simple heuristics on this axis, demonstrating the promising efficiency and potential of this scaling axis compared with the depth and width scaling. In TRM, both the input condition xx and the weights θ\theta are the parameters of the dynamical system zt+1=Fθ(zt;x)z_{t+1} = F_{\theta}(z_t; x). While θ\theta is fixed to preserve the desired properties learned during training, the input condition xx can be used to select the update function Fx∈FF_x \in \mathcal{F} without additional training. Therefore, search over F\mathcal{F} can be carried out as a search over input conditions on TRM, i.e., Input-Conditioned Search (ICS). Hypothesis injection search, restatements and the two random variants in Table 12 are all just simple heuristics for ICS. With these simple heuristics, ICS achieves the highest accuracy across the six tasks while requiring up to two orders of magnitude fewer FLOPs than the scaling baselines. Even hypotheses at random locations with random values reach 98.81% at 59 recursion steps, above depth and width scaling at 6.4k steps. The observation that such simple heuristics already bring these gains does not weaken the paper’s claims. Instead, it is the strongest evidence for the efficiency and the potential of the proposed scaling axis. A search with random locations and random values is still a search over input conditions. If the advantage of ICS came from its search strategy, this random search would lose it. Instead, it achieves 98.81% at 59 recursion steps, which is still above depth scaling and PTRM at 6,400 steps (Table 1). The advantage therefore comes from the axis itself. It also opens several future directions, including stronger search algorithms on this axis (e.g., models that learn to search over update functions), architectures that explicitly learns various recursive update functions Fi∈FF_i \in \mathcal{F}, and training objectives formulated on the dynamical system (e.g., ones that eliminate wrong attractors or converges faster). In our recent experiments, a model that learns to search the update functions without any task rules reaches comparable performance against ICS while requiring fewer FLOPs, further demonstrating the potential of this scaling axis.

Model guidance is still the better heuristic on this axis, and it matters most when no certificate is available. Without a certificate, it raises accuracy from 98.81% to 99.81% at the same compute, cutting the error rate from 1.19% to 0.19%. With a certificate, the random search is almost as good, 99.85% against 99.94% (Table 12). We will state this reading explicitly in the final version.

AuthorsOpenReview ↗

Response Part 3

Q1. How important is training-time access to partial puzzle states for the effectiveness of hypothesis injection?

ICS is effective both with and without it. The partial-state augmentation is used only on the PPBench tasks (Appendix A.2), following the training recipe of PTRM to compensate for small training sets. The Sudoku-Extreme and Maze-Hard backbones are trained without it, and ICS gains significantly over standard TRM under both settings.

A controlled experiment on Heyawake further confirms this. This question interested us as well, so we trained Heyawake backbones with everything unchanged except that the augmentation is removed using three different seeds. The corresponding results are shown in the following table and in Table 20 (Appendix G of the revised PDF). Removing the partial state augmentation, the performance of the trained backbone is surprisingly improved a little bit, and ICS leads the best baseline with a significant gap under both training settings and selection modes.

MethodSelectionWith augmentationWithout augmentation
Standard TRMcertificate86.4 ± 3.389.5 ± 2.5
TRM + depth scaling (6,400 steps)certificate87.8 ± 1.590.9 ± 1.3
PTRM (100 × 64 steps)certificate88.7 ± 1.590.7 ± 2.6
ICScertificate99.1 ± 0.097.4 ± 0.9
Standard TRMmodel-based86.4 ± 3.389.5 ± 2.5
TRM + depth scaling (6,400 steps)model-based84.3 ± 3.086.5 ± 2.7
PTRM (100 × 64 steps)model-based88.7 ± 1.590.6 ± 2.5
ICSmodel-based94.5 ± 1.093.4 ± 1.3

Q2. Accuracy-versus-FLOPs curves for all six tasks. Yes, we have conducted the experiments and included the curves in the revised PDF (Appendix G, Figure 33): on every task that the backbone does not already solve, no baseline reaches the accuracy of ICS at any budget we ran. The exception is Light-Up, where ICS and several methods tie at 100%. We will also release the code and trained weights of all the investigated methods upon acceptance, so that the community can reproduce the results and study more in-depth problems easily.

Dear Reviewer Zb9t, thank you again for your feedback.

We have provided the requested controlled experiment without partial-state augmentation in Table 20 and accuracy–FLOPs curves for all six tasks in Figure 33 of the revised PDF. Our response also clarifies the main contribution, provides the theoretical basis, and quantifies the improvement from model guidance over random search under model-based selection.

As the discussion period closes, could you please let us know whether any decision-critical concerns remain after considering this evidence?

We would appreciate the opportunity to clarify them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.