acceptodds
Under review as a conference paper at ICLR 2027

Best-of-Depth: Learning Exploratory Dynamics in Looped Transformer

Abstract

Looped Transformers increase inference-time computation through repeated latent-state updates with shared parameters. We propose training these updates to increase the utility of candidate solutions along a single trajectory. Our method, Best-of-Depth (BoD), minimizes the loss of the best prediction across depths, encouraging a match to the target anywhere within the depth budget without architectural changes. Experiments on N-Queens completion and graph coloring show improved solution discovery under greedy decoding relative to training that supervises only the prediction at the maximum recurrent depth. On N-Queens, new solutions continue to emerge beyond the training depth before discovery saturates. Code-generation experiments also show higher success across evaluated depths than maximum-depth training. Fixed recurrent-step-budget comparisons favor combining depth with multiple trajectories. These findings show how training a recurrent trajectory as a candidate set enables depth-wise solution discovery.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Looped Transformer, Large Language Models, Search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uTHaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Best-of-Depth (BoD), a training objective for looped Transformers that treats successive recurrent readouts along a single latent trajectory as a candidate set. Rather than supervising only the final depth (Last-Depth) or averaging losses across depths (Mean-Depth), BoD minimizes the minimum loss across all supervised depths. Without architectural changes, this encourages the model to visit distinct valid solutions across recurrent steps rather than collapsing to a single fixed point. Experiments on synthetic multi-solution tasks (N-Queens completion, graph coloring) and LLM code generation fine-tuning (Huginn-3.5B on HumanEval and MBPP) show consistent improvements in Success@Depth and Coverage@Depth.

Critical strengths
  1. Simple, Principled, and Well-Executed Idea: Bringing Multiple Choice Learning (MCL) to the recurrent depth axis of looped Transformers is natural and requires zero architectural modifications. The symmetry-breaking analysis (Proposition 3) clearly illustrates why Last-Depth, Mean-Depth, and Soft-Depth average out multi-modal targets while BoD preserves distinct solutions along a recurrent orbit.
  2. Insightful Mechanistic & Ablation Studies: The withheld-solution experiment (Sec. 4.5.3, Table 2) and trajectory visualizations (Sec. 5, Fig. 4) convincingly demonstrate that BoD learns exploratory latent dynamics that continue moving across valid solution basins rather than converging to a static fixed point.
  3. High Reproducibility and Rigor: The experimental controls (isomorphism/symmetry split separation, canonical coloring equivalence classes, temperature sweeps, and complete appendices) are meticulous and transparently documented.
Critical weaknesses
  1. The mechanism may not transfer directly from multi-target tasks to code fine-tuning. The combinatorial tasks repeatedly pair an input with different valid targets, whereas each CodeFeedback example has one teacher response and is used for one fine-tuning epoch. In this single-target setting, why should BoD induce multiple valid solutions for the same input rather than learn a preferred depth? The one-teacher graph-coloring ablation is suggestive, but does not establish the mechanism for code generation.
  2. Code generation lacks a compute-matched sampling baseline. The six evaluated depths require separate complete autoregressive generations. Success@32 establishes that at least one candidate passes the tests, but does not show whether depth-wise candidates outperform six stochastic samples at a fixed depth—or how a correct candidate would be selected in practice. Variation across independent fine-tuning runs is also unreported, and MBPP uses prompt-visible public tests.
  3. LLM training overhead and gradient truncation are insufficiently characterized. BoD scores all 32 depths, recomputes the winning depth, and backpropagates through at most its last eight recurrent updates. The relative wall-clock time and peak memory versus Last-Depth, and whether the two methods use the same truncation policy, should be clarified.
Questions for authors
  1. At comparable inference cost, how does the six-depth BoD candidate set compare with six stochastic samples from Last-Depth at a fixed depth?
  2. What evidence distinguishes within-trajectory solution diversity from per-input depth selection under single-target training?
  3. What are the respective fine-tuning time, peak memory, and backpropagation ranges?
3 comments

We thank the reviewer for the positive assessment and precise questions. We answer each point below.

W1: The mechanism may not transfer directly from multi-target tasks to code fine-tuning. ... In this single-target setting, why should BoD induce multiple valid solutions for the same input rather than learn a preferred depth? ...

Q2: What evidence distinguishes within-trajectory solution diversity from per-input depth selection under single-target training?

Code problems do admit multiple valid solutions; what is single in CodeFeedback is the supervision, one teacher response per problem. With one target per example, each training example updates only the depth that fits it best, so different depths are trained on different examples and come to produce different programs, giving complementary candidates for the same problem. The evidence that this is within-trajectory diversity, not only per-input depth selection, is that one trajectory often contains several distinct correct programs for the same problem. After removing docstrings and renaming identifiers, BoD's six depth-wise candidates contain two or more distinct correct programs for 32 HumanEval and 127 MBPP problems, versus 21 and 83 for Last-Depth. If BoD only selected one suitable depth per input, the other depths would not add distinct correct programs. The average number of distinct programs among the six candidates is also highest for BoD (4.54 on HumanEval and 4.45 on MBPP, versus 4.43 and 4.28 for the base model and 4.10 and 3.96 for Last-Depth). BoD also does not settle on one preferred depth: during fine-tuning, the selected depths of the middle 80% of examples span 13–31, and this spread is the same at the start and end of training.

W2: Code generation lacks a compute-matched sampling baseline. ... Success@32 establishes that at least one candidate passes the tests, but does not show whether depth-wise candidates outperform six stochastic samples at a fixed depth—or how a correct candidate would be selected in practice. ... MBPP uses prompt-visible public tests.

Q1: At comparable inference cost, how does the six-depth BoD candidate set compare with six stochastic samples from Last-Depth at a fixed depth?

The recurrent block, which runs dd times per token at depth dd, dominates inference cost. BoD's six candidates (depths 4, 8, 12, 16, 24, 32) therefore cost 4+8+12+16+24+32=964+8+12+16+24+32 = 96 recurrent steps per token, the same as three samples at depth 32 (3% more counting all layers); six samples would cost twice as much. We compare with three samples from the paper's Last-Depth model at depth 32 (temperature 0.8, top-pp 0.95).

Problems solved by at least one candidate (%), equal compute, the paper's models:

HumanEvalMBPP
Last-Depth (three samples at depth 32, pass@3)36.740.6
Best-of-Depth (one greedy program per depth, Success@32)37.844.2

At matched recurrent compute, BoD's six depth-wise candidates solve more problems than three samples from Last-Depth on both benchmarks. A correct candidate is selected by running tests, as in pass@k. Prompt-visible tests are the standard MBPP protocol (Austin et al., 2021); HumanEval uses hidden tests and shows the same trend.

W3: LLM training overhead and gradient truncation are insufficiently characterized. ... The relative wall-clock time and peak memory versus Last-Depth, and whether the two methods use the same truncation policy, should be clarified.

Q3: What are the respective fine-tuning time, peak memory, and backpropagation ranges?

Measured back to back on the same 8×H200 node (median over 20 updates):

Time per updatePeak memory per GPU
Last-Depth2.65 s68.1 GB
BoD2.83 s (1.07×)68.6 GB

Both use the same truncation: gradients flow through at most the last eight recurrent updates of the supervised depth (Last-Depth: updates 25–32 of depth 32; BoD: the last min(d, 8) updates of the selected depth d). BoD scores all 32 depths along one trajectory without gradients and keeps the recurrent states. Its only extra work over Last-Depth is therefore the readout at each depth.

We would be glad to clarify anything further.

Reviewer_uTHaOpenReview ↗

Response

Thank you for your explanation. I will keep my rating.

Thank you for reviewing our responses and for your constructive feedback. We appreciate your time and consideration.

Please let us know if any further clarification would be helpful.

Reviewer_D7FbOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents Best-of-Depth, a new method to train looped Transformer models. It takes inspiration from Stochastic Multi-Choice Learning and Explorative Modeling, and optimizes the loss at the optimal depth, i.e., the minimum loss among the per-depth candidates. The authors test their approach with a set of synthetic tasks and measure the coverage (% correct greedy decoded solutions) and success (presence of at least one correct greedily decoded solution) at different depth levels. They also fine-tune a 3.5B looped Transformer (Huggin) language model and evaluate it on code generation benchmarks (HumanEval, MBPP). The method shows improved performance in these exploratory metrics compared to common depth-based baselines.

Critical strengths
  • Strong performance on synthetic tasks. BoD achieves substantial improvements in solution coverage on N-Queens and graph coloring, with some further discoveries beyond the training depth.
  • Timely topic. The paper addresses a relevant question for looped Transformers: how to train models to make better use of additional inference-time computation.
Critical weaknesses
  • Limited evidence at a larger scale. Though the results on toy problems such as N-Queens look promising, the improvements seem much less convincing on the larger model. For code generation, the difference between Last-Depth and Best-of-Depth on HumanEval is approximately 3 percentage points, corresponding to only five problems out of 164. Considering that the authors evaluate one greedy generation per problem at each depth and do not report uncertainty across training runs, this does not provide convincing evidence of a statistically reliable improvement.
  • Disconnect between training and inference. Though the method may achieve high Success@Depth, at inference time the true label is unknown, so the best depth cannot be selected using the training loss. The practical meaning of this metric is therefore unclear when a reliable verifier is unavailable. An evaluation with an explicit selection mechanism would make the practical benefit clearer.
  • Structure and clarity. The text requires polishing and stronger logical connections between sections. In particular, I found Section 4.5.3 difficult to follow: its setup and the purpose of the experiment could be explained more clearly. The interpretations of the theoretical propositions are also too brief to make their relevance clear. For example, what does E[L]=Pr⁡(τ>D)\mathbb{E}[\mathcal{L}]=\Pr(\tau>D) tell us about the method’s behavior, and why study exact-match loss when the model is trained with cross-entropy? The text feels heavily AI-generated, with formal statements receiving more attention than their motivation and interpretation.
Questions for authors
  1. Could the authors evaluate voting across depths, or another candidate-selection mechanism that does not require the ground-truth label, to show whether improvements in Success@Depth and Coverage@Depth translate into better accuracy for a single selected answer?
  2. Could the authors assess the statistical significance of the code-generation improvements, particularly on HumanEval, through paired significance tests and repeated fine-tuning runs, with confidence intervals?
4 comments

We thank the reviewer for the detailed review. We answer each point below.

W1: Limited evidence at a larger scale. ... For code generation, the difference between Last-Depth and Best-of-Depth on HumanEval is approximately 3 percentage points, corresponding to only five problems out of 164. ... this does not provide convincing evidence of a statistically reliable improvement.

Q2: Could the authors assess the statistical significance of the code-generation improvements, particularly on HumanEval, through paired significance tests and repeated fine-tuning runs, with confidence intervals?

We fine-tuned all five objectives with two additional seeds (same data and hyperparameters), giving three seeds per objective together with the run in the paper.

Success@32 (%), mean ± SD over three seeds:

HumanEvalMBPP
Last-Depth34.6 ± 0.438.7 ± 1.1
Mean-Depth33.1 ± 1.638.5 ± 1.5
Random-Depth31.7 ± 1.038.6 ± 1.1
Soft-Depth33.1 ± 1.639.9 ± 1.3
Best-of-Depth38.2 ± 1.242.9 ± 1.8

In all three seeds, BoD's Success@32 is higher than that of every other objective on both benchmarks. A statistical test of its gain over Last-Depth gives +3.7 points on HumanEval (95% CI [+1.4, +6.3], p = 0.006) and +4.2 points on MBPP ([+2.4, +6.1], p < 0.0001).

W2: Disconnect between training and inference. ... at inference time the true label is unknown, so the best depth cannot be selected using the training loss. The practical meaning of this metric is therefore unclear when a reliable verifier is unavailable. ...

Q1: Could the authors evaluate voting across depths, or another candidate-selection mechanism that does not require the ground-truth label, to show whether improvements in Success@Depth and Coverage@Depth translate into better accuracy for a single selected answer?

We would like to clarify that BoD targets settings in which candidates can be verified, as in Best-of-N sampling and pass@k. It does not select a depth at inference, and the training-time minimum only shapes the trajectory. All readouts form a candidate set, and choosing among them needs no ground-truth label. We use constraint checks for N-Queens and graph coloring, and tests for code. Success@Depth is then the accuracy of the single answer returned by this check. Voting assumes candidates agree on one correct answer, whereas BoD deliberately produces different solutions at different depths, so agreement is not a useful signal. A label-free alternative is to return the readout at the most confident depth, where confidence is the maximum log-probability averaged over the predicted positions. BoD keeps its advantage.

Validity (%) of the answer at the most confident depth, without a verifier (multi-solution test inputs, three seeds):

Graph coloringN-Queens
Last-Depth30.023.5
Mean-Depth49.251.8
Random-Depth48.554.8
Soft-Depth80.855.5
Best-of-Depth88.176.2

W3: Structure and clarity. ... I found Section 4.5.3 difficult to follow: its setup and the purpose of the experiment could be explained more clearly. ... what does E[L] = Pr(τ > D) tell us about the method’s behavior, and why study exact-match loss when the model is trained with cross-entropy? ...

We revised the text to give the motivation and interpretation of each result before its formal statement, in particular for Section 4.5.3 and the propositions (changes in blue). We also moved the evaluation metrics and the limitation of Last-Depth training into a new preliminaries section (Section 2), and moved related work to Section 6.

Section 4.5.3 asks whether, on the same problems as in training, BoD finds new solutions that were not shown during training. It tests whether BoD learns to explore rather than to reproduce the solutions it was taught. To test this, we train with only kk of the valid solutions of each training graph as targets and count how many of the other solutions BoD finds. BoD finds 29.4% of them with k=1k = 1 and 46.2% with k=4k = 4, the highest among the compared methods (Table 2).

Regarding the propositions, E[L]=Pr⁡(τ>D)\mathbb{E}[L] = \Pr(\tau > D) means that BoD is penalized only if the target is never reached within the depth budget. Once it is reached, the trajectory is free to move on, so one trajectory can visit several solutions. We analyze exact-match loss because Success and Coverage count only candidates that exactly match a valid solution, so the link is exact. These identities hold exactly for exact-match loss. For cross-entropy, lowering BoD's loss raises a lower bound on coverage (Appendices D.2 and D.4).

We hope these answers address your concerns, and we would appreciate it if you could reconsider your assessment in light of these results. We would be glad to clarify anything further.

Thank you for the additional fine-tuning runs and for clarifying the evaluation metrics, the theoretical results, and Section 4.5.3. The consistent gains across seeds make the HumanEval results more convincing. My remaining concern is about the scope of the evidence. The HumanEval results show that BoD is more likely to generate a correct candidate, and the confidence-based results show that selection without a verifier can work on synthetic tasks. It remains unclear how much of the HumanEval gain carries over to a single returned program when the benchmark tests are unavailable for selection.

I’m therefore raising my score from clear reject to weak reject.

Thank you for raising your score. When the benchmark tests are unavailable, the input-output examples written in each problem statement can still check the depth-wise candidates. As in AlphaCode (Li et al., 2022), we filter the candidates with these examples. We return the deepest of the six candidates that reproduces them, and the depth-32 program if none does or the problem has no examples. The hidden tests are used only to score the returned program, and the same rule is applied to every objective.

Pass rate (%) of a single returned program on HumanEval, mean ± SD over three seeds:

No selection (depth 32)Selection by problem-statement examples
Last-Depth27.0 ± 0.931.9 ± 1.5
Mean-Depth27.8 ± 0.731.5 ± 1.5
Random-Depth26.4 ± 2.129.7 ± 0.4
Soft-Depth25.8 ± 0.931.1 ± 0.6
Best-of-Depth29.9 ± 0.634.8 ± 1.6

Selecting among BoD's depth-wise candidates raises its single-program pass rate by 4.9 points, and BoD is the best objective under the same rule in all three seeds. Success@32 checks all six candidates against the hidden tests, and BoD's single returned program even exceeds the Success@32 of every other objective (at most 34.6%).

We hope this addresses your remaining concern, and we would appreciate it if you could consider raising your score further.

Thank you again for your constructive feedback.

As the discussion period is nearing its end, we would appreciate it if you could take a look at our additional results addressing your remaining concern. We hope these results help resolve your concern, and we would be grateful to hear your thoughts.

Reviewer_nFhwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

the paper introduces a novel training objective for looped Transformers, termed BoD, which aims to improve output coverage across recurrent depths by minimising the lowest prediction loss. Focusing primarily on tasks with multiple valid answers, the authors theoretically establish a connection between BoD and solution coverage, and construct a case where BoD outperforms other typical objectives. Empirical studies on combinatorial tasks further support this idea, showing significant improvements in both Coverage@Depth and Success@Depth. Additionally, the authors evaluate BoD on code-generation tasks and conduct several ablation studies to better understand its exploratory recurrent dynamics.

Critical strengths
  1. The paper is well written and well organised, and the problem setting is also clearly formulated

  2. The paper provides an interesting perspective on an important question for looped Transformers: the choice of training objective. I find the idea of optimising solution discovery across depths, rather than just accuracy at a single depth, quite interesting.

  3. The authors provide theoretical justification for the proposed objective, establishing a connection between BoD and solution coverage.

  4. The empirical analysis is fairly comprehensive, including combinatorial tasks that are closely related to the theoretical motivation, as well as code-generation tasks that extend beyond these controlled settings. I also appreciate that the authors do not just show performance improvements, but further investigate the exploratory recurrent dynamics to better understand the properties of the proposed method.

Critical weaknesses
  1. although the paper focuses on the potential of solution discovery across depths, measured by Success@Depth and Coverage@Depth, I think it is also important to consider how easily we can identify the correct depth or candidate at inference time. like, how sensitive are the predictions across different depths? For example, if only a few depths produce correct answers while the others are incorrect, it might be difficult to select the correct answer without an external verifier. It would be interesting to further discuss the trade-off between exploratory and stable recurrent trajectories.

  2. the main motivation and advantage of BoD are based on solution coverage across depths. However, in many tasks, there is only one valid answer. i am wondering whether BoD would still be beneficial in such settings, especially compared to objectives that encourage accurate predictions at a particular depth. It would be interesting to see how BoD performs on tasks with a unique valid answer.

  3. I think the theoretical analysis is somewhat limited. In particular, Proposition 3 is based on a constructed setting where the recurrent dynamics are already designed to cover all valid answers. it mainly shows that BoD can take advantage of such dynamics, rather than explaining how BoD can learn these exploratory dynamics from training. It is also unclear under what conditions BoD would outperform other objectives, or whether there are settings where other objectives might perform better.

  4. another concern is that the performance of BoD might be sensitive to the target distribution. Most of the controlled experiments consider uniformly sampled valid answers, which also aligns with the theoretical analysis. However, in real-world tasks, the distribution of valid answers is not necessarily uniform. I wonder how BoD would behave under non-uniform target distributions, and whether its advantage in solution coverage would still hold

Questions for authors
  1. could you provide some discussion about fixed-point methods e.g. [1, 2]? How would fixed-point stopping work in your setting? For example, is it possible for different trajectories to converge to different fixed points corresponding to different valid solutions? I wonder how the authors view these methods in terms of solution coverage, compared to BoD, which aims to explore different solutions across depths.

  2. It seems that your objective is closely related to learning-based stopping methods such as PonderNet [3] and the RL approach [4], as they also seem to favour depths with lower prediction loss.

[1] Xiao-Wen Yang et al., Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models, ICML 2026.

[2] Sajad Movahedi et al., Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers, 2026.

[3] Andrea Banino et al., PonderNet: Learning to Ponder, 2021.

[4] Hsun-Yu Kuo et al., Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping, 2026.

4 comments

We thank the reviewer for the thoughtful review. We answer each point below.

W1: I think it is also important to consider how easily we can identify the correct depth or candidate at inference time. like, how sensitive are the predictions across different depths? ... It would be interesting to further discuss the trade-off between exploratory and stable recurrent trajectories.

We would like to clarify that BoD targets settings with multiple valid solutions in which candidates can be verified, as in Best-of-N sampling and pass@k. The goal is therefore to produce a set of candidates from one trajectory rather than to identify a single correct depth; a verifier returns a valid candidate (or all valid ones). How predictions vary across depths is shown in Figure 4 for graph coloring: the latent trajectory keeps moving among states that decode to different valid solutions (Figure 4c), and new valid solutions keep appearing over many depths across the test set (Figure 4b). With the constraint check, a valid candidate is found for 99.9% (graph coloring) and 99.1% (N-Queens) of multi-solution inputs. Even without a verifier, the readout at the most confident depth (mean maximum log-probability over the predicted positions) is valid for 88.1% and 76.2%.

Answer at the most confident depth is valid (%), no verifier, multi-solution test inputs, three seeds:

Graph coloringN-Queens
Last-Depth30.023.5
Mean-Depth49.251.8
Random-Depth48.554.8
Soft-Depth80.855.5
Best-of-Depth88.176.2

This reflects a trade-off: at a fixed depth without any selection, stable objectives can be better, whereas once candidates are selected across depths, by a verifier or by confidence as above, BoD's breadth pays off. We discuss when each objective is preferable in our answer to W3.

W2: However, in many tasks, there is only one valid answer. i am wondering whether BoD would still be beneficial in such settings, especially compared to objectives that encourage accurate predictions at a particular depth.

Many practical tasks have multiple valid answers, such as programs that pass the same tests, proofs of a statement, plans and schedules that satisfy given constraints, and molecules with target properties. BoD targets such settings, where candidates can be verified. When the answer is truly unique, BoD remains beneficial, because it only needs to reach the solution at some depth. Our N-Queens test set also contains 1,000 single-solution inputs besides the 1,000 multi-solution inputs used in the paper; on them, the solution appears at some depth for 97.3% of inputs with BoD, versus 89.8% (Last-Depth), 94.6% (Mean-Depth), 94.5% (Soft-Depth), and 94.7% (Random-Depth).

W3: Proposition 3 is based on a constructed setting where the recurrent dynamics are already designed to cover all valid answers. ... It is also unclear under what conditions BoD would outperform other objectives ...

Proposition 3 is a supporting result, not a main contribution: it is meant to provably present a failure mode of the conventional objectives, not how BoD learns exploratory dynamics in general. It constructs a setting in which Last-, Mean-, and Random-Depth training provably converge to zero solution coverage, despite sufficient model capacity for full coverage. BoD attains full coverage with the same architecture, initialization, and SGD schedule.

The proof of Proposition 3 also identifies how the conventional objectives fail. They have a unique minimizer that predicts the positionwise target marginals. In the constructed task, this symmetric predictor produces the same invalid greedy output at every depth. Thus, the failure persists even when the conventional objectives are successfully optimized. This complements our empirical observations (Section 5), where the conventional objectives also settle on fewer solutions.

This failure mode is also not specific to the construction of Prop 3. Let qd(⋅∣x)q_d(\cdot\mid x) denote the output distribution at depth dd. In fact, when the ground-truth distribution pY(⋅∣x)p_Y(\cdot\mid x) supports at least two distinct solutions, and the depth-wise predictions can vary separately, we can also more generally show the followings. The scope of the claim also covers distributional loss including CE:

  • BoD: a collapsed predictor (qd=qd′q_d = q_{d'} for all d≠d′d \neq d') is never a local minimum (for D≥2D\ge 2).
  • Soft-Depth: in this softmin approximation of BoD, the collapsed predictor that outputs the target distribution is a local minimum when the temperature is sufficiently high, and a strict saddle when it is sufficiently low.
  • Last-, Mean-, and Random-Depth: the collapsed predictor that outputs the target distribution is a global minimum.

Conversely, we may say that conventional objectives can be preferable when a single unverified answer at a fixed depth is needed. We will add this supplementary result to the revision (D.5).

W4: the performance of BoD might be sensitive to the target distribution. ... I wonder how BoD would behave under non-uniform target distributions, and whether its advantage in solution coverage would still hold

BoD's advantage does not rely on uniform targets. For any target distribution pp, BoD's exact-match risk, the probability that the target is not among its readouts, is one minus the total probability of the valid solutions it outputs, 1−∑y∈SD(x)∩Y∗(x)p(y∣x)1 - \sum_{y \in S_D(x) \cap \mathcal{Y}^*(x)} p(y\mid x). Every solution with p(y∣x)>0p(y\mid x) > 0 still lowers the risk, so BoD still aims to cover all solutions when the depth budget allows; non-uniformity only decides which solutions are found first. Empirically, we trained graph coloring with teachers that increasingly favor one reference coloring ycy_c:

Test coverage (%), graph coloring, three seeds. Each column gives the probability that the teacher outputs ycy_c, averaged over training graphs; under the uniform teacher it is one over the number of valid colorings, 0.15 on average:

Uniform (0.15)Skewed (0.30)Strongly skewed (0.60)ycy_c only (1.0)
Last-Depth18.8 ± 0.717.8 ± 1.213.4 ± 0.512.9 ± 0.6
Best-of-Depth72.7 ± 2.964.4 ± 2.453.4 ± 1.124.2 ± 2.0

BoD's coverage decreases as the teacher concentrates, but it stays well above Last-Depth at every level.

Q1: could you provide some discussion about fixed-point methods e.g. [1, 2]? How would fixed-point stopping work in your setting? For example, is it possible for different trajectories to converge to different fixed points corresponding to different valid solutions? ...

With fixed-point stopping, one trajectory yields one candidate, so diversity can only come from multiple starts. We trained FPRM [2] and EqR [5] in our settings and collected the fixed points of 64 trajectories from random initial states, which costs 64 times as much as one BoD trajectory.

Coverage (%), three seeds:

Graph coloringN-Queens
FPRM, fixed points of 64 trajectories6.89.3
EqR, fixed points of 64 trajectories10.856.4
Best-of-Depth, one trajectory72.291.0

Multiple starts do reach different fixed points, but even with 64 times the compute they cover fewer solutions than one BoD trajectory. Fixed-point methods aim at stable convergence, which is complementary to our goal.

[5] Benhao Huang et al., Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning, ICML 2026.

Q2: It seems that your objective is closely related to learning-based stopping methods such as PonderNet [3] and the RL approach [4], as they also seem to favour depths with lower prediction loss.

Both families favor low-loss depths, but they differ in what the depth is conditioned on. These methods learn a halting distribution p(d∣x)p(d\mid x) that depends on the input only. With several valid targets, their expected loss ∑dp(d∣x) Ld(x,y)\sum_d p(d\mid x)\, L_d(x, y), averaged over targets, is depth-additive in the sense of our answer to W3, so it is minimized by the collapsed predictor that pulls every depth with nonzero halting probability toward the mixture of targets. BoD instead chooses the depth for each (input, target) pair, so different targets of one input go to different depths. On the same backbone, an expected-loss objective with an input-conditioned exit distribution (Ouro, Table 16) reaches 18.6% (graph coloring) and 46.8% (N-Queens) coverage, versus 72.2% and 91.0% for BoD.

We would be glad to clarify anything further.

Thank you again for your thoughtful review. As the discussion period is nearing its end, we would appreciate it if you could review our responses and additional results.

We hope they address your concerns, and we would be grateful if you could reconsider your assessment.

Reviewer_nFhwOpenReview ↗

Response to the rebuttal

Thanks for the clarification; I am satisfied with the response. Therefore, I will keep my positive score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Updates

We thank all reviewers for their constructive feedback. We believe that the additional experiments and revisions address all concerns raised in the reviews.

Reviewers nFhw and uTHa are satisfied with our responses, noting that “the paper provides an interesting perspective on an important question for looped Transformers,” that we “provide theoretical justification for the proposed objective, establishing a connection between BoD and solution coverage,” and that the paper presents a “Simple, Principled, and Well-Executed Idea.”

We have also revised the manuscript to improve its clarity based on the reviewers’ suggestions. Revisions to the paper are marked in blue.

Reviewer D7Fb has a remaining concern regarding “the scope of the evidence”, stating that “it remains unclear how much of the HumanEval gain carries over to a single returned program when the benchmark tests are unavailable for selection.” We have provided an answer to this concern, please see our response.

Below, we provide the list of the points we have addressed:

  1. Statistical reliability of the code results (D7Fb).
    We fine-tuned all objectives with three seeds. BoD is the best objective in every seed on both HumanEval and MBPP.
  2. Selecting a single answer without a verifier (D7Fb, nFhw).
    Label-free selection, by confidence on the synthetic tasks and by the examples in the problem statements on HumanEval, keeps BoD ahead of all other objectives.
  3. Compute-matched sampling baseline (uTHa).
    At equal recurrent compute, BoD's depth-wise candidates solve more problems than samples from Last-Depth.
  4. Training cost (uTHa).
    BoD trains at almost the same cost as Last-Depth, with nearly the same time per update and memory.
  5. Scope of the theory (nFhw).
    A supplementary result (Proposition 4, Appendix D.5) shows that the failure of the conventional objectives in Proposition 3 is not specific to its construction.
  6. Robustness (nFhw).
    BoD keeps its advantage on single-solution inputs, under non-uniform target distributions, and against fixed-point methods with multiple restarts.
  7. Clarity (D7Fb).
    We added a preliminaries section with the evaluation metrics and the limitation of Last-Depth training, moved related work to Section 6, and rewrote Section 4.5.3 and the interpretation of the propositions.

We hope this summary is helpful for the final evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.