acceptodds
Under review as a conference paper at ICLR 2027

ReTrail: Hesitation-Guided Backtracking for Test-Time Reasoning

Abstract

Large reasoning models (LRMs) often revise earlier steps through reflection, but insufficient reflection leaves mistakes uncorrected, whereas excessive reflection leads to repetitive reasoning. Existing approaches that regulate reflection using fixed schedules cannot adequately address this difficulty because they do not adapt to whether the model needs to reconsider its reasoning or move forward. To address this limitation, we propose ReTrail, a training-free decoding method that uses the model's own reflection-token probabilities to guide when and how it revises its reasoning. To determine when to intervene, ReTrail monitors the hesitation mass, defined as the total probability the model assigns to reflection tokens at the end of each reasoning step. This probability measures the model's tendency to reflect, and it remains available even when no reflection token is sampled into the output text. When this signal rises sharply above its earlier values, ReTrail selects an earlier step with high hesitation mass as the starting point for revision. It rolls back to that step and regenerates the continuation. During regeneration, ReTrail first raises reflection-token logits to encourage the model to reconsider that step, and then lowers them to prevent repetitive reflection. The method operates on a single generation trajectory without requiring an external verifier model or multi-candidate search. Across four reasoning models and four benchmarks in mathematics and code generation, ReTrail achieves the highest or tied-highest mean accuracy in all sixteen evaluated settings under identical retained-output token limits. It improves accuracy over unmodified decoding by an average of 3.25 and up to 8.08 percentage points, with a 15.2% average increase in total generated tokens, including tokens discarded during rollback. Component ablations show that rollback improves accuracy while reducing generation length compared to applying reflection bias in place. Furthermore, controlled comparisons of revision timing and restart location show higher mean accuracy with hesitation mass than with representative alternatives like entropy. Code is available at https://anonymous.4open.science/r/retrail-8FC5/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large reasoning models, test-time reasoning, chain-of-thought reasoning, self-reflection, adaptive decoding, backtracking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CFJrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ReTrail uses reflection-token probability to decide when to backtrack and where to restart a reasoning trace. Regeneration briefly promotes then suppresses reflection. Math and coding experiments show higher mean accuracy across several models, with discarded generation counted separately.

Critical strengths
  1. Returning to an earlier prefix is a useful addition to reflection control. The fixed-trigger experiment isolates restart location and gives convincing evidence on AIME against entropy-based and random locations.
  2. The method needs no verifier or training. Evaluation spans four model families, and parameters selected on MATH-500 are shared across test settings.
Critical weaknesses
  1. The evidence is stronger for restart location than for hesitation-based triggering. Trigger controls use one seed, and their intervals in Appendix F include zero. Several Qwen-7B end-to-end gains are similarly unresolved. The claims should reflect this difference, or the trigger study should be repeated on more problems.
  2. Equal retained-output limits allow ReTrail more computation: total generation rises by 15.2% on average. The matched-generation s1 control helps, but covers only Qwen-7B and one seed. It does not establish a general advantage at equal compute, especially with hosted probe and replay costs.
Questions for authors
  1. Suggested control: does hesitation-triggered rollback still outperform entropy triggering across independent seeds?
  2. Clarification: what are the total hosted costs, including probability probes and prefix replay, relative to Original?
3 comments

Thank you for your careful review and concrete suggestions. We respond to each weakness and question below.

W1: The evidence is stronger for restart location than for hesitation-based triggering. The trigger study uses one seed, and several Qwen-7B end-to-end gains are unresolved.

We followed your second suggestion and repeated the trigger study on Qwen-7B with three seeds, which triples the number of responses (90 on AIME 2026, 240 on LiveCodeBench). Each variant below changes only when the intervention fires and keeps rollback and bias unchanged (Table R1).

Table R1. Trigger signals compared (Qwen-7B, three seeds).

MethodAIME 2026LiveCodeBenchReTrail's lead in percentage points (pp), mean ± SD over seeds (AIME / LCB)
ReTrail47.78 (43/90)39.58 (95/240)n/a
w/ entropy trigger41.11 (37/90)37.92 (91/240)+6.67 ± 3.33 / +1.67 ± 1.91
w/ random trigger (same firing rate)36.67 (33/90)34.17 (82/240)+11.11 ± 5.09 / +5.42 ± 4.02
w/ first trigger at a random position drawn from ReTrail's own first-trigger positions41.11 (37/90)36.25 (87/240)+6.67 ± 3.33 / +3.33 ± 1.91

The variants intervene at similar rates (2.3 to 2.7 times per problem). ReTrail's first seed here is a separate run made together with the variants, so its LiveCodeBench total (95) is two problems above Table 1 (93).

The single-seed result holds with three seeds. ReTrail is ahead of the random trigger and the random-position variant in all three seeds we ran on both benchmarks. Although the margin over the entropy trigger on LiveCodeBench is modest, ReTrail matches or exceeds it in all three seeds on both benchmarks, and its lead on AIME 2026 averages 6.67 pp. The last row keeps how early the first intervention happens and removes only its link to hesitation, which indicates that the advantage comes from the hesitation signal.

For the end-to-end gains, we added three new seeds for Original and ReTrail on Qwen-7B on all four benchmarks, so the main comparison now has six seeds (Table R2).

Table R2. Original vs. ReTrail over six seeds (Qwen-7B).

BenchmarkOriginal, six seedsReTrail, six seedsGain (pp)
AIME 202548.33 (87/180)50.56 (91/180)+2.22
AIME 202645.56 (82/180)47.22 (85/180)+1.67
HMMT 202636.87 (73/198)39.39 (78/198)+2.53
LiveCodeBench36.88 (177/480)38.75 (186/480)+1.88
All four40.37 (419/1038)42.39 (440/1038)+2.02

ReTrail is ahead on all four benchmarks and in each of the three new seeds, and the gain stays at about 2 pp.

In the revision, the trigger claim in Section 5.4 will rest on these three-seed results and will give the margin for each comparison, so that the text reflects where the evidence is stronger (restart location, random trigger) and where the margin is smaller (entropy trigger on LiveCodeBench).

W2: Equal retained-output limits allow ReTrail more computation. The matched-generation s1 comparison covers only Qwen-7B and one seed, and hosted probe and replay costs are not counted.

We extended the matched-generation s1 comparison on Qwen-7B from one seed to three. For every problem, s1 is forced to keep reasoning until it reaches ReTrail's token count on that problem. We match two budgets, ReTrail's retained length and ReTrail's total generation, which also counts the deleted tokens (Table R3).

Table R3. ReTrail vs. s1 at matched generation (Qwen-7B, three seeds).

MethodAIME 2026LiveCodeBenchMean generated tokens (AIME / LCB)ReTrail's lead in pp, mean ± SD over seeds (AIME / LCB)
ReTrail47.78 (43/90)39.58 (95/240)19.00k / 15.18kn/a
s1 matched to ReTrail's retained length41.11 (37/90)36.25 (87/240)18.09k / 14.30k+6.67 ± 3.33 / +3.33 ± 1.91
s1 matched to ReTrail's total generation41.11 (37/90)36.25 (87/240)19.22k / 15.34k+6.67 ± 6.67 / +3.33 ± 1.91

ReTrail's token count includes deleted tokens. s1 lands within 1.3% of its target length.

At equal compute, ReTrail is never below s1 in any seed on either benchmark. s1 with this budget does not exceed Original (42/90 and 89/240), so extra tokens alone do not produce the gain.

This comparison is limited to Qwen-7B, where we control decoding and can match the budget token by token. For the hosted models, ReTrail sends no separate probe calls, because the interface returns the top-20 token probabilities with every generated token and the hesitation mass is read from them. Prefix replay is caused by the hosted interface, which keeps no generation state between requests, and is not part of the method. We report its cost for each hosted model in our response to Q2 (Table R5).

We will replace the single-seed table in the appendix with the three-seed s1 results.

Q1: Does hesitation-triggered rollback still outperform entropy triggering across independent seeds?

Yes, in the three independent seeds we ran on Qwen-7B. Table R4 lists the number of solved problems in each seed. The first entropy variant replaces only the trigger, and the second also uses entropy to choose the restart point.

Table R4. Hesitation vs. entropy, solved problems per seed (Qwen-7B).

MethodAIME 2026, seeds 1 / 2 / 3 (of 30)LiveCodeBench, seeds 1 / 2 / 3 (of 80)
ReTrail17 / 10 / 1633 / 33 / 29
w/ entropy trigger15 / 9 / 1330 / 32 / 29
w/ entropy trigger + entropy-peak rollback14 / 8 / 1428 / 28 / 26

ReTrail matches or exceeds the entropy trigger in each seed on both benchmarks, and it scores higher than the fully entropy-based variant in each seed. The entropy variants intervene 2.6 to 2.7 times per problem, at least as often as ReTrail (Section 5.4), so ReTrail's lead is not explained by intervening more. The pooled accuracies are in Table R1.

Q2: What are the total hosted costs, including probability probes and prefix replay, relative to Original?

Table R5 gives the cost for the three hosted models, as per-problem averages pooled over the four benchmarks.

Table R5. Token cost on the hosted models.

ModelNewly generated tokens vs. Original (deleted tokens included)Replayed prefix, as a share of Original's lengthReplayed prefix is billed as
DeepSeek-V3.2-Speciale+9.4% (31.58k vs. 28.86k)17.5% (5.04k)output
GPT-OSS-120B+2.6% (4.94k vs. 4.81k)25.8% (1.24k)input
Kimi-K2.5+16.2% (19.40k vs. 16.70k)21.7% (3.62k)input

The first column pools all problems, so it differs slightly from the per-benchmark averages in the paper.

Table R5 counts every token the provider bills. ReTrail generates 2.6% to 16.2% more tokens than Original on the hosted models, with deleted tokens counted.

Probability probes add no cost. The interface returns the top-20 token probabilities with every generated token, and ReTrail reads the hesitation mass from them without sending extra requests. The two-request recovery in Appendix C.2 is a fallback for services that do not return usable token probabilities; the reported runs did not need it. The submitted appendix lists probability probes among the hosted costs (Appendices C.1, E, and G). That wording describes our early interface tests, which sent extra requests to read the probabilities at a given position. The reported runs did not send them. We apologize for the confusion this wording caused, and we will correct these descriptions in the appendix.

Prefix replay comes from the hosted interface and is not part of the method. A hosted interface keeps no generation state between requests, so each rollback passes the kept prefix to the model again. On a locally run model, generation resumes directly from the rollback point and replay costs nothing. For GPT-OSS-120B and Kimi-K2.5 the prefix goes into the request and is billed as input, at a lower price than output. The Speciale endpoint does not accept a supplied prefix (Appendix C.1), so the model rewrites the prefix before continuing and it is billed as output. Speciale is therefore the least favorable billing case, and we report it in full. Its billed output is about 1.27 times that of Original, for gains of 4.4 to 8.1 pp on the four benchmarks. This stays well below the cost of sampling a second solution, and ReTrail needs no verifier or additional model.

We will add Table R5 to the cost section of the appendix, and Appendix C will state that the two-request recovery is needed only on services that do not return usable token probabilities.

AuthorsOpenReview ↗

Follow-up from the Authors

Dear Reviewer CFJr,

Many thanks for your thoughtful review. We found the experiments you suggested very valuable and carried out each of them with care. Before the discussion period closes, we would like to highlight the main results from our response, which support the method on every point you raised.

  • Evidence across seeds (W1, Q1). Our three-seed trigger study confirms the single-seed result. ReTrail leads the entropy trigger by 6.67 pp on AIME 2026 and 1.67 pp on LiveCodeBench, and it beats both random triggers in every seed (Tables R1, R4). Over six seeds, ReTrail outperforms Original by 2.02 pp and leads on each of the four benchmarks (Table R2).
  • Compute cost (W2, Q2). At equal compute over three seeds, ReTrail leads s1 by 6.67 pp on AIME 2026 and 3.33 pp on LiveCodeBench (Table R3). s1 itself stays below Original, which shows that extra tokens alone do not produce the gain. On the hosted models, ReTrail generates only 2.6% to 16.2% more tokens than Original and reading the probabilities adds no cost. Table R5 gives the full billed cost, including prefix replay.

We believe the paper is clearly stronger with these additions, and we hope you will feel the same. Thank you for the suggestions that made this possible and for your time during this busy period.

Best regards, The Authors

Reviewer_7W98OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to keep long chain-of-thought models from getting stuck in wrong deductions or endless self-correction loops at test time. The authors propose ReTrail, a training-free decoding method that tracks the total next-token probability of four reflection words such as Wait and But at each step boundary. When that probability rises above the running mean plus 1.5 standard deviations of the trace, ReTrail rolls back to an earlier step with high reflection probability. It then regenerates under a short logit bias that first promotes and then suppresses reflection tokens. The method is compared against standard decoding, TIP, and CyclicReflex across four reasoning models on competition math and LiveCodeBench.

Critical strengths

(1)The core idea is simple and practical, and the paper cleanly tests how it chooses where to roll back. The method uses the model's own reflection-word probabilities to decide when to revise and where to resume, without extra training, an external verifier, or parallel sampling. In Table 4, the authors test the rollback step directly by keeping the initial trace, the trigger point, and the post-rollback logit bias fixed while varying only which earlier step the model returns to. Rolling back to the step with the highest hesitation mass beats the entropy peak by nearly 8 points and beats a distance-matched random step by 10 points on AIME 2026. This is a meaningful result because TIP and CyclicReflex never roll back at all, while prior backtracking methods either fine-tune the model like Yang et al. [1] or rely on a seperate verifier.

(2) The hesitation signal is cheap to compute during local decoding and captures different information from standard token entropy. Because it is read straight from the next-token distribution at each step boundary, it adds no compute overhead and is available even when no reflection word is sampled. Appendix M shows that the top 10% positions for hesitation mass and entropy overlap less than half the time, so the two signals flag different steps. Table 3 then tests an entropy trigger calibrated to fire at the same rate as the hesitation trigger. Under that matched rate, hesitation mass beats entropy by 6.7 points on AIME 2026 and 3.75 points on LiveCodeBench with the same or fewer interventions, which separates the method from simple uncertainty-based resampling.

[1] Yang et al., Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models, 2025, arXiv:2502.04404.

Critical weaknesses

(1)The ablation experiments do not establish that rollback and logit biasing improve over standard decoding, because they run on a single seed and leave out the unmodified baseline for that seed. Tables 2, 3, 4, 7, and 8 all use one seed, mostly on Qwen-7B, without reporting the Original score on that run. Sec 5.1 indicates this run is the first of the three seeds in Table 1, and from the paired differences in Table 5 we can work out that Original scored 53.3% or 56.7% on AIME 2026 and 37.5% to 42.5% on LiveCodeBench on that seed. Every Qwen-7B ablation variant in Tables 2 and 8 sits at or below those numbers, and full ReTrail is roughly tied with plain decoding. So the 23-point gain from w/o rollback to ReTrail in Sec 5.3 mainly comes from w/o rollback dropping ~20 points below standard decoding. That drop fits the looping failure mode described by Pipis et al. [2], where models facing a hard step fall back on easy repetitive actions and a +5 boost on reflection logits pushes them further into repeating themselves. In that setting, rollback helps by cutting off the looping text rather than by finding the wrong reasoning step. Please add the unmodified Original row to these tables and run the ablations on the other two seeds.

(2) Most of the end-to-end gains in Table 1 are within benchmark sampling noise, and the largest gains appear to come from a hosted setup that alters the detection signal. On Qwen-7B, GPT-OSS-120B, and Kimi-K2.5, the gains over Original are mostly 1 to 3 points on benchmarks of 30 to 80 problems. In Appendix F, all end-to-end confidence intervals for Qwen-7B include zero, and no intervals are reported for the other models. Hochlehnert et al. [3] found seed standard deviations of 5 to 15 points for single runs on 30-problem AIME'24, which matches the 13-point swing between Qwen-7B's first seed and the other two seeds here. Meanwhile, the ~8 point gains on DeepSeek-V3.2-Speciale appear to use the hosted API workaround in App. C, which estimates raw probability m_raw before temperature scaling and top-p truncation instead of the post-temperature, post-top-p mass defined in Eq 1. Applying the same lambda = 1.5 threshold to m_raw need not trigger at the same steps, and App. C notes that this equivalence still requires validation.

[2] Pipis et al., Wait, Wait, Wait... Why Do Reasoning Models Loop?, 2025, arXiv:2512.12895. [3] Hochlehnert et al., A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility, COLM 2025, arXiv:2504.07086.

Questions for authors

(1)What did unmodified Original score on the first seed for Qwen-7B and GPT-OSS-120B in Table 2, or what did the w/o rollback and regeneration bias check variant in App. D score? Please add that baseline row to Tables 2 to 4 and 8, and run Table 2 on the other two seeds so we can see if the components beat plain decoding or mostly recover from bias-induced looping [2].

(2) For DeepSeek-V3.2-Speciale, did the detector use m_raw from App. C.2 or the post-temperature, post-top-p mass from Eq 1, and what is the real token cost once regex prefix replay and probe calls are counted? Were GPT-OSS-120B and Kimi-K2.5 run locally or through the same hosted path?

(3) In Fig 6, the token-level detector fires its first alarm within the first 110 tokens on every single trace because the early variance is near zero. Does the step-level detector with H_min = 3 also fire right near the beginning? What does the distribution of first-alarm positions look like, and how would ReTrail compare to a trivial baseline that just rolls back once after the first few steps?

5 comments

Response to Reviewer 7W98 (Part 1 of 4)

Thank you for your time, careful reading, and suggestions. Our point-by-point responses follow.

W1: The ablations use one seed and omit the same-seed Original baseline, and the gain over w/o rollback may come from cutting off looping text.

Thank you for working out the per-seed baseline. We added the Original row and ran the ablations on the other two seeds, as you suggested. The new runs confirm two of your points: over three seeds, every single-component variant on Qwen-7B is below Original, and the 23-point gap to w/o rollback on the first seed is larger than the three-seed average.

(1) The gain is due to combining the components. A partial intervention can leave the model worse off than no intervention. This matches the starting point of the paper, that both too much and too little reflection hurt (Table R1, solved problems per seed on Qwen-7B).

Table R1. Component ablation, solved problems per seed (Qwen-7B).

MethodAIME 2026, seeds 1 / 2 / 3 (total of 90)LiveCodeBench, seeds 1 / 2 / 3 (total of 240)
Original17 / 10 / 15 (42)28 / 31 / 30 (89)
ReTrail17 / 10 / 16 (43)33 / 33 / 29 (95)
w/o rollback10 / 9 / 13 (32)30 / 29 / 27 (86)
w/o regeneration bias13 / 8 / 13 (34)28 / 28 / 27 (83)
w/ promotion-only bias11 / 7 / 11 (29)26 / 26 / 25 (77)
w/ suppression-only bias13 / 9 / 14 (36)30 / 30 / 27 (87)

ReTrail's first seed here is a separate run made together with the variants, so its LiveCodeBench total (95) is two problems above Table 1 (93).

ReTrail is ahead of every variant in all three seeds we ran on both benchmarks, and it is the only row above Original. An effect of extra tokens or of simply perturbing the trace would lift the variants as well. Each variant falls below Original because it lacks the component designed to prevent that failure (Table R2).

Table R2. Why each variant falls below Original.

VariantWhy it falls below Original
w/o rollbackThe text that caused the hesitation stays in context, so the model keeps reflecting on it with no place to revise.
w/o regeneration biasNothing prompts reconsideration after the rollback, so the model tends to rewrite a similar line of reasoning.
w/ promotion-only biasReflection is never damped, so some chains run very long and a few reach the length limit.
w/ suppression-only biasReflection is discouraged right after the rollback, so the rewritten step gets less checking than in Original.

In ReTrail, rollback gives the model a place to revise, promotion makes it reconsider there, and suppression then lets it move on. The lead over w/o rollback is present in each of the three seeds and averages 12.22 percentage points (pp) on AIME 2026 and 3.75 pp on LiveCodeBench.

The tie with Original on the first seed falls within the seed variation you cite from Hochlehnert et al. [3], and that seed is Original's best on AIME 2026 (17 solved, against 10 and 15). Over six seeds and four benchmarks, ReTrail outperforms Original by 2.02 pp (see our response to W2).

(2) Looping appears whenever a component is missing, with or without the bias. We measured looping directly, following Pipis et al. [2] (Table R3, share of responses in which the same 64-token span appears three or more times).

Table R3. Share of responses that loop (Qwen-7B).

MethodAIME 2026LiveCodeBench
Original11.11%8.75%
ReTrail11.11%9.17%
w/o rollback20.00%15.00%
w/o regeneration bias16.67%12.92%
w/ promotion-only bias23.33%17.92%
w/ suppression-only bias14.44%11.25%

ReTrail applies the same promotion bias as the variants and repeats about as often as Original. The variant with no bias at all (w/o regeneration bias) and the one that lowers reflection (suppression-only) both repeat more than Original and score below it. The reflection boost alone therefore does not account for the looping.

(Our response to W1 continues in Part 2.)

[1] Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models

[2] Wait, Wait, Wait... Why Do Reasoning Models Loop?

[3] A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility

Response to Reviewer 7W98 (Part 2 of 4)

W1 (continued from Part 1)

(3) The effect of rollback depends on which step the model returns to. If rollback only cut off looping text, any restart of similar length would help equally. Table R4 fixes the trigger and the bias, changes only the restart location, and includes the no-intervention row you suggested (three continuations per problem).

Table R4. Restart locations compared (Qwen-7B, fixed trigger).

MethodAIME 2026LiveCodeBench
No intervention (continue from the trigger point)47.78 (43/90)35.42 (85/240)
Restart at the hesitation peak (ReTrail)52.22 (47/90)37.92 (91/240)
Restart at the entropy peak44.44 (40/90)35.83 (86/240)
Restart at a distance-matched random step42.22 (38/90)35.83 (86/240)
w/o rollback (bias applied at the trigger point)48.89 (44/90)35.00 (84/240)

Table R4 applies a single intervention. Here, applying the bias in place matches no intervention, so there is no bias-induced damage to repair, yet the hesitation peak is still higher (on LiveCodeBench by 2.92 pp over w/o rollback, 95% CI [0.83, 5.42], Appendix F). The drop of w/o rollback in Table R1 appears under the full policy, where the bias is applied in place up to three times per problem. A random restart deletes a similar amount of text and falls below no intervention on AIME 2026, where the hesitation peak leads it by 10.00 pp [3.33, 16.67] (the comparison in your Strength 1).

In the revision, we will add the Original row to Tables 2, 3, 4, 7, and 8, and state in the abstract, the introduction, and Section 5.3 that rollback and the bias help in combination, using the three-seed margins in place of the single-seed 23-point figure. We will also discuss Yang et al. [1], Pipis et al. [2], and Hochlehnert et al. [3].

W2: Most end-to-end gains are within sampling noise, and the largest gains come from a hosted setup that reads mrawm_{\mathrm{raw}} and may trigger at different steps.

Thank you for separating these two concerns.

(1) The gain over Original holds across seeds. During the rebuttal period, we ran additional seeds. We placed them on Qwen-7B, where the gains in Table 1 are among the smallest, and ran three new seeds (seeds 4 to 6) for Original and ReTrail on all four benchmarks (Table R5).

Table R5. Original vs. ReTrail over six seeds (Qwen-7B).

BenchmarkOriginal, six seedsReTrail, six seedsGain, six seeds (pp)Gain, new seeds only (pp)
AIME 202548.33 (87/180)50.56 (91/180)+2.22+2.22
AIME 202645.56 (82/180)47.22 (85/180)+1.67+2.22
HMMT 202636.87 (73/198)39.39 (78/198)+2.53+3.03
LiveCodeBench36.88 (177/480)38.75 (186/480)+1.88+2.08
All four40.37 (419/1038)42.39 (440/1038)+2.02+2.31

Summed over the four benchmarks (173 problems per seed), ReTrail solves 4, 5, and 3 more problems than Original in the three new seeds. The 95% interval across these seeds is [+0.88, +3.75] pp and excludes zero. We agree that the interval on a single benchmark of 30 to 80 problems is wide. The direction, however, is consistent. ReTrail is above Original in all 16 model and benchmark settings of Table 1.

(2) The mrawm_{\mathrm{raw}} reading does not appear to inflate the gain. We ran the validation that Appendix C calls for. On Qwen-7B, where both readings are available, we ran ReTrail with mrawm_{\mathrm{raw}} under the same threshold on three seeds (Table R6).

Table R6. Two readings of the hesitation mass (Qwen-7B, three seeds).

MethodAIME 2026LiveCodeBench
Original46.67 (42/90)37.08 (89/240)
ReTrail with the mass of Eq. 147.78 (43/90)39.58 (95/240)
ReTrail with mrawm_{\mathrm{raw}}47.78 (43/90)38.75 (93/240)

The two readings differ by at most one problem in each seed, and mrawm_{\mathrm{raw}} does not score higher, so the larger gains on Speciale are unlikely to be due to the mrawm_{\mathrm{raw}} reading.

The two readings also agree on where hesitation is high. We computed both on the original traces of two locally run models, Qwen-7B and GPT-OSS-20B (330 traces each). Within a trace, their rank correlation is 0.95 to 0.98 at the boundaries where the mass of Eq. 1 is nonzero (top-p truncation sets it to exactly zero at the others). Between 80% and 94% of the alarms fall on the same step, and the first alarm selects the same rollback point in 75% to 90% of cases. mrawm_{\mathrm{raw}} does raise its first alarm earlier (median fifth boundary, against sixth to eleventh), but accuracy stays within one problem per seed.

The six-seed results will be added to Appendix F and the mrawm_{\mathrm{raw}} validation to Appendix C.

Response to Reviewer 7W98 (Part 3 of 4)

Q1: What did Original score on the first seed for Qwen-7B and GPT-OSS-120B in Table 2? Please add that row to Tables 2 to 4 and 8, and run Table 2 on the other two seeds.

On the first seed, Original solves 17/30 on AIME 2026 and 28/80 on LiveCodeBench with Qwen-7B, and 22/30 and 62/80 with GPT-OSS-120B. Your derivation matches the AIME 2026 value (56.7%). The LiveCodeBench value (35.0%) is slightly below your range, because the ReTrail row of Table 2 is a separate run from the first seed of Table 1 (33 vs. 31 solved), while the paired differences in Table 5 are computed on the Table 1 runs. The Qwen-7B rows for all three seeds are in Table R1. Table R7 gives the first seed of GPT-OSS-120B.

Table R7. Component ablation on GPT-OSS-120B (first seed).

Method (GPT-OSS-120B, first seed)AIME 2026LiveCodeBench
Original73.33 (22/30)77.50 (62/80)
ReTrail80.00 (24/30)82.50 (66/80)
w/o rollback70.00 (21/30)77.50 (62/80)
w/o regeneration bias76.67 (23/30)75.00 (60/80)
w/ promotion-only bias73.33 (22/30)68.75 (55/80)
w/ suppression-only bias76.67 (23/30)78.75 (63/80)

ReTrail is the highest row on both benchmarks and is above the same-seed Original by 6.67 pp and 5.00 pp. No variant reaches ReTrail, which matches the pattern on Qwen-7B. The hosted budget allowed only one seed for this model. On that seed, the full method scores higher than plain decoding, as it does on Qwen-7B over three seeds.

We ran the continuation without rollback or bias as the no-intervention row of Table R4, where it sits below the hesitation-peak restart on both benchmarks. Whether the components beat plain decoding or only recover from looping is addressed in W1 with the three-seed ablation (Table R1) and the looping measurement (Table R3).

The Original row will be added to Tables 2 to 4 and 8, and Table 2 will report three seeds for Qwen-7B.

Q2: Did Speciale use mrawm_{\mathrm{raw}} or the mass of Eq. 1? What is the real token cost with prefix replay and probe calls? Were GPT-OSS-120B and Kimi-K2.5 run locally or through the same hosted path?

(1) Speciale uses mrawm_{\mathrm{raw}}, read without probe calls. All three hosted models use mrawm_{\mathrm{raw}}, because a hosted interface does not expose the distribution after temperature and top-p. The interface returns the top-20 token probabilities with every generated token, and ReTrail reads mrawm_{\mathrm{raw}} from them as the model generates. No request is sent only to read probabilities, so probe calls add no cost. The two-request recovery in Appendix C.2 is a fallback for services that do not return usable token probabilities; the reported runs did not need it. The submitted appendix lists probability probes among the hosted costs (Appendices C.1, E, and G). That wording describes our early interface tests, which sent extra requests to read the probabilities at a given position. The reported runs did not send them. We apologize for the confusion this wording caused, and we will correct these descriptions in the appendix. Table R6 shows that mrawm_{\mathrm{raw}} and the mass of Eq. 1 give the same accuracy on Qwen-7B within one problem per seed.

(2) GPT-OSS-120B and Kimi-K2.5 are also hosted, with a cheaper way to continue after a rollback. A hosted interface keeps no generation state between requests, so each rollback passes the kept prefix to the model again. The Speciale endpoint does not accept a supplied prefix (Appendix C.1), so the model rewrites the prefix before continuing, which is billed as output. For GPT-OSS-120B and Kimi-K2.5, the prefix goes into the request and the model continues from it, which is billed as input, at a lower price.

(Our response to Q2 continues in Part 4.)

Reviewer_xMwVOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper focuses on the problem of ineffective reflection in large reasoning models. Existing methods often rely on fixed strategies to control reflection, which may lead to either insufficient error correction or unnecessary reasoning. To address this issue, the authors propose ReTrail, a training-free method that uses the probability of reflection tokens to decide when and where to revise the reasoning process. When the model shows increased hesitation, ReTrail rolls back to an earlier step and regenerates the reasoning, while adjusting reflection-token probabilities to encourage correction and avoid repetitive thinking. Experiments on four reasoning models across math and coding benchmarks show consistent improvements over existing methods, with an average accuracy gain of 3.25 percentage points at the cost of 15.2% more generated tokens.

Critical strengths
  1. The paper focuses a meaningful problem in test-time reasoning. The paper focuses on under-reflection and over-reflection in large reasoning models. While existing methods often rely on fixed strategies to control reflection, ReTrail uses signals from the model's reasoning process to decide when to revise earlier steps. This is a meaningful direction compared with simply increasing the number of reasoning tokens.

  2. The method is simple. ReTrail uses the model's own reflection-token probabilities to determine when to roll back and where to resume generation. It does not require training an additional verifier or reward model, nor does it need to maintain multiple reasoning trajectories. This makes the method relatively simple and potentially useful in practice.

  3. The experimental evaluation covers multiple models and tasks. The authors evaluate ReTrail on four reasoning models across four mathematics and coding benchmarks. The paper also provides several ablation studies examining the effects of rollback, reflection bias, trigger selection, and rollback location. In particular, the comparisons of different rollback locations provide additional evidence supporting the proposed design.

Critical weaknesses
  1. While the proposed method combines hesitation-based triggering, rollback, and reflection-token biasing, several of these techniques have been explored in previous work. Although the authors provide some ablation studies, further evidence is needed to demonstrate the unique contribution of hesitation-based decisions compared with existing approaches.

  2. The main experiments primarily compare ReTrail with methods such as TIP and CyclicReflex, which regulate reflection without backtracking. Since backtracking is a key component of ReTrail, comparisons with existing backtracking-based reasoning methods would provide stronger evidence of its effectiveness.

Questions for authors

Please see the Weakness. And I also have the following concern:

  1. Although ReTrail achieves the best or tied-best average accuracy across all evaluated settings, some improvements are relatively small. In Appendix F, several reported confidence intervals include zero, suggesting that the observed improvements may not be statistically significant. More extensive experiments would help establish the reliability of these results.
3 comments

We thank the reviewer for the constructive comments. We respond to each point below.

W1: The components have been explored before; more evidence is needed for the unique contribution of hesitation-based decisions.

Thank you for raising this point. ReTrail differs from prior work in how the decision is made. A single signal that the model already produces, the hesitation mass, decides both when to intervene and where to restart. It does not require training, a verifier, or parallel sampling. Existing backtracking methods make this decision with a trained model (process reward models, TAAR) or with entropy (Entropy-Cut MH, entropy-gated branching). In the experiments below, replacing hesitation with entropy or with a random choice lowers accuracy for both decisions.

We reran these comparisons on Qwen-7B with three seeds (90 responses on AIME 2026, 240 on LiveCodeBench). Leads are reported in percentage points (pp).

(1) When to intervene. We replace only the trigger signal and keep rollback and bias unchanged (Table R1).

Table R1. Trigger signals compared (Qwen-7B, three seeds).

MethodAIME 2026LiveCodeBenchReTrail's lead in pp, mean ± SD over seeds (AIME / LCB)
ReTrail47.78 (43/90)39.58 (95/240)n/a
w/ entropy trigger41.11 (37/90)37.92 (91/240)+6.67 ± 3.33 / +1.67 ± 1.91
w/ random trigger (same firing rate)36.67 (33/90)34.17 (82/240)+11.11 ± 5.09 / +5.42 ± 4.02
w/ entropy trigger + entropy-peak rollback40.00 (36/90)34.17 (82/240)+7.78 ± 1.92 / +5.42 ± 1.44

These variants intervene 2.56 to 2.72 times per problem, at least as often as ReTrail (2.40 on AIME 2026 and 1.93 on LiveCodeBench, Section 5.4). ReTrail's first seed here is a separate run made together with the variants, so its LiveCodeBench total (95) is two problems above Table 1 (93).

(2) Where to restart. We fix the initial trace and the trigger point and change only the restart location (Table R2, three continuations per problem). The no-intervention row is newly added in this rebuttal.

Table R2. Restart locations compared (Qwen-7B, fixed trigger).

MethodAIME 2026LiveCodeBench
No intervention (continue from the trigger point)47.78 (43/90)35.42 (85/240)
Restart at the hesitation peak (ReTrail)52.22 (47/90)37.92 (91/240)
Restart at the entropy peak44.44 (40/90)35.83 (86/240)
Restart at a distance-matched random step42.22 (38/90)35.83 (86/240)

Only the hesitation-selected location is above no intervention on both benchmarks. On AIME 2026, the entropy and random locations fall below it.

We will add these multi-seed results to Sections 5.4 and 5.5 and make the distinction from prior work explicit in Related Work.

W2: The main baselines (TIP, CyclicReflex) do not backtrack; comparisons with backtracking-based methods would be stronger evidence.

We agree that a comparison with backtracking-based methods is important. Backtracking is central to ReTrail, and we tested it against other ways of backtracking. ReTrail is training-free and works on a single trajectory. Published backtracking methods fall into two groups. The first fine-tunes the reasoning model [1] or trains a separate process reward model or diagnostic model (TAAR) to choose the restart point, so its accuracy also reflects that extra model and its training data. The second chooses the restart point by entropy (Entropy-Cut MH, entropy-gated branching), and we test its selection rule in our setting below. We chose TIP and CyclicReflex as the main baselines as they share ReTrail's setting.

We compared ReTrail against two training-free backtracking rules on Qwen-7B over three seeds (Table R3).

Table R3. Training-free backtracking rules (Qwen-7B, three seeds).

MethodAIME 2026LiveCodeBenchReTrail's lead in pp, mean ± SD over seeds (AIME / LCB)
Original (no backtracking)46.67 (42/90)37.08 (89/240)see Table R4
ReTrail47.78 (43/90)39.58 (95/240)n/a
Entropy-based backtracking (entropy decides when and where)40.00 (36/90)34.17 (82/240)+7.78 ± 1.92 / +5.42 ± 1.44
Restart from the beginning at the first trigger40.00 (36/90)35.83 (86/240)+7.78 ± 3.85 / +3.75 ± 2.17

Entropy-based backtracking applies the entropy selection rule of Entropy-Cut MH and entropy-gated branching in our training-free, single-trajectory setting. All three backtracking rules use the same regeneration bias, and the two alternatives intervene at least as often as ReTrail.

ReTrail is ahead of both rules in all three seeds we ran on both benchmarks. Both rules fall below Original while ReTrail is above it, which suggests that the benefit comes from choosing the restart point by hesitation.

We will add Table R3 to Section 5.4 and explain the choice of baselines in Section 5.1.

[1] Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models

Q1: Some improvements are small and several confidence intervals in Appendix F include zero; more extensive experiments would help establish reliability.

Thank you for pointing this out. During the rebuttal period, beyond the three-seed results in Tables R1 to R3, we ran three new seeds (seeds 4 to 6) for Original and ReTrail on Qwen-7B on all four benchmarks, so the main comparison now has six seeds (Table R4).

Table R4. Original vs. ReTrail over six seeds (Qwen-7B).

BenchmarkOriginal, six seedsReTrail, six seedsGain, six seeds (pp)Gain, new seeds only (pp)
AIME 202548.33 (87/180)50.56 (91/180)+2.22+2.22
AIME 202645.56 (82/180)47.22 (85/180)+1.67+2.22
HMMT 202636.87 (73/198)39.39 (78/198)+2.53+3.03
LiveCodeBench36.88 (177/480)38.75 (186/480)+1.88+2.08
All four40.37 (419/1038)42.39 (440/1038)+2.02+2.31

The gain holds on the new seeds. ReTrail scores higher than Original on all four benchmarks and in the three new seeds. We agree that the interval on a single benchmark of 30 to 80 problems is wide. Across the four benchmarks, however, the gain is consistent and stays at about 2 pp over six seeds. The six-seed results will be added to Appendix F.

AuthorsOpenReview ↗

Follow-up from the Authors

Dear Reviewer xMwV,

Thank you again for your time and your constructive review. As the discussion period is drawing to a close, we hope our response resolves your concerns and questions. In brief, we added:

  1. Hesitation vs. other signals (W1). We replaced hesitation with entropy or a random choice over three seeds, and accuracy drops for both decisions, when to intervene and where to restart (Tables R1, R2).
  2. Backtracking baselines (W2). We compared ReTrail with two training-free backtracking rules, and it leads both in every seed on both benchmarks (Table R3).
  3. Reliability (Q1). We ran three new seeds for the main comparison, six in total, and ReTrail stays about 2 pp above Original on all four benchmarks (Table R4).

In our view, these results strengthen the paper's central claim that the gain comes from letting the model's own hesitation decide when and where to backtrack, not from backtracking alone, and it persists with twice as many seeds.

We understand this is a busy time and that there may be no chance for further exchange. We would simply be grateful if you could keep these results in mind as you form your overall view of our work. Your comments have made the paper stronger.

Best regards, The Authors

Comments on the paper

2 comments

We sincerely thank the reviewers and the Area Chair for the careful and constructive feedback. The shared questions are (1) whether the gains are reliable across seeds, (2) whether they come from the hesitation signal or from backtracking, early timing, or extra tokens in general, and (3) what ReTrail costs at equal compute and on hosted models. In short, the gain over Original stays at about 2 percentage points (pp) over six seeds. Accuracy drops when hesitation is replaced with entropy or a random choice, and extra tokens alone do not produce the gain. Please see the individual responses for the full tables.

1. The gain over Original holds over six seeds. We extended the main comparison from three seeds to six on Qwen-7B, where the gains in Table 1 are among the smallest. ReTrail leads Original on all four benchmarks, by 2.02 pp overall (440 vs. 419 of 1038). In each of the three new seeds it solves more problems than Original (4, 5, and 3 more of 173). The mean gain over these three seeds is 2.31 pp (95% interval [+0.88, +3.75]; reply to 7W98, W2). A single benchmark has only 30 to 80 problems, so we base this conclusion on the pooled result.

2. The gain comes from the hesitation signal. All results below are on Qwen-7B over three seeds, reported as AIME 2026 / LiveCodeBench.

  • When to intervene. With the rest of the method fixed, replacing hesitation as the trigger lowers accuracy from 47.78 / 39.58 to 41.11 / 37.92 (entropy) and 36.67 / 34.17 (random, same firing rate).
  • Where to restart. With the trigger fixed, only the hesitation-selected step is above no intervention on both benchmarks (52.22 / 37.92 vs. 47.78 / 35.42). Entropy-selected and random restarts fall to 44.44 and 42.22 on AIME 2026.
  • Backtracking and early timing in general. Over the 330 responses, ReTrail solves 138 and Original 131. Entropy-based backtracking solves 118, restarting from the beginning 122, and one rollback after the first few steps 131, so none of them improves on Original.
  • Components. The gain requires the full method. In the three-seed ablation, ReTrail is the only configuration above Original, and it is ahead of every single-component variant (106 to 123 solved) in each seed on both benchmarks. Rollback and the two-sided bias therefore act as one mechanism, in which rollback gives the model a place to revise, promotion makes it reconsider there, and suppression lets it move on (reply to 7W98, W1).

3. Extra tokens do not explain the gain, and the hosted cost is modest. At equal total generation on Qwen-7B over three seeds, s1 scores 41.11 / 36.25, which is below Original (46.67 / 37.08) and below ReTrail (47.78 / 39.58). On the hosted models, ReTrail generates 2.6% to 16.2% more tokens than Original, with deleted tokens counted, and sends no extra requests to read probabilities. Each rollback on a hosted interface also resends the kept text, 17.5% to 25.8% of Original's length. This is billed as input on two models and as output on Speciale, where the billed output is about 1.27 times that of Original (reply to CFJr, W2 and Q2).

We thank the reviewers again for their time and feedback, which have made the narrative of the paper clearer.

AuthorsOpenReview ↗

Follow-up from the Authors

Dear AC and Reviewers,

Thank you for the time and care you have given to our submission. As the discussion period is drawing to a close, we would like to briefly restate why we believe the paper, together with the additional experiments, offers useful insight and a concrete contribution to the community working on reasoning models:

  1. Formulation. We cast reflection control as two linked decisions, when to revise and where to return, and let one signal the model already produces decide both: the hesitation mass, the probability it assigns to reflection words such as "Wait" at the end of each reasoning step.
  2. Method. We propose ReTrail, which rolls back to the most hesitant recent step when hesitation rises sharply and regenerates with reflection first encouraged and then damped, with no training, verifier, or parallel sampling.
  3. Results. Across four reasoning models and four math and coding benchmarks, ReTrail attains the highest or tied-highest mean accuracy in all 16 settings, averaging +3.25 percentage points (pp) over unmodified decoding (up to +8.08 pp) for 15.2% more generated tokens.
  4. New evidence. The additional experiments strengthen three points: the gain is stable (six seeds on Qwen-7B, about +2 pp on every benchmark), it comes from the hesitation signal (replacing it with entropy or a random choice lowers accuracy for both decisions), and it is not explained by extra tokens (forcing longer reasoning to the same token count, s1, stays below unmodified decoding).

The full tables are in our individual responses. We thank you again for your time and feedback, which have made the paper more solid.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.