acceptodds
Under review as a conference paper at ICLR 2027

Escaping Answer Anchoring: Fixing Shortcut Tokens in Reasoning Models via Context- Replay Gaps

Abstract

Long-form reasoning models are increasingly deployed as autonomous agents, but longer inference and larger context windows do not ensure they use the right evidence. We find that even after extensive reinforcement learning with verifiable rewards (RLVR), reasoning models remain anchored to their earlier answers: when a previous attempt is wrong, they typically restate it rather than produce a different answer. Outcome-level RLVR supplies no direct token-local label, while standard supervised fine-tuning (SFT) weights valid target positions uniformly; neither identifies which tokens depend on the preceding solution. We introduce the context-replay gap, a per-token diagnostic that compares each token’s log-likelihood when the preceding solution is present in context with its log-likelihood when it is absent. Tokens whose likelihood is strongly increased by the replayed solution are separated from those receiving little or negative likelihood change. The gap guides Context-Replay Guided Tuning (CRT) by selecting tokens for an ordinary negative log-likelihood (NLL) loss. This targeted supervision improves mathematical reasoning across model families and benchmarks. Behavioral interventions show that CRT reduces dependence on previously stated answers while preserving sensitivity to the reasoning that supports them. The context-replay gap therefore provides both a diagnostic of answer anchoring and a practical signal for post-training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Chain-of-Thought Reasoning, Answer Anchoring, Self-Correction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_53RPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies answer anchoring in reasoning models: the tendency to repeat an earlier answer even when it is incorrect, despite extensive RLVR training. The authors introduce Context-Replay Gap, a token-level metric that measures how token log-likelihood changes when a previous response is added to the context. They then propose Context-Replay Guided Tuning (CRT), which applies negative log-likelihood (NLL) supervision only to tokens with the lowest gaps. Experiments on three reasoning model families and four mathematical reasoning benchmarks show consistent accuracy gains over standard SFT and other token-selection baselines. Behavioral analyses further suggest that CRT reduces reliance on previous answers while largely preserving sensitivity to prior reasoning.

Critical strengths
  1. Context-Replay Gap provides a novel way to quantify how previous responses influence token predictions. The resulting CRT method is straightforward to implement and improves reasoning accuracy through selective NLL supervision.
  2. The improvements are consistent across three model families and four mathematical reasoning benchmarks. The paper also includes extensive ablations and behavioral interventions to examine how the method works.
Critical weaknesses
  1. Context-Replay Gap measures the influence of the entire previous response, so it does not isolate the effect of the previous answer itself. It is therefore unclear whether selecting low-gap tokens specifically addresses answer anchoring or captures broader contextual effects.
  2. CRT improves accuracy but also produces substantially longer reasoning trajectories. The reported ablations do not fully separate the benefits of reduced answer anchoring from those of additional inference computation.
Questions for authors
  1. What evidence would help distinguish whether Context-Replay Gap captures answer anchoring specifically or broader contextual effects, such as changes in response length or reasoning behavior?
  2. Could you compare CRT with the baselines under matched inference token budgets? This would help clarify how much of the accuracy gain comes from longer reasoning.
  3. Since CRT applies NLL supervision to self-generated trajectories without correctness rewards, how do correct and incorrect trajectories each contribute to the gains? Would training only on correct trajectories yield similar results?
  4. CRT improves single-turn accuracy, but the additional gains from multi-turn self-correction appear relatively small. How should we interpret this finding in relation to the claim that CRT reduces answer anchoring, as opposed to improving reasoning ability more generally?
4 comments

length-controlled ablations

(W2). CRT improves accuracy but also produces substantially longer reasoning trajectories. The reported ablations do not fully separate the benefits of reduced answer anchoring from those of additional inference computation.

(Q2). Could you compare CRT with the baselines under matched inference token budgets? This would help clarify how much of the accuracy gain comes from longer reasoning.

Because CRT naturally increases response length, we additionally optimize CRT with an RL objective that improves reward while penalizing excessively long generations. In the table, “CRT, 20k-length-RL, 314step” denotes CRT followed by length-constrained RL using a 20K-token length setting, evaluated at training step 314; the 18K variant uses a tighter length setting. “no CRT, 20k-length-RL” applies the same RL objective without CRT, controlling for gains from reward optimization alone, while “CRT, no RL” is the original CRT model without length control. At nearly unchanged length (+2.4%), CRT improves four-benchmark average accuracy by 2.20 percentage points, whereas RL without CRT yields only a 0.51-point gain. Larger gains still require some additional inference computation: longer CRT length-RL training improves accuracy by 3.38 points with only a 7.8% token increase, substantially below the 56.2% increase incurred by CRT-only training.

Table: length-controlled ablations.

ModelBenchmarkMean TokensTok. Increase vs. BaseAccuracy
BaseAIME2412,832—68.85
AIME2514,321—59.58
HMMT14,649—37.92
HMMT-Nov13,822—42.66
4-set avg.13,906—52.25
no CRT,20k-length-RL, 156stepAIME2412,179−5.1%69.90
AIME2513,683−4.5%61.67
HMMT14,065−4.0%38.96
HMMT-Nov13,017−5.8%40.52
4-set avg.13,236−4.8%52.76
CRT, 18K-length-RL, 156stepAIME24<br>12,475−2.8%72.40
AIME2514,275−0.3%62.71
HMMT14,747+0.7%38.96
HMMT-Nov15,479+12.0%43.75
4-set avg.14,244+2.4%54.45
CRT, 20k-length-RL, 314stepAIME2413,581+5.8%72.71
AIME2515,551+8.6%63.65
HMMT15,918+8.7%41.67
HMMT-Nov14,939+8.1%44.48
4-set avg.14,997+7.8%55.63
CRT, no RL (main table)AIME2419,820+54.5%74.11
AIME2523,241+62.3%66.20
HMMT23,069+57.5%43.28
HMMT-Nov20,792+50.4%47.76
4-set avg.21,731+56.2%57.84

(Q1). Context-Replay Gap measures the influence of the entire previous response, so it does not isolate the effect of the previous answer itself. It is therefore unclear whether selecting low-gap tokens specifically addresses answer anchoring or captures broader contextual effects.

(W1). What evidence would help distinguish whether Context-Replay Gap captures answer anchoring specifically or broader contextual effects, such as changes in response length or reasoning behavior?

  1. Length-Controlled. We do not entirely rule out the possibility that CRT benefits from other factors. However, these reasoning-related factors are difficult to quantify. For the readily measurable effect of response length, our preceding rebuttal shows that CRT can improve reasoning accuracy while keeping inference length nearly unchanged.

  2. Reduced Answer Copying. We further evaluate the length-controlled models from the preceding analysis under the two-pass setting. Even after controlling for length, the CRT variants retain a strong tendency to correct prior errors, achieving wrong-answer copying rates of 69.32 and 55.87, respectively. This suggests that CRT’s resistance to answer anchoring persists independently of increased inference length.

ModelP1 Acc.P2 Acc.Δ Acc.Copy | P1 CorrectCopy | P1 WrongP2 Correct | P1 Wrong
Base59.5859.79+0.2197.6481.963.99
no CRT,20k-length-RL, 156step61.6761.25−0.4297.4783.972.99
CRT, 18K-length-RL, 156step62.7163.54+0.8396.6469.327.67
CRT, 20k-length-RL, 314step63.6564.27+0.6395.0955.8710.32
  1. Answer vs. Reasoning. We agree that the raw context-replay gap may capture broad contextual differences, including response length and changes in reasoning behavior. To isolate answer anchoring more directly, we added a crossed 2$\times$2 intervention that independently varies the correctness of the previous reasoning and terminal answer while holding the question, Q-A1-Q structure, and evaluation protocol fixed. The results appear in Table 12.
Constructed A1A_1 historyBase A2A_2 Acc.CRT A2A_2 Acc.
Correct reasoning + correct answer88.8282.89
Correct reasoning + wrong answer59.2170.39
Wrong reasoning + correct answer51.3238.82
Wrong reasoning + wrong answer11.8431.58
Answer-correctness main effect+34.54 pp+9.87 pp
Reasoning-correctness main effect+42.43 pp+41.45 pp

For each answer contrast, we hold the question, reasoning trace, and prompt structure fixed, replacing only the terminal answer. We also mask answer-equivalent spans in the reasoning and check for residual leakage. CRT reduces the resulting answer effect from 34.5434.54 to 9.879.87 pp, while leaving the reasoning main effect nearly unchanged at 42.4342.43 versus 41.4541.45 pp. If CRT broadly downweighted prior context, both effects would contract. Instead, CRT preserves sensitivity to reasoning correctness while substantially reducing sensitivity to the terminal answer.

  1. Broader Reasoning Improvement. However, we also find evidence that CRT may also induce broader reasoning improvements beyond answer anchoring. Using a broad-first extraction rule, we identify the model’s first answer within a single reasoning attempt, defined as either the first boxed answer or the first textual answer declaration eg˙,˙“theansweris”e\.g\., “the answer is”, and compute accuracy based on these answers. CRT models also improve under this metric, suggesting that CRT training may also enhance reasoning capabilities beyond iterative correction.
ModelAIME24AIME25HMMTHMMT-NovAvg.Δ vs. Base
Base62.2955.3136.3041.5148.85—
No CRT-20k s15663.6557.8136.8839.5849.48+0.63
CRT-18k s15666.2558.7536.2542.2950.89+2.04
CRT-20k s31466.2558.7540.0042.1951.80+2.95
Main CRT s5066.7260.5240.5746.3553.54+4.69

Overall, performance largely follows the ordering Base < CRT-18k < CRT-20k < Main CRT.

(Q3) Since CRT applies NLL supervision to self-generated trajectories without correctness rewards, how do correct and incorrect trajectories each contribute to the gains? Would training only on correct trajectories yield similar results?

We compared training on all-positive versus all-negative examples. The all-positive variant performed slightly better, but the difference was not significant.

Training dataStepAIME24AIME252-set Avg
ACE Base072.4065.7369.07
Original data4074.5871.8873.23
Original data5074.5870.9472.76
AllPos4074.7971.8873.33
AllPos5075.1071.7773.44
AllNeg4075.8370.7373.28
AllNeg5074.5869.3871.98

(Q4) CRT improves single-turn accuracy, but the additional gains from multi-turn self-correction appear relatively small. How should we interpret this finding in relation to the claim that CRT reduces answer anchoring, as opposed to improving reasoning ability more generally?

Our fourth analysis addressing Q1/W1, Broader Reasoning Improvement, evaluates the accuracy of the model’s first answer claim. We next examine answer anchoring within a single reasoning trajectory by comparing the first and final answer claims in each response. Using template-based extraction, we identify the first and final answer claims in each response and compute their respective accuracies. A larger Broad-final minus Broad-first gain indicates a stronger net tendency to move beyond an early answer claim and reach the correct final conclusion. CRT consistently exhibits larger gains than its corresponding Base model: +4.30 pp versus +3.40 pp for JustRL and +6.38 pp versus +5.26 pp for ACE. These results suggest that CRT reduces answer anchoring not only across turns but also within a single reasoning trajectory.

Family / ModelBroad-firstBroad-finalΔRelative gain
JustRL Base48.8552.25+3.40pp+6.96%
JustRL No CRT-20k-RL 156step49.4852.76+3.28pp+6.63%
JustRL CRT-20k-RL 314step51.8055.62+3.82pp+7.37%
JustRL CRT53.5457.84+4.30pp+8.03%
ACE Base54.4859.74+5.26pp+9.66%
ACE CRT57.8164.19+6.38pp+11.04%
Reviewer_53RPOpenReview ↗

Response to Authors

Thank you for the detailed rebuttal and additional experiments. The length-controlled results and behavioral analyses help address some of my concerns.
However, while CRT appears to reduce reliance on previous answers, it remains unclear whether mitigating answer anchoring is the primary driver of the accuracy gains, given the substantial improvements in first-answer accuracy and relatively modest self-correction gains. Thus, my main concern regarding the underlying mechanism remains only partially resolved.

Reviewer_zhfzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a way to select tokens in SFT training by constructing a context window (x,y,x,y) and measure the log likelihood difference between the first appearance and second appearance of tokens in the response. Then show that training with SFT on only 10% of the tokens show improved perfomance than full SFT.

Critical strengths

The paper proposed a straightforward way of selecting tokens for SFT training and showed that the proposed token selection method outperforms full SFT on four reasoning datasets.

Critical weaknesses
  • Given a training dataset of instances (x, y), the paper proposes constructing a context window xyxy and measure the difference in loglikelihood in the two appearances of y. The concern is that, since the two responses are exactly the same, the loglikelihood of the second appearance is affected by the model recognizing the repeating pattern and "learns" to simply copy the previous response. One experiment worth doing is to investigate how the gap changes at different positions of the response. One can imagine that it may be unclear at the beginning of the second appearance of the response that we are copying, but it can be clearer as the copying continues, so maybe it's easier for the model to do the copying instead of reasoning at later positions of the second appearance of the response, and there may be a position bias in the proposed token selection pipeline?
  • Table 2 shows that random 10% token selection can sometimes outperform full SFT, especially on ACE-7B, random 10% outperforms full SFT on all four benchmarks. The paper proposes a token selection method for SFT, but didn't discuss much why SFT would benefit from token selection in the first place. More analysis on this would be helpful.
Questions for authors
  • Figure 2 shows that when a correct answer is inserted in the first turn, CRT performs worse than the base model, why?
2 comments

Position analysis of selected tokens.

(W1) Given a training dataset of instances (x, y), the paper proposes constructing a context window xyxy and measure the difference in loglikelihood in the two appearances of y. The concern is that, since the two responses are exactly the same, the loglikelihood of the second appearance is affected by the model recognizing the repeating pattern and "learns" to simply copy the previous response. One experiment worth doing is to investigate how the gap changes at different positions of the response. One can imagine that it may be unclear at the beginning of the second appearance of the response that we are copying, but it can be clearer as the copying continues, so maybe it's easier for the model to do the copying instead of reasoning at later positions of the second appearance of the response, and there may be a position bias in the proposed token selection pipeline?

We thank the reviewer for this testable hypothesis. As shown in the following table, low-gap selected tokens are nearly position-uniform on the actual SFT pool. All deciles fall within 0.75–1.16× the 10% baseline, and the first decile is below baseline at 9.07%. The largest enrichment (1.16×) occurs mid-response rather than at the start or the end, with a mean relative position of 0.487. The only pronounced enrichment is rear-loaded and belongs to a different selector: low replay log-probability reaches 13.05% in the 80–90% bin.

Table: Relative-position distribution of selected tokens (uniform baseline = 10.00% per bin)

Position binLow gapLow log prob (regular)Low replay log prob
[0.0, 0.1)9.078.886.25
[0.1, 0.2)9.8310.528.23
[0.2, 0.3)11.6010.6710.73
[0.3, 0.4)10.9210.159.99
[0.4, 0.5)10.6910.8810.45
[0.5, 0.6)9.9210.7210.91
[0.6, 0.7)10.6510.6611.77
[0.7, 0.8)10.1710.6812.13
[0.8, 0.9)9.6511.2713.05
[0.9, 1.0)7.525.586.48
Mean relative position0.4870.4890.524
Front-half share (%)52.1051.0945.67
Max bin (position / ratio)20–30% / 1.16×80–90% / 1.13×80–90% / 1.31×

We also conduct training experiemtns on positions:

  • Pos-balanced selector: each response is split into 10 equal-width position bins, each receiving an equal token quota filled by the lowest-gap tokens within that bin, forcing uniform position marginals while preserving gap ranking.
  • Prefix-restricted random mask: the identical 10% random token budget, but candidates are drawn only from the first half of each response, retaining front-loaded positions while removing gap-based selection entirely.

Table: Postion Control Experiments

BenchmarkFull low-gapPos-balancedPrefix-random
AIME 202474.1175.1068.33
AIME 202566.2066.6759.27
HMMT43.2842.0839.58
HMMT-Nov47.7147.2942.50
Average57.8357.7952.42

Balancing positions leaves the gain essentially unchanged (average 57.79 vs. 57.83), whereas restricting supervision to front positions yields little gain.

(W2) Table 2 shows that random 10% token selection can sometimes outperform full SFT, especially on ACE-7B, random 10% outperforms full SFT on all four benchmarks. The paper proposes a token selection method for SFT, but didn't discuss much why SFT would benefit from token selection in the first place. More analysis on this would be helpful.

This is because SFT on self-generated trajectories resembles entropy minimization. Continued over-optimization can make the model increasingly confident in its existing predictions, eventually causing performance to decline. Random token selection only slows this process. Lowering the learning rate (5e-7) has a similar effect. We do not claim that random-token SFT is superior or inferior to full SFT. Random SFT is included only as a control for varying the sparsity ratio in the comparison with CRT.

Table: AIME25 step dynamics

SettingEarlyMiddleLate
Full SFT, lr = 1e-610 steps: 62.620 steps: 64.5830 steps: 63.12
Random 10%, lr = 1e-610 steps: 65.020 steps: 64.3830 steps: 63.23
Full SFT, lr = 5e-78 steps: 65.1016 steps: 65.5224 steps: 63.75 / 32 steps: 63.23

Table: AIME24 step dynamics

SettingEarlyMiddleLate
Full SFT, lr = 1e-610 steps: 73.7520 steps: 72.7130 steps: 72.29
Random 10%, lr = 1e-610 steps: 74.2720 steps: 72.9230 steps: 73.23
Full SFT, lr = 5e-78 steps: 72.6016 steps: 73.1224 steps: 73.12 / 32 steps: 73.23

(Q1) Figure 2 shows that when a correct answer is inserted in the first turn, CRT performs worse than the base model, why?

We thank the reviewer for this question and have extended the Figure 2 same-turn intervention to two additional datasets (AIME 2024 and HMMT) in the table below. A trade-off is intrinsic to this setting: because CRT weakens adherence to the pre-stated answer and revises it more often, some initially correct tentative answers are overwritten as well. At the extreme, a model that never changes its answer would score 100% when the tentative answer is correct but 0% when it is wrong, accuracy under the correct-answer condition alone therefore cannot measure robustness; the relevant quantity is the balance between the two sides of this trade-off. Across all three datasets, the degradation under correct tentative answers (−6.91, −0.33, and −8.64 pp) is far smaller than the recovery under wrong ones (+29.27, +19.90, and +19.85 pp). The net effect is thus strongly positive: CRT exchanges a small loss of answer-copying benefit for a much larger gain in resistance to wrong-answer anchoring, which is consistent with its improved ordinary accuracy.

Table: Accuracy under correct vs. wrong tentative answers.

Accuracy %AIME25AIME24HMMTMacro mean
Correct tentative Base/CRTBase / CRT77.47 / 70.5687.50 / 87.1787.68 / 79.04—
Degrade Δ (correct condition)−6.91−0.33−8.64−5.29
Isolated wrong tentative Base/CRTBase / CRT26.32 / 55.5954.61 / 74.5119.30 / 39.15—
Recover Δ (wrong condition)+29.27+19.90+19.85+23.01
Reviewer_VzhDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Context-Replay Guided Tuning (CRT), which selects training tokens using their log-likelihood difference with and without a replayed complete response. It applies ordinary supervised fine-tuning to the lowest-gap tokens under regular context. Experiments on three reasoning checkpoints show improvements on four mathematical benchmarks, while behavioral interventions indicate reduced dependence on previously stated answers.

Critical strengths
  1. The method is simple and empirically effective. Low-gap selection consistently outperforms alternative NLL token selectors across three reasoning checkpoints, without additional inference-time machinery.
  2. Cross-turn and same-turn interventions support reduced answer anchoring, while the ablations distinguish low-gap selection from random sparsity, uncertainty, and explicit revision markers.
Critical weaknesses
  1. Each benchmark contains only 30 problems, key interventions use 19 AIME25 questions, and results across independent training seeds are not reported.
  2. Experiments on newer and larger models would clarify whether answer anchoring remains substantial and whether CRT’s benefits persist as reasoning capability improves.
Questions for authors
  1. Do the main gains and behavioral findings persist across training seeds and a larger intervention set outside AIME25?
3 comments

Multi-seed robustness, larger evaluation set and generality of the 19-question behavioral panel (1/2)

  1. W1W1 Each benchmark contains only 30 problems, key interventions use 19 AIME25 questions, and results across independent training seeds are not reported.

  2. Q1Q1 Do the main gains and behavioral findings persist across training seeds and a larger intervention set outside AIME25?

11 Multi-seed robustness.

We repeated the main experiment with two additional independent training seeds:

RunAIME24AIME25HMMTHMMT-NovAvg.
Seed 4275.62565.62543.75047.29258.073
Seed 4375.20866.56243.33344.68857.448
Seed 4174.11566.19843.28147.70857.826
Mean74.98366.12843.45546.56357.782
± Std.±0.780±0.472±0.257±1.637±0.315

The results are reproducible. The two additional runs differ by only 0.625 percentage points in average accuracy mean:577˙60mean: 57\.760%, and their mean is within 0.07 percentage points of the Seed-41 result reported in the main paper 578˙2657\.826%. Given the high training and evaluation costs of the other two models, we will complete the remaining CRT runs with the full set of training seeds and include the results in a subsequent revision.

(2) large-scale held-out evaluation (674 questions).

To address the concern that our main benchmarks contain only 30 questions, we evaluate Base, CE, and CRT on an additional held-out test set of 674 questions OlympiadBench. Every question is decoded with 4 independent rollouts. Accuracy is averaged per question over the 674-question set.

ModelAccuracy (674 questions)Δ vs. Base
Base76.8965.625
CE77.00+0.11 pp
Seed 4178.45+1.56 pp

The ranking observed on the four small benchmarks is preserved on this larger set: CRT > CE ≈ Base, with CRT improving over Base by +1.56 pp .

Multi-seed robustness, larger evaluation set and generality of the 19-question behavioral panel (2/2)

  1. W1W1 Each benchmark contains only 30 problems, key interventions use 19 AIME25 questions, and results across independent training seeds are not reported.

  2. Q1Q1 Do the main gains and behavioral findings persist across training seeds and a larger intervention set outside AIME25?

33 Generality beyond the 19-question AIME25 intervention panel.

We thank the reviewers for raising this concern. We replicated the full same-turn intervention protocol (Analysis 3) on two additional datasets, using the identical setup: the same heterogeneity-selection criterion questionswithbothcorrectandincorrectBaserollouts,allowingcontrolledcorrect/wrongsourceconstructionquestions with both correct and incorrect Base rollouts, allowing controlled correct/wrong source construction, the same Base and CRT checkpoints, the same decoding settings, and 32 matched rollouts per question-condition pair.

This expands the panel from 19 AIME25 questions to 55 questions across three datasets: AIME24 1919, AIME25 1919, and HMMT 1717.

Accuracy %AIME25 Base / CRTAIME24 Base / CRTHMMT Base / CRT
Isolated wrong tentative answer (no wrong reasoning)26.32 / 55.5954.61 / 74.5119.30 / 39.15
Complete wrong reasoning supplied0.33 / 0.490.00 / 0.000.00 / 0.18
Correct tentative answer77.47 / 70.5687.50 / 87.1787.68 / 79.04
Answer-only effect bracket (correct − wrong)51.15 / 14.9732.89 / 12.6668.38 / 39.89
Replacing only the last wrong step+13.49 / +14.80+15.95 / +22.53+8.82 / +13.79

All qualitative findings replicate consistently across the three datasets:

  1. CRT resists isolated wrong answers. When only a wrong tentative answer is provided, without supporting wrong reasoning, CRT outperforms Base by +29.27, +19.90, and +19.85 percentage points on AIME25, AIME24, and HMMT, respectively.

  2. Complete wrong reasoning overwhelms both models. Supplying a full incorrect reasoning trace drives accuracy near zero in all six conditions allall \leq 0.49%$$. CRT therefore does not provide immunity to incorrect evidence; its advantage is specifically in reducing answer-only anchoring. As incorrect reasoning steps are progressively restored, both models converge to this floor on every dataset.

  3. CRT selectively reduces answer-level anchoring while retaining sensitivity to reasoning. The answer-only effect bracket consistently contracts under CRT, from 51.15 to 14.97 points on AIME25, 32.89 to 12.66 on AIME24, and 68.38 to 39.89 on HMMT. At the same time, replacing only the final incorrect reasoning step remains comparably beneficial for both models. This shows that CRT attenuates reliance on the preceding answer while preserving ordered, step-level sensitivity to reasoning evidence.

  4. The anchoring attenuation is symmetric but strongly asymmetric in magnitude. When the tentative answer is correct, CRT tracks Base closely with a small cost: 70.56 vs. 77.47 on AIME25 −69˙1pp−6\.91 pp, 87.17 vs. 87.50 on AIME24 −03˙3pp−0\.33 pp, and 79.04 vs. 87.68 on HMMT −86˙4pp−8\.64 pp. The gains under a wrong tentative answer +292˙7,+199˙0,+198˙5pp+29\.27, +19\.90, +19\.85 pp are 2.3–60× the corresponding losses under a correct one.

  1. W2W2 Experiments on newer and larger models would clarify whether answer anchoring remains substantial and whether CRT’s benefits persist as reasoning capability improves.

We thank the reviewer for this suggestion. We add experiments on two newer 27B-scale families, Qwen3.5-27B and Qwen3.8-27B , and further continued CRT on the Qwen3.8-27B. The new evidence shows that the CRT benefit is preserved.

11 Anchoring on Qwen3.5/3.8-27B.

We measure the average context-replay gap, conditional copy rate, and self-correction rate:

ModelAvg. context-replay gapcopy-wrong 2−pass2-passself-correct 2−pass2-pass
Qwen3.5-27B0.050.660.02
Qwen3.8-27B0.400.440.28

Moving from Qwen3.5 to Qwen3.8 yields a substantial improvement in the context-replay gap. This may suggest that Qwen3.8’s training implicitly optimized gap-related behavior, or incorporated other training changes that improve the model’s ability to use contextual information. The 8× larger replay gap, lower copy-wrong rate, and higher self-correction rate indicate that the stronger model relies less on blindly reusing a prior answer and is substantially more capable of overturning an incorrect one.

22 Multi-pass results: CRT continues to provide gains.

For Qwen3.8, we run an n-pass experiment on APEX2025, where each new pass conditions on all preceding answers:

Qwen3.8-27BPass 1Pass 2Pass 3
Base52.8%62.5% +97˙+9\.766.7% +42˙+4\.2
CRT56.3%68.8% +125˙+12\.572.9% +42˙+4\.2

Two conclusions follow. the CRT benefit persists on the newer model: CRT raises both the absolute level pass−1528˙→563˙pass-1 52\.8→56\.3 and every multi-pass operating point, with a large and significant pass-2 gain +125˙pp+12\.5 pp.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.