acceptodds
Withdrawn from ICLR 2027

Conflict Steering: Resolving Late-Layer Disagreement Repairs Non-Termination in Greedy Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reasoning models are shipped with a warning not to decode them greedily: on a long trace a greedy decoder resolves every recurring near-tie in the same way, so it can loop or wander to the token limit. Sampling breaks such ties at random, but the output then depends on the draw. We show that the model's own late layers supply a deterministic tie-breaker: within a decodable band of the last layers, an intermediate layer sometimes commits to a token with low entropy and the final layer overrides it. We call these steps fights and resolve them by conflict steering, a spherical interpolation of the final hidden state toward the intermediate one, followed by ordinary hard-token decoding. Conflict steering repairs a failure mode, not a capability: the non-termination of a derivation the model can complete. Over four Qwen3 models spanning 4B to 32B one steered greedy decode wins 189 paired problems and loses 141 against greedy on AIME 2024 to 2026 and three code benchmarks. The gain sits where greedy leaves traces open: at 4B and 8B, where a fifth of AIME traces reach the token limit, it wins 112 and loses 70 and surpasses the accuracy of temperature-0.6 sampling on AIME in three of four serving configurations. On multiple-choice, near-ceiling and beyond-competence controls it predicts no gain and finds none. At 14B and 32B, where under a tenth of the traces are open, one decode gains little on code and, at 14B, stays below the sampled mean on AIME. The steering fallback, conflict steering only on non-terminating greedy traces, never loses in scope (92 wins, no losses over the four models). Under sampling, whose randomness already breaks ties, no training-free latent-reasoning method we evaluate, ours included, improves on plain sampling.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reasoning models, decoding, deterministic inference, greedy decoding, hidden-state steering, spherical interpolation, early-exit layers, repetition loops, test-time compute, entropy matching, Qwen3, AIME

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_WzNgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free decoding method to reduce non-termination in greedy reasoning. When confident intermediate layer disagrees with final layer, it interpolates the hidden states based on their distributional divergence. Experiments on Qwen3 math and code benchmarks show improvements. The authors also consider two alternatives: "always-on" steering where their proposed conflict steering is always utilized and "steering fallback" where conflict steering is used only if the token budget is exceeded during standard greedy decode.

Critical strengths

The problem targeted by the authors is of practical relevance. The proposed intervention is also simple and intuitive: requiring neither any training or some external verifier. Improvements across 4B/8B models across range of tasks and closure of repetition loops compared to alternate greedy decoding configurations demonstrate evidence that conflict steering works.

The paper covers good analysis of trace transitions, comparison against CLD and various baselines (such as presence penalty). Sensitivity to the model serving configurations are useful. Reporting negative results under sampling and unsuccessful transfer of always-on steering makes the experiments more informative.

Critical weaknesses

Method and resulting contribution builds on prior approach CLD heavily. CLD utilizes confident intermediate layer selection. Spherical steering is also a known approach. The main addition is therefore the disagreement-dependent interpolation. With Qwen3’s RMSNorm and linear output head, the proposed interpolation can also be expressed as a weighted mixture of intermediate and final logits for greedy token selection. The comparison against hard CLD supports mixing over hard replacement, but does not establish the importance of the particular spherical parameterization or JSD schedule.

The practical advantage over simpler repairs remains unclear, and the "steering fallback" cost appears understated. Appendix C reports that steering solves 15 of 35 greedy-open AIME problems, versus 16 for a second greedy configuration and 16 in expectation for one sampled retry. Thus, the strong fallback win/loss count does not establish a distinctive advantage. Moreover, the cost estimate neglects the length of the selected reruns.

Questions for authors

What evidence isolates the benefit of the JSD-controlled interpolation over a simpler approach using the same intermediate-layer selection? Please also clarify the endpoint normalization in Equation 1, given the norm-preservation claim.

Can the fallback cost be reconciled with the actual rerun token totals or measured runtime, and its practical advantage over the alternative retries already evaluated in Appendix C be clarified?

Reviewer_nnV3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Conflict Steering, a training-free decoding method intended to reduce non-termination in greedy reasoning. The method decodes a band of late intermediate layers, identifies an entropy-trough layer whose preferred token differs from the final layer’s prediction, and interpolates the final hidden state toward that intermediate state using a disagreement-dependent spherical interpolation. The resulting state is decoded into an ordinary hard token. Experiments cover four Qwen3 models, AIME, three code benchmarks, and several control tasks. The authors also propose a fallback that reruns only non-terminating greedy traces with steering. Results show improvements over greedy decoding in selected regimes, particularly smaller models with frequent non-termination, but limited benefits under sampling and inconsistent improvements on larger models and an additional distilled reasoning model.

Critical strengths
  1. The paper addresses a practical failure mode in reasoning-model inference: greedy decoding can fail to terminate even on problems the model can solve. Using disagreement between late layers to select an alternative deterministic trajectory is an interesting approach. The method requires neither additional training nor a verifier and preserves ordinary token-based generation.

  2. The experimental reporting is unusually transparent about the method’s limitations. The paper distinguishes termination gains from answer changes among completed traces, includes presence-penalty and answer-budget controls, reports serving-configuration sensitivity and inference overhead, and documents cases where steering hurts or fails to transfer. These analyses provide useful information beyond aggregate benchmark scores.

Critical weaknesses
  1. The practical advantage over simpler recovery strategies remains insufficiently established, particularly after accounting for inference cost. Appendix C reports that, on the 35 open AIME traces at 4B and 8B, another greedy serving configuration and one sampled rerun each recover 16 problems, compared with 15 for steering. Appendix F reports approximately half of greedy throughput at these scales. A single steered decode also remains below the sampled mean on 8B code (76.6 versus 81.1) and 14B AIME (72.2 versus 76.1). These results do not invalidate the restricted deterministic use case, but they leave its advantage over cheaper deterministic repairs unclear. The fallback’s 92 wins and zero losses is encouraging, yet preserving completed greedy outputs already protects most existing successes, so this result alone does not establish the value of the steering operator. The relevant comparison is recovery quality and end-to-end cost under the same fallback policy. Furthermore, the approximate cost formula 1 + 2r uses the fraction of requests requiring retries, although those requests have already exhausted the token budget and may be substantially longer than an average request. Measured total runtime is needed to establish the claimed cost advantage.

  2. The experiments do not sufficiently establish which components of conflict steering are responsible for the gains. The method combines entropy-trough selection, top-token disagreement, JSD-dependent interpolation strength, and spherical interpolation, but the principal operator comparison is against hard commitment to the intermediate token. This does not isolate whether the specific conflict rule and interpolation geometry outperform simpler ways of changing a greedy trajectory. The paper itself reports that steering diverges from greedy within a median of 38 or 77 tokens, well before repetition, and that fight frequency does not predict whether intervention helps. Thus, the evidence establishes a useful trajectory perturbation more clearly than a specific explanation of non-termination through late-layer conflict. Matched controls using linear hidden-state interpolation, logit interpolation, and deterministic perturbations with comparable intervention frequency are needed. This matters because always-on steering is essentially flat across the broader collection of tasks, loses accuracy on 32B code, and does not improve AIME on the additional distilled model.

Questions for authors
  1. Under the same fallback policy, how does steering compare with a sampled rerun, a second deterministic configuration, and a presence-penalized greedy rerun in recovered correct answers, total GPU time, and latency on failed requests? Appendix C provides informative partial comparisons; a complete cost-matched comparison would materially affect my assessment.

  2. What happens when spherical interpolation is replaced by linear hidden-state interpolation or logit interpolation, while keeping the selected layers and intervention steps fixed? How does the method compare with a deterministic perturbation that changes a comparable number of token decisions without using the entropy-trough/JSD rule? These controls would clarify the contribution of the proposed operator.

  3. Can the authors report measured end-to-end fallback cost rather than estimating it from the fraction of open requests? In particular, does the 1 + 2r approximation account for the substantially greater length of budget-exhausted traces?

These questions concern the central comparative and methodological claims, rather than additional application settings.

Reviewer_bEfKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates non-termination in greedy decoding of reasoning models and proposes Conflict Steering. The method detects disagreements between an entropy-trough intermediate layer and the final layer, then adjusts the final hidden state through spherical interpolation with a JSD-dependent strength. The authors also propose a fallback strategy that applies steering only when the initial greedy trace fails to terminate. Experiments report improvements on selected mathematical reasoning and code-generation tasks. However, I have concerns about the methodological novelty, the evidence supporting the proposed mechanism, and the robustness and computational cost of the improvements.

Critical strengths
  1. The paper addresses the practical problem of non-termination in greedy reasoning.
  2. The method requires no retraining or external verifier.
  3. The experiments include trace-level analysis and evaluations of the method's limitations.
Critical weaknesses
  1. The methodological contribution appears incremental, and the distinction from existing layer-selection and activation-steering techniques is not sufficiently established.
  2. The empirical gains are inconsistent across tasks and model scales, and the results do not establish a convincing performance–cost advantage over simpler decoding alternatives.
  3. The evidence does not sufficiently establish that the detected layer disagreements are specifically related to non-termination or that resolving them explains the observed improvements.
  4. The sensitivity analysis is too limited to justify the fixed thresholds and interpolation settings or establish their robustness across models and tasks.
  5. Generalization across independent model families and generation budgets remains insufficiently demonstrated.
  6. The ablations do not adequately isolate the contributions of spherical interpolation, JSD-dependent weighting, and the selected intervention direction.
Questions for authors

see weakness

Reviewer_hGo3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Reasoning models are shipped with sampling temperatures rather than greedy decoding, because a greedy decoder resolves every recurring near-tie identically and can cycle to the token limit. Reproducible evaluation and audited serving instead want one deterministic trajectory. The paper observes that within a band of the last M layers, decoded through the model's own final norm and unembedding, an intermediate layer sometimes commits to a token at an entropy trough and the final layer overrides it. It calls these steps fights, and resolves them by spherical interpolation of the final hidden state toward the intermediate one, followed by ordinary hard-token decoding.

Critical strengths

1.The problem is easy to follow and well-defined. Reproducible evaluation, audited pipelines, and cost-constrained serving need one deterministic trajectory, but greedy loops and a single sample is a random draw. 2.Reading the CLD entropy trough as a conflict signal rather than a commit target is a good improvement over the closest prior method.

Critical weaknesses

1.The contritbution over CLD comes down to one design choice, interpolate instead of commit, so the contribution seems to be narrow. 2.The mechanism does not hold at 32B. At 32B the AIME gain of 5.6 points comes through re-decisions rather than termination, which is the opposite of what the mechanism predicts. 3.Serving noise is comparable to the effect. The paper reports that at 8B, batch sizes 12 and 16 produce no identical AIME completions and solve 58/63 of 90. That is a 5.6% swing in greedy accuracy from floating-point reduction order alone, against a claimed 7.8% steering gain at that scale.

Questions for authors

The AIME-only comparison is 33 wins against 20 losses (p = 0.10). Do you consider the math claim established, and what would it take to power it? 2.Batch size moves greedy at 8B by 5.6 points. Can you report steering against the best of several greedy configurations rather than a single paired one, so the effect is measured against serving noise rather than one draw from it?3 3.Why does the 32B AIME gain come through re-decisions instead of termination? Does that change how the mechanism should be stated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.