- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a training-free decoding method to reduce non-termination in greedy reasoning. When confident intermediate layer disagrees with final layer, it interpolates the hidden states based on their distributional divergence. Experiments on Qwen3 math and code benchmarks show improvements. The authors also consider two alternatives: "always-on" steering where their proposed conflict steering is always utilized and "steering fallback" where conflict steering is used only if the token budget is exceeded during standard greedy decode.
Critical strengths
The problem targeted by the authors is of practical relevance. The proposed intervention is also simple and intuitive: requiring neither any training or some external verifier. Improvements across 4B/8B models across range of tasks and closure of repetition loops compared to alternate greedy decoding configurations demonstrate evidence that conflict steering works.
The paper covers good analysis of trace transitions, comparison against CLD and various baselines (such as presence penalty). Sensitivity to the model serving configurations are useful. Reporting negative results under sampling and unsuccessful transfer of always-on steering makes the experiments more informative.
Critical weaknesses
Method and resulting contribution builds on prior approach CLD heavily. CLD utilizes confident intermediate layer selection. Spherical steering is also a known approach. The main addition is therefore the disagreement-dependent interpolation. With Qwen3’s RMSNorm and linear output head, the proposed interpolation can also be expressed as a weighted mixture of intermediate and final logits for greedy token selection. The comparison against hard CLD supports mixing over hard replacement, but does not establish the importance of the particular spherical parameterization or JSD schedule.
The practical advantage over simpler repairs remains unclear, and the "steering fallback" cost appears understated. Appendix C reports that steering solves 15 of 35 greedy-open AIME problems, versus 16 for a second greedy configuration and 16 in expectation for one sampled retry. Thus, the strong fallback win/loss count does not establish a distinctive advantage. Moreover, the cost estimate neglects the length of the selected reruns.
Questions for authors
What evidence isolates the benefit of the JSD-controlled interpolation over a simpler approach using the same intermediate-layer selection? Please also clarify the endpoint normalization in Equation 1, given the norm-preservation claim.
Can the fallback cost be reconciled with the actual rerun token totals or measured runtime, and its practical advantage over the alternative retries already evaluated in Appendix C be clarified?