- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work focuses on the influence of token pairs of irrelevant reasoning on the rollout process of reasoning. The paper uses four different semifactual perturbations and calculates a probability drift by scoring with a teacher-forced method after placing the same response token and its prefix under different prompts. The negative part of this drift is stopped from gradient training. The results are verified in two types of tasks: Mathematical reasoning and Out-of-distribution generalization.
Critical strengths
The method pipeline is complete. Starting from token-level semifactual sensitivity, it provides a complete process from diagnosis, fixed response detection to training correction. Moreover, the detection can reuse the already generated responses, and does not need to generate new trajectories for perturbations. In the experimental results, by randomly altering the correction positions, using perturbations that change the answers, and comparing negative/positive/bidirectional corrections, ablation studies were conducted directly to examine the signal position, intervention semantics, and correction direction. This further supports the claims of the paper.
Critical weaknesses
- There is no basis for equating task invariance and token-by-token probability invariance. Equivalent questions may reasonably induce different phrasings or solutions; fixing the prefix of the original answer and then observing the high drift, it may reflect harmful dependencies or normal language adaptation or context compatibility changes. The paper does not report how to distinguish between the two.
- The connection between unscaled drift to a unified negative correction is still heuristic. Probability increases and decreases are both included in the same drift, but both methods reduce the corresponding advantages; it is not an optimization of consistency loss. The bounded auxiliary gradient and finite-time optimization bound in the appendix only constrain the scope of the correction's influence, and do not prove that this correction reduces drift or improves returns.
- Training only covers Qwen backbones, and it cannot be confirmed whether the probability drift described in the paper is also widespread in other model structures. In addition, according to Appendix B.1, the perturbations generated using prompts do not have semantic supervision. The manuscript does not report the generation quality of the perturbations, etc., and cannot simply rely on the instructions to determine that the generation meets the requirements.
Questions for authors
See weeknesses.
3 comments
Response to Reviewer 9k67 (2/2)
[W3, data quality] What is the quality of the generated perturbations?
Thank you for highlighting this. To assess the generation quality, we used Qwen3.8-27B as a judge model to evaluate all 68,968 perturbations (17,242 problems 4 types) and then invited three human experts to review a subset.
(1) LLM evaluation. The judge, which is from a different family than the generator, checks task preservation, answer preservation, and whether hints were added or removed.
Judge outcome Paraphrase Typo noise Scenario wrap Irrelevant context All Valid 99.27% 98.82% 99.68% 99.54% 99.33% (68,503) Invalid or Uncertain 0.73% 1.18% 0.32% 0.46% 0.67% (465) (2) Human evaluation. Three human experts reviewed all 465 flagged prompts and 465 prompts randomly sampled from those the judge marked valid.
Judge decision Human: valid Human: invalid Human agrees with the judge Valid (random sample of 465) 464 1 99.78% Invalid or Uncertain (all 465) 190 275 59.14% Weighting the human validity rates by the full judge-stratum sizes gives an estimated overall validity of 99.39%.
Conclusion: The generated perturbations largely preserve the task and the answer (>99%).
We have added a summary to Section 2 and the audit protocol and detailed results to Appendix B.1.1 of the revised manuscript (text in blue).
We sincerely thank the reviewer again for the constructive feedback. We hope the additional experiments and clarifications address your concerns.
Dear Reviewer 9k67,
As the discussion period draws to a close, we wanted to ask whether our responses have addressed your concerns. If anything remains unclear, we would be happy to clarify.
Your feedback is really important to us. Thank you very much for your time and consideration.
Best regards,
The Authors
Response to Reviewer 9k67 (1/2)
We sincerely thank you for recognizing the complete pipeline and the targeted ablations. Your comments helped us identify key areas to clarify and improve, and we have carefully addressed your concerns in detail below.
[W1] How is harmful dependence distinguished from normal language adaptation?
[W2] Why are probability increases and decreases treated in the same way?
[W3, model scope] Does the drift pattern occur beyond Qwen?