- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes Surgical Post-Training (SPOT) to improve LLM reasoning while reducing catastrophic forgetting. Its main contribution combines minimal-edit data rectification with a reward-based objective. (1) Minimal-edit data rectification: Converts model failures into erroneous–corrected response pairs close to the student’s distribution. (2) Reward-based objective: Uses reference-relative BCE/BCO to reinforce corrected traces, suppress errors, and adaptively attenuate updates.
Critical strengths
- Good empirical analysis and clear motivation: The analysis of the Elastic Tether, pull-up effect, and DPO’s learning dynamics clearly motivates each component. It provides useful insight into how training objectives affect capability retention during post-training.
- Simple and efficient approach: SPOT achieves substantial improvements with a simple design, limited training data, and short training time.
- The paper is well organized and easy to follow,
Critical weaknesses
- SPOT combines existing BCE/BCO objectives with teacher-guided correction. Much of the analysis also builds on findings from prior work, such as pull-up effects and DPO learning dynamics. While the empirical results on reasoning improvement and capability retention are valuable, I find the methodological contribution somewhat incremental.
- Experiments only cover models up to 8B and two training domains. OOD reasoning is evaluated only on GPQA-D and Connect4, while general capabilities are mainly assessed using IFEval. This makes it unclear how well the findings generalize to larger models and more diverse tasks.
Questions for authors
- Appendix F.1 reports a 56.94% parsing-failure rate for Qwen3-8B in a diagnostic evaluation. Could the authors report corresponding rates for the base and trained models on the main Connect4 benchmark? This would help assess whether improved output formatting contributes to the reported gains.
3 comments
Dear Reviewer,
With the discussion deadline approaching, we would greatly appreciate your feedback on our rebuttal. Please let us know if any concerns remain or if further clarification would be helpful.
Thank you for your time and consideration. We look forward to hearing from you.
Providing SPoT checkpoints/ Follow up
Dear Reviewer,
As the discussion deadline approaches, we would like to highlight the additional evidence and resources provided in our rebuttal. We have added MBPP+ results, complete evaluations on MMLU-Pro and TruthfulQA, and experiments with Phi-4 (14B), extending our evaluation to a larger model from a different model family.
We have also released the SPoT checkpoints, (Qwen3-8B and Llama-3.1-8B-Instruct) and the 500-position Connect4 evaluation set to support reproducibility, available on the anonymous download page. Alongside our evaluation suite, and code shown in the supplementary materials to facilitate reproducibility and further research. These resources enable the reviewers and community to directly evaluate our checkpoints on additional out-of-distribution tasks across other domains.
We hope these additional results and efforts can be taken into account in the final assessment. We would greatly appreciate any feedback on whether our responses address your concerns.
We thank the reviewer for the positive recognition on our analysis and motivation, as well as the constructive feedback. Below, we address each point in detail:
1. Contribution Beyond Combining Existing Objectives
As noted in Sec. 4.3, our contribution lies not in inventing the BCE/BCO loss forms themselves, but in uncovering how objective design and data proximity interact:
Finding the objective's independent role in knowledge retention (Sec. 3, App. D.2): While prior work (Chen et al., 2025; Shenfeld et al., 2025) attributes capability retention primarily to on-policy data, but not loss design like KL-regularized effect, we come to a different conclusion. We show that data proximity alone is insufficient and that the training objective plays an independent role. Comparing
SFT+andReward-SFTon the exact same positive-only rectified data (y+),SFT+still suffers monotonic IFEval decay (83.0 → 80.0) as does self-generatedRFT(81.5), whereas switching only the loss toReward-SFTpreserves IFEval (83.7).Text-only black-box OPD and superior RL initialization. Unlike logit-based OPD, SPoT requires only text edits. For subsequent RL exploration, SPoT injects new knowledge while retaining diverse valid paths.
2. Generalization to Larger Models and More Diverse Tasks
We conducted additional evaluations across a larger model family, coding, and broader general-capability benchmarks:
Phi4-14B): Usinggpt6-lunawith only 2k rectified pairs, SPoT improves Phi4-14B from 43.96 → 45.87 (+1.91 average across 9 benchmarks, e.g., +2.70 on AIME24, +4.00 on AMC23, and +2.52 on GPQA-Diamond) while preserving IFEval (61.74 → 61.28).MBPP+) & general capabilities (TruthfulQA,MMLU-Pro): We additionally eval on coding task. On Qwen3-8B, math-trained SPoT improves MBPP+ from 63.23% → 64.81% (+1.58%).We will include these results in the revision.
3. Connect4 Output Parsing-Failure Rates and Source of Gains
On Connect4, the parsing-failure rate drops from 56.94% (Base) to 35.2% (SPoT) (−21.74%). Decomposing the net gain, 79.8% comes from improved format adherence and 20.2% comes from correcting the answer content on already-parseable outputs.
This should not be viewed as inflated by format fitting: the Table 1 SPoT model was trained exclusively on math data with zero Connect4 format supervision. Maintaining output formatting during unfamiliar OOD reasoning is itself a core generalization gain—whereas
SFT/SFT+lose −3.4/−3.0 on IFEval. Together with the 20.2% gain on already-parseable answers, both OOD instruction adherence and reasoning contribute to the improvement. We will add this breakdown to Appendix F.1.We hope these additional experiments and clarifications address your concerns. We would be glad to answer any further questions.