- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies whether repeatedly fine-tuning on identical reasoning traces makes reasoning vulnerable to subsequent training. All models perform similarly at handoff, but only the drilled model loses substantial reasoning accuracy after later instruction or answer-only training.
Critical strengths
-
The paper shows that two models with the same immediate reasoning accuracy can encode that capability with very different robustness to later training, which is useful for refinement of how data efficiency should be assessed.
-
The paper does not only identify the problem: it shows that varying solution traces, replaying a small amount of reasoning data, or briefly restoring the reasoning format can prevent or repair the failure.
Critical weaknesses
-
The base model already scores 94.7%, and reasoning training provides almost no handoff improvement. On the synthetic task where the base lacks the procedure, repetition mainly harms acquisition rather than causing the same post-training collapse.
-
The evaluation is only performed on particular reasoning format, only qwen's <think> block and uses a selected subset of MATH-500, boarder reasoning tasks and models need to be included to strengthen the claims.
Questions for authors
-
Does replay remain effective under the harsher later stages?
-
Can the main design be replicated in a setting where reasoning training produces a substantial capability gain at handoff?
Reply to Reviewer 3Fyx
Thank you for the review. In short: replay remains effective under the harsher stages, and the paper now states its scope precisely: reasoning the model already has, the setting s1 and LIMO target.
Q1. Does replay still work under the harsher stages? Yes (Appendix F, registered). We replayed the drilled model's own training texts in 6.25% of each batch.
The once-trained model lost nothing in either stress test, so we did not rerun it with replay.
Q2. Can the design be repeated where reasoning training gives a real gain at handoff? We tried the most natural candidate, and it does not provide such a setting.
We trained an instruct model (Qwen2.5-7B-Instruct, the family s1 uses) on gpt-oss-120b's traces. It lost forced accuracy instead of gaining it (69.6% to 61–65%). One epoch then took even the untrained student from 69.6% to 25.3%. Because the reasoning block is not native to that model's format, we also measured free accuracy. It fell as far (69.5% to 27.3%). That stage harms this chat model whatever its training, so the setting cannot test repetition in either direction.
The synthetic skill is our setting with a large gain, and there repetition mainly cost learning, as you say. The paper therefore states that the effect concerns reasoning the model could already do. What reasoning training changes in our setting is the default: the base model opens its reasoning block in 39% of free answers, the drilled and once-trained models in all of them.
Breadth: one reasoning format and a subset of MATH-500. Nemotron 3 Nano is another model family with its own chat template (it also reasons in a think block), and it, the arithmetic task and AIME were in the submission. The break appears in all of them: two model families, two scales, a stronger model's traces, three independent runs and a second task. The limitations state the remaining breadth: mathematics with integer answers and one arithmetic task.
Scope: released weights. One new result sharpens the claim and gives a remedy: from merged weights with a new adapter, one epoch no longer harms the drilled model, while the instruction stress test still costs it 9.4 points more than the once-trained model (general response, part 1). The title and abstract now state this condition, and the abstract says that every replication continues the adapter.
The writing. The paper is restructured: a Method section with design tables, results ordered by question, one name per stage, and 9 appendices by question instead of 15.