- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
RS-SFT selects self-generated responses by final answer correctness alone, which admits flawed trajectories that make failed attempts or keep generating after answering, and iterative self-training amplifies these behaviors. IMIT fixes this by keeping a correct response only if it has exactly one final answer and stops immediately after it. On Qwen3-1.7B-Base and MATH-500, IMIT beats standard RS-SFT by 2.12 points in one round and 4.37 points iteratively, while preserving solution diversity close to the base model. Outcome correctness alone is therefore insufficient for selecting responses to imitate.
Critical strengths
-
The paper makes a compelling case that outcome-only selection amplifies undesirable behaviors across self-training rounds, and that last-answer evaluation can conceal this by rewarding within-sample retries. This is consequential because it re-examines a widely used pipeline and turns the observation into a concrete, actionable criterion.
-
IMIT operates purely at the data-selection stage and adds only a simple trajectory-level condition on top of the existing verifier, so it can be dropped into standard RS-SFT pipelines without extra models, process reward models, or additional compute.
Critical weaknesses
-
All evidence comes from a single small model on a single benchmark, and the central improvement is not robust under greedy decoding, where the authors report no significant difference. Combined with the acknowledged validity limits of the primary first-answer metric, the practical significance of the reported gains remains unclear.
-
IMIT only enforces a single-box, immediate-termination condition and explicitly does not verify reasoning. A crucial ablation is absent, namely whether simply truncating accepted responses at the first box before SFT would recover most of the gain. Without it, it remains unclear whether the selection framing adds anything beyond a simple preprocessing rule, which materially affects the assessment of novelty and significance.
Questions for authors
-
Could the authors report greedy-decoding results with confidence intervals for both settings, and clarify the practical interpretation of a gain that holds only under sampling?
-
What happens if, instead of rejecting multi-box or continued responses, the authors truncate each accepted RS-SFT response at its first \boxed{} before SFT? If this recovers most of IMIT's gain, the contribution reduces to a preprocessing trick. The authors should report this ablation.
-
The conservative analysis bounds rather than quantifies the intermediate-box bias. Could the authors provide a human- or model-verified estimate of the true rate on a sample, so the bias of the primary metric is measured rather than bounded?
-
Do the authors have preliminary results on a second model or benchmark showing the same iterative amplification trend?
-
IMIT discards a substantial share of otherwise-solvable problems simply because no sampled response satisfies its trajectory criterion. Would performance change if, under the same total generation budget, the authors kept those problems and continued sampling until a criterion-compliant response was found?
8 comments
W1. What does the first-box gain establish in practice?
We found no reliable greedy gain on the submitted checkpoints. Crediting intermediate first boxes when the last answer was correct narrowed the sampled accuracy gap. With a fixed token budget on retrained late-pool models, final-answer voting favored IMIT over RS. Stopping alone scored similarly to IMIT. IMIT's tendency to stop after the answer also appeared on SVAMP. On the two 8B models, however, IMIT showed no clear last-box accuracy advantage over simpler rules. The clearest change is how the model presents its answer. A general reasoning gain remains unproven.
Revised manuscript. Appendix E.2, p. 23, lines 1221~1227 (voting). Appendix E.5, p. 25, lines 1298~1312 (8B models and SVAMP). Appendix E.6, p. 25, lines 1316~1327 (first-box audit, scoring sensitivity and greedy decoding).
W2. How much of the gain can simple truncation explain?
IMIT checks answer format and stopping. It does not verify each reasoning step. Truncation alone modestly raised sampled first-box accuracy and lowered last-box accuracy in both pools. Verifying the answer at the cut recovered nearly all of IMIT's early-pool first-box gain. IMIT retained a larger first-box lead in the late pool, where a stopping-only rule did best on last-box accuracy in the matched-question comparison.
Revised manuscript. Appendix E.3, p. 24, lines 1244~1247 (selection rules), Table 10, lines 1249~1256 (truncation results), and lines 1258~1263 (matched-question controls).
Thank you for suggesting the first-box truncation experiment. This comparison strengthened the paper by showing how much of the early-pool gain a simple preprocessing step could recover. Verifying the truncated answer recovered nearly all of IMIT's early-pool first-box gain. The late-pool results depended on both the choice of training target and which answer was scored.