- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how SFT gains depend on the output budget. It crosses initial and adapted checkpoints as prefix generators and continuation models, then exchanges stopping probabilities and conditional content distributions between standard and EOS-ablated SFT. Experiments find a budget reversal for Llama on synthetic algebra and attribute most of the EOS-ablation improvement to stopping in the tested algebra and MBPP settings.
Critical strengths
The matched EOS ablations and four-cell exchanges provide more informative controls than endpoint accuracy alone. The main algebra and EOS comparisons include three training seeds, and the question-level bootstrap analyses keep repeated samples grouped within each question. The paper also explicitly distinguishes recovery through EOS ablation from the full SFT deficit.
Critical weaknesses
W1. The empirical contribution remains limited to a narrow case study of a damaging adaptation regime. A consistent positive-to-negative budget reversal is demonstrated on only 50 synthetic algebra questions. The additional models do not consistently reproduce this pattern, and the original GSM8K subset does not exhibit the predicted budget trajectories (Table 2; Appendices C.1 and E). All main adaptations use the same 500-example corpus and learning rate. On MATH-500, mean standard-SFT accuracy drops from 43.10% to 15.53% for Llama and from 63.20% to 43.27% for Qwen, while EOS ablation recovers only 12.94% and 6.19% of these deficits, respectively (Appendix A.2). These results leave unresolved whether the approach provides a broadly useful diagnostic or primarily characterizes a specific training failure. Establishing the broader significance of the findings requires validation-selected SFT controls and evidence beyond the synthetic reversal. W2. The stage attribution is insufficiently validated against native stopping behavior. The protocols in Appendices C and G allow prefixes that have already terminated to receive nonempty continuations, which can replace the extracted answer. This issue is consequential: in the 50-question GSM8K diagnostic, 584 of 800 initial Llama prefixes and 699 of 800 adapted Llama prefixes have already terminated by the handoff. The reported source and continuation effects may therefore partly reflect the restarting of completed answers. The paper acknowledges this limitation but neither quantifies its contribution to the stage ordering nor demonstrates that the ordering persists when EOS is absorbing. The greedy MATH-500 NLL diagnostic in Appendix A.4 excludes early-ending pairs and therefore does not resolve this concern. Supporting the diagnostic interpretation requires a direct control with absorbing EOS.
Questions for authors
- Do the budget reversal and stage ordering persist for a validation-selected SFT checkpoint or under a modest learning-rate variation, rather than only the reported fixed recipe?
- What are the four-cell contrasts when terminated prefixes remain absorbing, and how much of the current GSM8K stage ordering comes from reopened responses?
Adaptation Settings and EOS Handling
Thank you for the comments. We tested a lower learning rate, validation-selected checkpoints, and a handoff that leaves completed responses unchanged.
W1 and Question 1 — Adaptation setting and scope
Both settings reduce the long-budget loss, while all three seeds still improve at the short budget and lose accuracy at the long budget. On the original 50 algebra questions, with 16 samples per question, the lower learning rate gives mean gains of +10.625 pp at 192 tokens and −2.375 pp at 768 tokens. Validation selection gives +11.250 and −2.000 pp at the same budgets.
We lowered the learning rate from 1e−4 to 5e−5 and kept 100 training steps, the corpus, LoRA, and other optimization and batch settings fixed. For validation selection, candidates come from steps 0, 25, 50, 75, and 100 of the original 1e−4 runs. Selection by 192-token accuracy on 200 other GSM8K training questions chooses steps 50, 75, and 50. These questions exclude the adaptation corpus and official test split. The selected checkpoints are evaluated on algebra, so both task and budget change.
We also repeat the stage exchange for these checkpoints, generating prefixes for up to 192 tokens and allowing up to 576 more tokens for continuation. Source effects change the prefix-generating model while fixing the continuation model. Continuation effects change the continuation model while fixing the prefixes. Effects calculated from seed-averaged cells are below, in pp.
The margin measures how much the smaller source-effect magnitude exceeds the larger continuation-effect magnitude. It is positive in every seed. The lower-learning-rate interval is above zero, while the validation-selected interval includes zero. Bootstrap resampling is by question and keeps all samples, seeds, and cells together, including the shared initial reference.
Supplementary Tables 1–2 and 4 give the counts and individual contrasts. All effects use the exchange’s own diagonals. Their mean differences are −3.208 and −2.708 pp, compared with −2.375 and −2.000 pp under native generation.
On full-test GSM8K, Llama’s larger loss follows the prefix source (−7.127/−6.710 pp), while Qwen gains mainly through continuation (+4.321 to +5.099 pp). Other settings do not all show the budget reversal. On MATH-500, standard SFT reduces accuracy from 43.10% to 15.53% for Llama and from 63.20% to 43.27% for Qwen. EOS ablation closes 12.94% and 6.19% of those gaps.
W2 and Question 2 — Absorbing EOS and reopened responses
The larger source effect remains when completed responses are left unchanged. We tested initial Llama and the original d100 seed-1 checkpoint on 200 fixed GSM8K test questions, with four samples per question and a 768-token total allowance. The two evaluations reuse the same stored prefixes and all continuations of active prefixes.
Source effects fix the initial/adapted continuation models, respectively. Continuation effects fix the initial/adapted sources. All three margin intervals are above zero.
At 192 tokens, the source gaps fall from 56/49 to 52/46 successful responses under absorbing EOS, reductions of 7.1%/6.1%. The margin decreases by 0.250 pp, or 4.5%. Supplementary Tables 3 and 5 report the four-cell counts and reopening effects.
These percentages apply to the matched 200-question collection. The 584/800 and 699/800 ended-response counts in your review concern the original 50-question subset. Full-test GSM8K retains its separate population of 5,276 responses per cell.