- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes PhCS, which separates checkpoint selection from checkpoint storage. During training, full checkpoints (weights and optimizer state) are saved at a fixed interval, and the input and metrics of every step are logged. After training, a selection rule picks a step, and that model is rebuilt by replaying training from the nearest saved checkpoint. Exact replay relies on deterministic kernels shared by vLLM and TorchTitan. For SFT, the rule combines training loss, token accuracy and entropy. For GRPO, it narrows an interval using AIME24 pass@4. Experiments use a ReTool SFT-then-GRPO math pipeline with Qwen3-4B-Instruct, Qwen3.5-4B and Ministral-3-8B. PhCS reports higher pass@4 than the best interval checkpoint and the final checkpoint on most benchmarks, and a storage saving of over 51x with less than 2x time overhead.
Critical strengths
- The problem is practical. RL training curves are noisy, and the best checkpoint is often somewhere in the middle of training. Separating what is stored from what can be selected is a clean way to frame this, and the authors are open that their GRPO search has no optimality guarantee.
- The system work is solid. The authors make SFT and agentic GRPO training bitwise reproducible for several recent models, including Qwen3.5 and Ministral-3, and release the code. This could be useful beyond checkpoint selection, for example for debugging RL runs.
Critical weaknesses
- The experiments do not fully answer the main question of the paper: is an unsaved step picked after training better than the best checkpoint saved at a fixed interval? Table 1 compares whole pipelines, and judging from Fig. 7 and Fig. 8, the Best Interval and Final Step rows also start GRPO from different SFT checkpoints, so each row is a different RL run. There is one run per setting, each AIME set has 30 problems, and pass@4 uses only four samples. On the held-out AIME sets the differences are one to three problems, with no error bars, and one goes the other way (AIME26 for Qwen3.5-4B: 0.467 for PhCS vs. 0.567 for Final Step). In the ablation, PhCS solves 21 of the 60 AIME25/26 problems, and final-step SFT with early-stopped GRPO solves 20. The case study suggests the answer may be no, at least for GRPO. In Fig. 9(b), the selected step 149 sits on the same 25-step grid as the saved checkpoints, and all four replayed steps around it score lower. The released code saves checkpoints at zero-based steps 24, 49, and so on, so step 149 seems to be a normally saved checkpoint. The selected Qwen3-4B step (174) looks the same. For these two models, the replay found nothing better than what interval saving already keeps, so their GRPO-stage gains cannot come from recovering unsaved steps. The SFT picks do fall between saved checkpoints, but they are never compared with the best saved SFT checkpoint under the same rule.
- The main results include the validation set. AIME24 is used to select the GRPO checkpoint (Section 4.1). Table 1 marks it as the validation set, but it is still reported next to the test sets, and the biggest gain in the paper is on it (0.733 vs. 0.500). The abstract also lists AIME24 among the improved benchmarks. Since PhCS evaluates more candidates on the same 30 problems, it should score higher there even if the selected model is not better. The same checkpoint also gets 0.60 on AIME24 in Fig. 9(b) but 0.733 in Table 1, and the paper does not explain the gap. Scores used for selection should not be presented as evidence of improvement. A validation set that is separate from all test sets is needed.
Questions for authors
- Within the same SFT and GRPO runs, how does the selected unsaved step compare with the best saved checkpoint under the same selection rule? Please report this on benchmarks not used for selection, over several seeds, with error bars.
- Are the selected GRPO steps 149 (Qwen3.5-4B) and 174 (Qwen3-4B) saved checkpoints? How is the Best Interval checkpoint chosen at each stage?
- How does PhCS do when the validation set is separate from all reported test sets? Why does the selected checkpoint score 0.60 on AIME24 in Fig. 9(b) but 0.733 in Table 1?