- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the problem of grading the final boxed answer of a reasoning model, more specifically, grading the answer of the student model when training it using the generations of a teacher model.
They found the strict scoring reads the last boxed answer, which gives credits to boxed answers in generations that are never finished, penalizes answers committed in finished generations but unboxed.
They show that two cheap counts, unfinished responses containing a box and finished responses without a box, bound how wrong a strict score can be. They recommend people to report termination, delivery and correctness separately.
Critical strengths
The problem they study is very practical. Even if they are only studying reasoning models generating answers in boxes, the method can be applied to other output format, such as answers in a fixed JSON format.
I find the two judge-free counts ("a" being the share of responses that are unfinished but contain a box, "b" being the share of responses that finished but contain no box) bounding the error useful, anyone can audit old evaluation logs without running a judge.
Critical weaknesses
First of all, this paper is very hard to read, the language and the writing are hard to follow even for a reader with the right background.
As I mentioned in the strengths, this method can be applied to answers required to be in other fixed format, but I don’t think the authors touch on that.
Questions for authors
- Do you know the two teacher model’s performance on the benchmarks you trained the student models on?
- Why do you think for most of the student models being trained, the training hurt their performance on the benchmark?
- why do you choose step 25 and 200?
2 comments
Brief Summary of Readability Overhaul and New Format Suites
Dear Reviewer WRQP,
As the discussion closes on November 18 (AoE), we wanted to briefly highlight how substantially we revised the paper following your feedback: we rewrote the Abstract, added a plain-language summary box (p. 2) and §3 glossary, and replaced internal labels throughout the main text. We also added the fixed-format suites you suggested across JSON, XML and unit-tested LiveCodeBench v6 code with no LLM judge (§7, Fig. 7b), evaluated both teachers on all four benchmarks (§3), and documented why token-budget overruns and unfiltered targets hurt most runs alongside two boxed-SFT cases that improve, plus a 5-step rerun before our routine 25-step save schedule (§6, Appendix K). We hope the revised manuscript is much easier to read and would be glad to clarify any remaining question before the window closes.
Thank you for the encouraging review. Each of your questions led to a change in the paper.
Responses to Weaknesses
We agree, and rewrote the parts a first-time reader meets. The Abstract and a new summary box on p. 2 state the main result in plain terms, and the main text replaces internal labels with direct names (
delivered accuracy,prompt mode,grading gap). §3 now has a glossary of the three termination states and two prompt modes. New results sit in two small tables and Fig. 7 (pp. 7, 9–10), with substantive additions in blue; we are glad to clarify any remaining passage.Added, following your suggestion: JSON, XML and unit-tested code (§7, Fig. 7b, Appendix T.2–T.3). The two-count bound holds in all 40 JSON/XML settings and all five code settings. The error changes sign under both JSON and XML, though not on every trajectory. Examples from gpt-oss-trained students:
Responses to Questions
Yes; both teachers are now evaluated on all four benchmarks (§3, Appendix T.4). Delivered accuracy (%) at the students' 32,768-token budget:
Teachers are scored on all 30 / 198 AIME 2025 / GPQA Diamond items; Qwen3-30B on 29 / 197, because the first item of each was used for piloting. Gemini was sampled through its API at temperature 0.7; the other two at temperature 1. Both teachers box over 99% of finished benchmark answers here; the Gemini SFT set (0.08% boxed) used a prompt without a box instruction.
Three candidate factors, measured but not isolated (§3, §6):
Training does help in two boxed-target cases (§6, Appendix T.5):
<think></think>, two training seeds; AIME 2024, step 200Step 25 was not hand-picked: checkpoints were saved on a routine 25-step schedule, making step 25 the first saved checkpoint (§3, Appendix A.2). Steps 25–200 formed the pre-registered grid for the initial two-teacher comparison (Appendix O), with step 200 after the early length surge and still inside epoch 1 (no example seen twice). Both sides have been checked:
On a denser grid for Qwen3-4B (training seed 42, after empty
<think></think>), the GPQA-main error shrinks step by step and crosses zero between steps 100 and 150 (Appendix T.5).