- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SparseOPD recovers sparse language-model execution by distilling from a frozen dense execution of the same checkpoint. The sparse student generates its own training trajectories as it changes, and compact teacher/student token supports with a residual tail define the distributional targets. LowP further reduces supervision to positions with low unwarped sampled-token probabilities recorded by the rollout student. The paper first diagnoses prefill and decode degradation, then tests recovery across three backbones, including Ministral. For example, OPD-RKL reaches 63.33 on AIME against 63.75 for dense Qwen3-4B. The study also evaluates long-context behavior, compares with GRPO and OPRD, and reports recovery cost and amortization, giving a useful picture of both the learning effect and its computational cost.
Critical strengths
S1. The prefill/decode diagnosis is useful because it separates two sources of sparse degradation. Same-checkpoint comparisons and fixed dense-state/sparse-state replay show where recovery helps. A forward-KL comparison with the target held fixed further supports supervising on sparse-student states.
S2. On Qwen3-4B, OPD-RKL reaches 63.33 on AIME, close to dense execution's 63.75. Recovery is also tested on two other backbones, including Ministral, and the paper reports full recovery cost and amortization, which helps assess whether the recovered quality is useful in practice.
Critical weaknesses
W1. I cannot tell how much continual rollout refresh adds to the reported recovery. The same-target FKL result supports supervision on sparse-student states, and the appendix documents the collection schedule. What is missing is a trained control whose trajectories all come from the initial sparse student. That control would still receive sparse-state supervision, so it would distinguish the value of continually updating the trajectories from the value of using sparse trajectories at all.
W2. LowP-40 reduces recovery time from 22.70 to 19.04 hours, with accuracy differences of −0.41, +0.38 and −0.57 points. Supervising fewer positions already reduces the amount of work. The logit-gradient proxies explain why low-probability positions might be useful. There is no trained random-40% control to tell whether this ranking preserves quality better than an equally small unranked subset.
Questions for authors
Q1. Does refresh improve recovery over trajectories collected entirely from the initial sparse student? Please use one backbone with the same prompt schedule, distributional targets and optimizer-update budget. For both runs, report total recovery GPU-hours, including teacher scoring and rollout collection, to show whether any gain is worth the added cost.
Q2. How does LowP-40 compare with a trained random-40% baseline at the same objective and supervised fraction? Quality, recovery time and available variation across training runs on one backbone would be enough to judge the value of the ranking.
3 comments
Thank you for adding both requested controls. The fixed-initial-policy comparison addresses my concern about continual refresh.
LowP-40 improves the reported math scores over Random-40 and matches its code accuracy. This supports the ranking, although one Random-40 training seed leaves the reliability of these gains uncertain.
The added evidence strengthens my assessment. I keep my rating at 3.
Thank you for your follow-up and for recognizing the value of the added controls. We will include both the fixed-initial-policy comparison and the LowP-40 versus Random-40 comparison in the forthcoming revised manuscript. We appreciate your constructive feedback, which has helped strengthen our experimental evaluation.
Thank you for identifying the controls that isolate continual refresh and LowP ranking. We completed both on Qwen3-4B with the same prompt schedule, FKL targets, sparse backend, and 200-step update budget.
Response: Continual refresh improves recovery beyond supervision on initial sparse-policy states. In the new fixed-policy control, all trajectories are generated by the initial sparse student; the learner still receives dense-teacher FKL supervision on those trajectories. The refreshed run updates its collection policy every 20 steps.
AIME/HMMT report Mean@8 and LCB reports Pass@1. Time is total recovery wall-clock on eight RTX 6000D GPUs, including rollout collection, teacher scoring, and training.
Refresh improves AIME/HMMT by 0.83/0.38 pp and LCB by 5.14 pp, at 5.12 additional recovery hours. Refreshing supplies supervision on states induced by the updated policy and delivers a clear gain on code generation.
The refresh-frequency ablation provides further evidence (for details, please see our response to Reviewer AMET, Q2). With the same 200 updates, refreshing sparse-student trajectories every H=5/20/50 steps yields AIME scores of 64.58/62.08/62.50 and HMMT scores of 37.12/35.98/35.61. The most frequent refresh achieves the highest score on both tasks. Together, these results support maintaining on-policy state coverage as the sparse student evolves.
Response: LowP improves performance recovery at the same supervised fraction. Random-40 uniformly samples 40% of positions within each trajectory; LowP-40 selects the 40% with the lowest unwarped sampled-token probabilities. Both retain the OPD-FKL objective and refresh schedule. The added Random-40 control uses one training seed. Metrics and timing follow the table above.
LowP gains 1.25 pp on AIME and 1.89 pp on HMMT while matching LCB accuracy. At an equal supervision fraction, this comparison directly measures the value of ranking. The result complements the paper's gradient analysis: the lowest-probability 40% of positions capture 99.3% of the FKL logit-gradient energy. LowP therefore concentrates supervision on informative positions while retaining the computational benefit of selective supervision.
We appreciate your review once again and hope our responses adequately address your concerns.
If any further questions remain, please do not hesitate to raise them.