- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The method optimizes the sparse-attention policy using a standard on-policy RL loss by replaying the rollout’s retention mask in the learner. An additional KL loss aligns the full-attention policy with the sparse-attention policy, which shares the same parameters and serves as a detached teacher. The ultimate goal is deployment with full attention.
Critical strengths
-
Well-motivated objective. Mask replay enables on-policy optimization of the sparse policy, while the KL term addresses the gap to full-attention deployment.
-
Transparent reporting. The paper discloses failed runs and reports both conditional and all-run statistics, helping readers assess the method’s benefits and limitations.
Critical weaknesses
-
Sparse-RL baseline fidelity. In the supplementary code (
code/masklift/losses.py, lines 149–158 and 212), the loss is normalized by the number of accepted trajectories rather than the original group size specified in Sparse-RL. In my independent three-seed control, correcting only this denominator improved GSM8K accuracy by 6.1–7.6 percentage points. This weakens the reported comparison with Sparse-RL. -
Insufficient evidence for the performance–cost tradeoff. The paper does not report held-out performance against wall-clock time, including learner and transfer costs. Figure 2 omits intermediate-checkpoint performance and emphasizes non-collapsed runs. On GSM8K, including all runs reduces MaskLift’s mean accuracy from 72.8% to 61.6%, compared with 72.3% for Dense GRPO. The results therefore do not establish a favorable overall performance–cost tradeoff or show whether the 200-step conclusions persist with longer training.
Questions for authors
Please address the concerns raised in the weaknesses above, particularly the Sparse-RL implementation and the end-to-end performance–cost comparison.
Independent reproduction under two prompt settings
Thank you for the updated code.
TL;DR: In my independent experiments, MaskLift performs worse on average than corrected Sparse-RL under a more reasonable prompt setting for GSM8K; hence, my score remains unchanged.
The two prompt settings are shown schematically below; the worked examples are abbreviated.
The missing answer cue can elicit continuation with another example or problem instead of an answer to the target question. The observed failure mode is illustrated schematically below, rather than quoted from a specific output:
Under the same evaluation decoding settings and 256-token output budget, the untrained Qwen3-1.7B scores 41.24% with the authors' prompt and 67.32% with my prompt. Adding only
Solution:to the authors' prompt raises the untrained score to 60.50% and eliminates the 168 empty scoring prefixes caused by continuation with a new example or problem. This indicates that prompt-induced output behavior contributes substantially to the low starting accuracy.Below are my independent three-seed results. Both settings use Qwen3-1.7B, 200 outer steps, group size 8, 30% prompt KV retention, and dense greedy evaluation on all 1,319 GSM8K test questions with a 256-token output limit. All seeds are included. "Corrected" uses the original group-size denominator; "uncorrected" uses the accepted-trajectory denominator.
My prompt setting
The authors' prompt setting
With three seeds, the means and sample standard deviations are descriptive.
Corrected Sparse-RL outperforms uncorrected Sparse-RL in my prompt setting by 6.1–7.6 percentage points across the three seeds. Interestingly, it performs worse in the authors' prompt setting, with a mean decrease of 1.74 percentage points. MaskLift is 1.52 percentage points below corrected Sparse-RL on average in my setting, losing on two seeds and winning on one, while exceeding it by 9.58 percentage points on average in the authors' setting.
The authors' reported relative advantage is successfully reproduced under their prompt setting, and their method shows a real advantage there, although the exact scores and gap are not fully reproduced. However, their method does not show an average advantage over corrected Sparse-RL under a more reasonable prompt setting that does not encourage the model to continue with another question instead of answering the target question. The scope and robustness of the advantage therefore remain insufficiently established. Furthermore, in the authors' matched 200-loop, five-seed comparison on one H200, Dense GRPO achieves 74.24% test accuracy in 0.844 hours per run, whereas MaskLift achieves 68.43% in 1.188 hours. Thus, MaskLift achieves lower accuracy with greater end-to-end training cost in this setting. Together, these concerns support maintaining my original score. A future direction is to make the objective more robust. Finally, I really appreciate the authors' willingness and courage to open-source their code.