- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes SparseOPD, which computes full-vocabulary reverse-KL corrections before selecting sparse logit coordinates for differentiation. It combines correction-magnitude selection, signed residual compensation, and batch-global budget allocation. Experiments across mathematical, chemical, and multimodal reasoning show improvements over sampled-token and standard TopK distillation, with reduced backward memory and high gradient similarity to full-vocabulary supervision.
Critical strengths
- Principled sparse correction.
The method uses full-vocabulary teacher–student discrepancies to select important corrections and preserves positive and negative correction mass. The derivations are coherent, and the released code includes useful numerical tests.
- Consistent improvements over the evaluated sparse baselines.
SparseOPD outperforms standard TopK across six task–scale averages. Component ablations and gradient-fidelity measurements further support the proposed mechanism.
Critical weaknesses
- Practical efficiency gains are less substantial than suggested.
The reported 70.5% memory reduction at 8K concerns extra backward allocation; total peak memory decreases by only 14.4%, while complete-update time increases by 14.5%. SparseOPD is also slower than TopK. Moreover, efficiency profiling uses full-parameter training rather than the LoRA setting used for accuracy evaluation, with only one measured update per configuration.
- Limited statistical validation of performance gains.
Improvements over TopK are relatively modest (0.58–2.2 points). Supplementary results show variability, including a five-point regression against Full Vocabulary on AIME26. Multi-seed experiments cover only SparseOPD on 1.7B mathematics, without paired baseline runs, leaving the reliability of smaller gains uncertain.
- The accuracy–memory trade-off needs stronger controls.
Comparisons with memory-optimized exact reverse-KL implementations (e.g., chunked or recomputed backward) are missing. The supplementary code also includes an alternative sparse projection with lower coefficient error in a unit test, but no downstream evaluation. These comparisons would better establish the benefits of the proposed approximation.
Questions for authors
-
Can you compare against a fused or custom-backward exact reverse-KL implementation to better isolate the benefit of sparse differentiation?
-
The supplementary code includes an orthogonal_sparseopd variant that minimizes coefficient L2 error under a weaker constraint. How does preserving separate positive and negative correction masses affect downstream accuracy?
-
Can you report repeated complete-update timing and total peak memory, including planning, under the LoRA setting used for accuracy evaluation?
-
Can you provide paired multi-seed comparisons against TopK and Full Vocabulary to establish the reliability of the reported improvements?