- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces RGSD, a method that uses per-prompt rubrics as privileged context for a frozen teacher and performs self-distillation with a clipped Jensen–Shannon divergence. The authors position the method as an alternative that achieves comparable performance to rubric-based RL while removing the expensive verifier or LLM judge from the training loop and generating only one sample per prompt. Across Qwen2.5 models, RGSD achieves comparable rubric satisfaction with substantially shorter responses than GRPO. Experiments on Qwen3-Thinking models also show competitive performance, although the response-length advantage is not consistent. The evaluation covers both in-domain and out-of-distribution benchmarks in medical and science domains.
Critical strengths
-
The paper proposes a different way of using prompt-specific rubrics in model training as an alternative to the prevalent rubric-based RL approach. Using rubrics as privileged teacher context removes the need for an LLM verifier during training.
-
The paper is well presented and easy to follow.
Critical weaknesses
-
Limited methodological novelty. I view RGSD as a specific instance of existing self-distillation methods such as SDPO and OPSD, with the main distinction being the use of per-prompt rubrics as privileged context. The methodological contribution beyond this choice appears limited.
-
Narrow domain coverage. The experiments focus only on medical and science domains, which seems insufficient to establish RGSD as a broadly comparable alternative to rubric-based GRPO.
-
Limited comparative evaluation across model families. Although Appendix C.3 includes OLMo-2-1124-7B-Instruct and Mistral-7B-Instruct-v0.3, these are older checkpoints, and neither is compared with a matching GRPO baseline. Comparisons using more recent models from other families would provide stronger support for the broader applicability of RGSD.
-
Limited evaluation of overall response quality. Rubric satisfaction alone may not fully capture the quality of the resulting models. It would be helpful to also report hard or soft win rates against reference responses from newer models, the starting policies, and the corresponding GRPO-trained models.
-
Limited evidence of broader generalization. Although the paper includes OOD evaluation within medical and science, it would be more informative to train on prompts spanning a broader range of domains and evaluate on benchmarks covering different held-out subdomains.
Questions for authors
-
Could the authors clarify why a general-purpose LLM judge was used rather than a generative reward model for the GRPO baseline?
-
Is the 4096-token completion budget sufficient for the thinking models? Could the authors report how often responses are truncated?
-
How was the learning rate of 4.2 * 10^-6 selected?
-
Could the authors clarify why the entire thinking trace is masked rather than only tokens that explicitly refer to the rubrics?
-
Could the authors explain the choice to focus only on the medical and science domains of RubricHub?
-
Could the authors evaluate the trained models using additional performance metrics beyond rubric satisfaction?