- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GRASP, a rubric-aware on-policy self-distillation framework for personalized question answering. The student receives a question and retrieved user history. A frozen teacher additionally receives user-specific rubric aspects and provides token-level supervision along student-generated responses. The method also introduces Rubric-based Teacher Validation (RTV), which filters training instances based on the performance of teacher-generated answers. Experiments on LaMP-QA across three domains and four backbones show higher average rubric-coverage scores than the selected baselines. Ablations examine RTV thresholds, teacher size, and rubric alignment. The appendix provides further details on the benchmark, training procedures, and related aspects.
Critical strengths
-
GRASP offers a straightforward way to internalize rubric-based supervision into a model without requiring rubrics at inference time. It achieves state-of-the-art average rubric-coverage scores on LaMP-QA across four backbones. The paper also includes substantial ablations of its main components, strengthening the empirical support for the framework.
-
The shuffled-rubric experiment and the rubric-completeness analysis in Appendix D are particularly insightful. They test whether the gains depend on correctly aligned and complete rubric information, rather than simply additional teacher context. Together, they provide strong evidence that GRASP learns from the semantic content of the rubrics.
Critical weaknesses
From a methodological perspective, it remains unclear whether the gains primarily come from on-policy trajectories, soft-label distillation, or RTV-based instance selection. Additional controlled experiments, such as matching the number of training instances across settings, would help clarify the contribution of each component.
Questions for authors
- RTV identifies instances where the teacher generates a good answer, but this does not necessarily mean that the teacher can provide reliable guidance along student-generated trajectories. Although the paper includes detailed RTV ablations, reporting the number of training instances retained at each threshold, together with comparisons of different thresholds that control training data size, would clarify RTV’s contribution.
- I hope to know more about the relationship between the benchmark’s training and test sets, especially the user overlap. This is important for assessing whether GRASP learns a transferable ability to infer users’ needs from their history and apply this ability to new topics.
- RTV retains instances with high-scoring teacher responses. These responses could also provide useful targets for SFT or off-policy distillation. Comparing these alternatives would help isolate the contributions of on-policy trajectories and soft-label supervision.
- Under capacity or information constraints, forward KL may favor overly averaged predictions. An ablation of forward KL, reverse KL, and potentially JS divergence would help justify the chosen objective.
- Does GRASP affect the model’s ability to answer general questions? Evaluating general capabilities before and after training would clarify whether personalization gains come at the cost of the model’s general performance.
- The method uses the top-10 retrieved history entries as context, but the paper does not establish whether the target rubric aspects can be inferred from this context. This distinction matters for interpreting the contribution: does teacher supervision teach the student to use user history, or mainly encourage more comprehensive answers that achieve higher rubric coverage?
- Similar rubric-conditioned self-distillation methods have been explored in other domains, including RuPI [1], RGSD [2], and RCSD [3]. In personalization, Aligning Language Models from User Interactions [4] also uses additional user context to construct token-level self-distillation signals. Please clarify which components GRASP shares with these methods and which adaptations or innovations specifically address personalized question answering.
If it is difficult to address all the questions within the limited rebuttal period, please prioritize Questions 1, 2, 3, and 5, as these are my main concerns.
[1] Rubrics as Privileged Information for Open-Ended Generation.
[2] Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers.
[3] Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation.
[4] Aligning Language Models from User Interactions.
5 comments
Thank you for the helpful rebuttal. They adress most of my questions. But I still have two brief questions about the reported numbers:
- For Qwen2.5-7B, the seen and unseen scores are 0.5306 and 0.5308 in R2, while Table 1 reports an overall score of 0.5357. If you could explain the difference, it will be better.
- The authors refer that IAP is the strongest baseline with a score of 0.5078 in R4, whereas Table 1 reports 0.3975. Could you clarify whether this is an updated result or a reporting error?
Clarification of Unclear Reporting
Dear Reviewer,
Thank you for your kind reply and follow-up questions! We apologize for the unclear reporting, and we provide our clarifications below:
- Seen/unseen scores: We apologize for inadvertently evaluating wrong data in R2, which caused the discrepancy with the main results in Table 1. We have re-evaluated the correct model across three random seeds. The corrected results are provided below. All scores are macro-averaged across the three categories (as in Table 1). In the revised manuscript, we have updated Table 1 with the three-seed results and added the seen/unseen results to Table 17.
| Backbone | Seen | Unseen |
|---|---|---|
| Gemma2-9B | 0.5218 | 0.5277 |
| Qwen2.5-7B | 0.5368 | 0.5332 |
| Qwen2.5-14B | 0.5581 | 0.5649 |
| Qwen3-4B | 0.6176 | 0.6147 |
- IAP results: The 0.5078 reported in R4 is an updated result. Specifically, we re-implemented IAP on the Gemma2-9B backbone using the swift framework, which achieved stronger performance than our earlier adaptation of the original repository. Details and updated results are provided in the revision.
We have updated our revised manuscript to reflect and clarify these results. Thank you again for pointing out these issues and helping us improve the quality of our paper. We welcome further discussion and appreciate your continued feedback.
Dear Reviewer,
Thank you for the detailed and constructive feedback! We value the opportunity to address the weaknesses and questions you kindly raised and to further enhance the quality of our work.
Thank you for this suggestion. The retained sample counts in our threshold ablation are:
We additionally matched the training dataset used in Gemma2-9B experiment with random selection:
This supports RTV's benefit beyond reducing data size.
Thank you for raising this point. Our analysis identifies 1,383 test instances from seen users and 1,447 from unseen users. Performance on each partition is reported below:
These comparable results suggest that GRASP learns a transferable ability to generate personalized answers rather than relying solely on familiarity with training users.
Thank you for this helpful suggestion. We compared two alternatives on the same RTV-selected instances: SFT learns from the retained teacher answers, while off-policy KD learns from teacher distributions along those answers.
Both alternatives perform well on RTV-selected data, but remain substantially behind GRASP, supporting the benefit of on-policy distillation.
Thank you for this suggestion. We evaluated alternative objectives on Gemma2-9B:
All three objectives clearly outperform the strongest baseline (IAP, 0.5078), so GRASP is robust to the choice of divergence. Forward KL and JSD perform comparably, while reverse KL is slightly lower. We attribute this to mode-seeking: a rubric-informed teacher spreads probability over multiple valid ways to address the target aspects. Forward KL encourages the student to cover them, rather than producing overly averaged predictions, whereas reverse KL may collapse onto a subset. We therefore keep forward KL as the default.
Thank you for raising this concern. We evaluated all four backbones on MMLU-Pro, a benchmark of broad knowledge and reasoning, before and after GRASP training:
These results show no substantial degradation on MMLU-Pro, suggesting that GRASP strengthens personalization while largely preserving general capabilities.
Thank you for this insightful question. The top-10 history entries provide personalization context rather than directly specifying rubric aspects. Replacing them with other users' profiles across five random mappings reduces Gemma2-9B's score from 0.5255 to 0.4789 ± 0.0016. This suggests that GRASP benefits from user-specific history, beyond simply generating more comprehensive answers.
Thank you for highlighting these relevant works. GRASP shares rubric-conditioned token-level self-distillation with RuPI, RGSD, and RCSD. Our contribution is its adaptation to history-conditioned personalized QA through user-specific rubric supervision and RTV-based teacher validation. Aligning Language Models from User Interactions derives supervision from subsequent user messages, limiting learning from interactions without follow-ups. GRASP instead learns from user-specific rubrics during training and personalizes from existing history at inference, supporting a broader range of personalization scenarios. We include more detailed discussion and comparison in the revision.