- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PARL, a framework for the evaluation of personalized LLM outputs that learns preference-aware evaluation rubrics from user histories. Using an LLM as a rubric generator, PARL combines rubric induction and rubric consistency validation with a discriminative reinforcement learning objective to capture user-specific preferences and improve the discriminative power of the generated rubrics. Experiments show that PARL-generated rubrics better differentiate user-authored ground-truth responses from personalized baseline outputs, demonstrating that the learned rubric generator effectively captures user-specific preferences.
Critical strengths
-
This paper addresses an important problem. Evaluating personalized LLM outputs using user-specific rubrics is well-motivated.
-
Framing user-specific rubric formulation as a learnable generation problem is well-motivated. The methodology, including rubric induction, consistency validation, and RL, is well-structured.
-
The improvement in user coverage, from around 49-87% for the baseline methods to 99-100% for PARL, is substantial and demonstrates the overall effectiveness of PARL. The ablation studies further demonstrate the effectiveness of the key components of PARL, and the experiments on out-of-domain category also demonstrates that PARL generalizes across datasets.
Critical weaknesses
-
In this paper, the operational definition of personalized alignment is whether user-authored ground-truth responses have a higher accuracy than baseline responses under the generated rubrics. While this definition is a reasonable and practical proxy, there is no independent validation showing that it is indeed consistent with real human preferences. For example, is it possible that a human user would rank an SFT+GRPO response on par with their own response, given that preferences can be subjective and multi-dimensional? To demonstrate that PARL truly captures user preferences, a user study comparing the rankings of responses induced by PARL-generated rubrics with human rankings would provide a stronger validation beyond the current accuracy metrics alone.
-
The rubric generator is trained only using Qwen3-8B, and the zero-shot generators used as baselines are almost all from the Qwen family (except GPT-4o-mini in Table 7). It is thus unclear whether the PARL pipeline generalizes to other model families or scales, and how its discriminative power varies with model backbone choice.
Questions for authors
Please refer to weaknesses.
2 comments
Follow-up on Our Response and Revision
Dear Reviewer,
Thank you again for your time and thoughtful feedback. We have provided detailed responses to address your concerns and added the following experiments and clarifications, with the revised manuscript highlighted in blue:
- User study: An additional human preference study on 100 Amazon Review cases (Appendix H).
- Generalization beyond the Qwen family: An additional PARL-A rubric generator initialized from Gemma-2-9B-it and trained on Amazon Review, evaluated using the same protocol. We report user coverage, Max-Diff, and results from five runs with paired significance tests (Appendix G.6, Table 8).
- Stronger prompting baselines: Gemini-3.8-Flash, DeepSeek-v4.1-Flash, and GPT-6-luna, evaluated with the same historical consistency filter and judge (Appendix G.7, Table 9).
As the discussion deadline approaches, we would greatly appreciate any further feedback and welcome continued discussion.
Best regards,
The Authors
Dear Reviewer,
We sincerely appreciate your invaluable comments on our work. We have carefully considered all your concerns and have provided detailed responses below.
Thank you for this important suggestion. To directly validate whether PARL's rubric-based judgments align with real human preferences, we conducted a human evaluation on the Amazon Review Generation task. We recruited 5 human evaluators and sampled 20 different test cases for every evaluator. For each case, evaluators were provided with (i) the target item to be reviewed, (ii) the target user's historical profile (past reviews), and (iii) two anonymized and randomly ordered candidate responses (the SFT+GRPO output and the user's actual ground-truth review), without access to their sources, PARL's rubrics, or PARL's scores. Evaluators were asked to judge which response better reflects the target user's preferences and writing style, or indicate a tie if they are on par. We then compared these human judgments against the judgments produced by PARL-A.
Across the 100 cases, human evaluators preferred the user-authored ground-truth response in 90 cases (90%), preferred the SFT+GRPO response in 3 cases (3%), and indicated a tie in 7 cases (7%). Meanwhile, PARL-A preferred the user-authored response in 73 cases (73%), preferred the SFT+GRPO response in 6 cases (6%), and indicated a tie in 21 cases (21%). On decisive (non-tie) judgments, human evaluators selected the ground-truth response in 96.8% of cases (90/93), and PARL-A similarly selected the ground-truth response in 92.4% of cases (73/79). These results suggest that, in the human study, user-authored responses remain clearly distinguishable from SFT+GRPO-generated outputs, and that PARL aligns well with human preferences in direction whenever it makes a decisive judgment.
Thank you for raising this concern. To explore whether the PARL pipeline generalizes to other model families or scales, we trained an additional PARL-A rubric generator initialized from Gemma-2-9B-it on the Amazon Review dataset and evaluated it using the same protocol. In addition, we also performed paired t-tests to examine whether GT responses achieve significantly higher user-level accuracy than competitive baselines across five runs.
The Gemma-based generator achieves 99.51% user coverage and a Max-Diff of 2.35 percentage points, measured between GT and the strongest generation baseline, SFT+GRPO. GT scored significantly higher than all competitive baselines in paired t-tests across the five runs; all comparisons remained significant. These results provide evidence that PARL's ability to distinguish authentic user texts from generated outputs extends beyond the Qwen generator family.
We additionally evaluated Gemini-3.8-Flash, DeepSeek-v4.1-Flash, and GPT-6-luna using the same generation prompts, candidate-generation procedure, historical consistency filter, and Qwen3-235B judge as our existing prompting baseline.
Full-target user coverage is 92.73%, 91.84%, and 97.28%, respectively. Gemini achieves a small positive Max-Diff, whereas DeepSeek and GPT score GRPO and SFT+GRPO above GT. Thus, stronger prompting-only backbones do not consistently produce more discriminative rubrics under this protocol.
Together, these experiments broaden our backbone evaluation and support the applicability of RL-based rubric learning beyond Qwen3-8B. We will include these results in the revision.