- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PersonaAlign evaluates writing-style personalization through repeated interactions with persona-conditioned simulated users, followed by evaluation on unseen topics without feedback. It provides 1,000 curated personas and three writing domains. The main experiments evaluate ten prompting/baseline configurations on 100 personas across four base models; four fine-tuning methods are additionally tested on OLMo-3-7B. Oracle preference prompting generally performs best, while learned methods vary by backbone. Human annotation studies assess simulator behavior, style scoring, and agreement with rankings of selected methods.
Critical strengths
-
Separating preferred writing style from communication style allows the benchmark to vary how preferences are expressed without changing the target behavior. Retaining learned state across episodes and evaluating without further feedback then tests whether personalization transfers to new topics (§3.5). This makes the protocol useful for studying persistent adaptation.
-
The human validation addresses several distinct components: simulator prompt selection, persona curation, style detection, and final pairwise rankings (Appendix I). The 36,000 pairwise annotations and comparison with two judge models provide meaningful evidence for the selected methods’ broad ordering, giving the evaluation more grounding than an unvalidated automated score.
-
The comparisons reveal consequential interactions between backbone and personalization method: CoT and context compaction perform particularly well with Gemma, while Adaptive and FSE perform better with Qwen (Figure 3b). This supports evaluating algorithms across multiple backbones. Appendix C.1 further checks that rankings remain broadly stable under the tested feedback perturbations.
Critical weaknesses
-
The human validation supports a narrower conclusion than the claimed external validity. Section 5 reports agreement with annotators applying supplied style preferences, using four deliberately well-separated methods across two models. This does not establish that learning from simulated feedback predicts success with users expressing their own preferences, or that the judge reliably distinguishes nearby leading methods. The authors acknowledge simulator limitations, but their conclusion about real-world human preferences remains stronger than the evidence. This matters because external validity is central to the benchmark’s motivation.
-
The fine-tuning results leave evaluation robustness and checkpoint selection unresolved. Appendix D candidly reports repetition and length collapse under satisfaction-based training, then selects checkpoints with high scores and non-collapsed average length. However, the selection rule is not operationally specified, and its relationship to the three held-out evaluation tasks is unclear. The paper also does not report whether the final pairwise judge rejects these degenerate outputs. This leaves uncertainty about whether Figure 4b reflects reliable personalization gains under a reproducible selection procedure.
-
The reported win rates need a consistent, explicit definition. Figures 3a and 12 both describe task-wise results averaged across agent LLMs, yet IF-Prompt on About Me is approximately 65% in Figure 3a and 77% in Figure 12. Figure 9 explicitly assigns half-credit to ties; the main figures do not clearly specify their convention. Different tie treatment or opponent pools could explain the discrepancy, but these must be reconciled before readers can reliably interpret performance gaps or reproduce the leaderboard.
Questions for authors
-
Does human agreement persist for close comparisons among SF-Prompt, CC-Prompt, and Adaptive-Prompt? A focused comparison would help establish whether the benchmark distinguishes competitive methods.
-
What exact checkpoint-selection rule was used, was selection independent of final evaluation data, and does the pairwise judge reject the degenerate outputs described in Appendix D?
-
What numerator, denominator, tie credit, and opponent pool underlie Figures 3a, 9, and 12? Does reconciling these definitions change any method ordering?
4 comments
Thanks for the response and for running the extra experiments.
These additions help clarify the win-rate definitions and address some of my earlier questions, though I will keep my rating at 2. My main hesitation with the follow-up human study is that a single aggregate win rate (53.8%) leaves out too much detail: it is not clear whether statistical significance was tested across the 600 independent samples rather than all 6,000 clustered annotations, and there are no per-model/task breakdowns or inter-annotator agreement numbers. Also, the heuristic for checkpoint selection (checking for empty or repetitive outputs) makes sense, but you still haven't confirmed whether that check was run strictly on training/validation outputs independently of the held-out evaluation tasks.
To clarify, the statistical test for significance was carried out over all 100 personas (style sets). This aggregates much of the correlated information at a high-level (for example, repeat votes for the same sample), resulting in a more conservative statistical test. Using a paired t-test, the summarized feedback method is preferred 53.8% of the time with a significance of .
Inter-annotator agreement across all LLMs and tasks in this new study is moderate-low, with Fleiss' . This reflects the close nature of these two methods, in which there is not a clear best method across samples and in which many annotators disagree about which method is superior. This finding is predicted by the PersonaAlign benchmark, which does not find the two methods to be statistically significantly different when aggregated across all models and tasks.
Results broken down by task and LLM are given in the table below:
| Agent LLM | Task | Human SF win | p | Judge SF win | p |
|---|---|---|---|---|---|
| Qwen3.5-35B | About Me | 57.3% | 0.0039* | 57.5% | 0.039 |
| Qwen3.5-35B | Blog | 53.9% | 0.14 | 54.5% | 0.23 |
| Qwen3.5-35B | Product review | 55.5% | 0.041 | 59.0% | 0.014 |
| Gemma-4-31B | About Me | 53.2% | 0.18 | 52.7% | 0.50 |
| Gemma-4-31B | Blog | 51.2% | 0.61 | 40.0% | 0.0058 |
| Gemma-4-31B | Product review | 51.4% | 0.58 | 49.0% | 0.79 |
| Qwen3.5-35B | All tasks | 55.6% | 0.00043* | 57.0% | 0.0026* |
| Gemma-4-31B | All tasks | 51.9% | 0.20 | 47.2% | 0.22 |
| Both | All tasks | 53.8% | 0.0016* | 52.1% | 0.19 |
Where * indicates a result is significant after Bonferroni correction ().
Finally, the checkpoint selection was conducted by checking for repeated n-grams and empty turns over all of the outputs from a training iteration, which included samples in the held-out evaluation set.
Thanks for the follow-up and the extra details. Running the paired t-test at the 100-persona level makes the significance test much cleaner, and the human–judge agreement on Qwen3.5-35B is consistent across tasks.
Still, I am keeping my rating at 2 because two core issues remain:
-
Since checkpoint selection checked outputs that included the held-out evaluation set, the fine-tuning results in Figure 4b are affected by test-set selection rather than coming from an independent validation split.
-
In the breakdown, the judge favors CC over SF on Gemma-4-31B blog posts (40.0% SF win rate, unadjusted p = 0.0058), whereas human annotators see no real difference (51.2%, p = 0.61). Coupled with the low inter-annotator agreement (Fleiss' κ = 0.157) and the reliance on prescribed personas rather than users' own preferences, the evidence for fine-grained and real-world reliability is still not strong enough.
We thank the reviewer for their thoughtful review.
We agree that the human validation is narrower than validating how well PersonaAlign predicts success with users expressing their own preferences. We will update the text to make this limitation and distinction more clear as well as mentioning this extension in future work. However, it was a specific design decision made to avoid non-trivial sources of noise and uncertainty in our results. The benchmark operates in the setting where a user knows their preferences and is able to use this preference information to guide their feedback. Successful preference solicitation is an active area of research and is an unsolved problem that is confounded by users not always knowing their preferences in advance. Therefore, establishing gold standard, ground truth preference for participants is not possible, making it difficult to identify if performance mismatches are due to ground truth preference solicitation protocols versus issues with the benchmark. As a first step towards a benchmark for learning from feedback, we chose a human validation setting where noise from the user was removed as a possible confounding factor.
Yes, the final pairwise judge can catch reward-hacked results and penalize them. To answer this question, we ran the judge for the LoRA and SFT+DPO methods across various checkpoints, comparing to the checkpoint that was used in the paper. The results are below:
These results show that the win rates decrease with time, as the models begin to reward hack more and more. Finally, we bucket all fine-tuned responses and label them as either repeating n-grams (loops) or non-degenerate (clean). Comparing to a zero-shot baseline, we find the following:
These results show that the pairwise judge is able to detect and reject degenerate outputs, as the repetitive outputs are consistently not preferred, and the more reward-hacked fine-tuning models are also consistently not preferred. We will add these investigations and results to Appendix D.
We thank the reviewer for noting this discrepancy. Win rates are computed by assigning ties half-credit. This will be clarified in the main paper and it is consistent in the code. Figure 3 in the paper used a different definition (ties as losses), but fixing this results in minimal changes to results. To verify this, we ran a correlation between the ties-as-losses results and the ties-as-half-credit results, which showed a Kendall’s tau ranking correlation of 0.96.
Checkpoints were selected by a heuristic that checked for empty responses or repetitive n-grams. Once these appeared in any of the generations, we chose the most recent checkpoint as the “final” for evaluation purposes.
We assigned ties half credit, wins full credit, and losses no credit. The results ordering is unchanged between the two methods, and the win-rate calculation will be explicitly outlined in the final paper.