- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper shows that existing evaluations for personalized LLM representations conflate four separate questions: whether users can be told apart, whether a specific representation property (such as mean preference direction) varies across users, how reliably this property can be estimated for an individual user from limited history, and whether it improves downstream prediction. It proposes Target Aligned Validation, an audit built around a predefined user-level functional; this audit verifies sufficient shared user content for target estimation, calculates the target on two disjoint splits of each user’s history, applies cross-fitting to prevent sampling noise from being mistaken for user differences, removes a reference baseline made by permuting user labels within identical-content groups, and reports split-half reliability R(n) and utility separately. The statistic’s expectation matches the squared distance between true user means only when errors from the two splits are uncorrelated (Assumption 1). Controlled simulations show common user identification tests reject the null in 60–75% of trials when users share identical means but differ in covariance orientation, while the mean target test rejects at most 2% of the time. On 133 MultiPref users and three LLMs (Llama 3.1 8B, Qwen2.5 7B, Mistral 7B), the mean target test detects heterogeneity across all nine model-history combinations under chronological splitting, yet reliability is only around 5% given 64 judgments per user and no matched comparison outperforms the pooled preference direction; post-hoc buffered gap analysis weakens effects such that none of the three models satisfy the predefined rule, and SFF shows the reverse pattern with prediction gains despite no detectable mean heterogeneity. The work also provides a partial audit of PerFit’s learned shift on LaMP 2M and adds a within-user dispersion target in the appendix.
Critical strengths
First, the conceptual separation is correct, and the controlled experiments make it concrete. Being able to identify users, showing that a named property differs, being able to estimate that property per person, and getting a prediction gain are different claims, and the paper shows each can hold without the others. The equal mean, different covariance worlds, where identity tests fire in 60 to 75 percent of trials while the mean target test stays at or below 2 percent, are a clean demonstration. The real data contrast between MultiPref (target detected, no utility gain) and SFF (utility gain at 16 judgments with Qwen, target not detected) shows the dissociation in both directions. Personalization papers do often infer the wrong thing from identification or utility alone, so this is a useful message.
Secondly, the paper is unusually upfront about what its own procedure cannot do. Appendix I shows the false positive rate of the mean target test going from 0.034 to 0.980 at a cross half error correlation of only 0.05, and shows that chronological blocking does nothing against a persistent user specific offset (false positive rate 1.000 under both random and blocked splits). The buffered gap arm fails the prespecified rule in all three models and is reported rather than buried. Several analyses are labeled post hoc, the dispersion bias is documented with two attempted corrections, and the PerFit reliability ratio is explicitly declared not estimable as a target reliability. This level of disclosure is rare and makes the work easy to assess.
Critical weaknesses
One thing which I think is critial is the central real data claim cannot be separated from the confound the paper itself identifies. The mean target test is valid only if errors in the two histories are uncorrelated (Assumption 1), and the paper's own simulations show that a tiny violation, or any persistent user specific offset (for example annotator style or length preference that is not the target), produces rejection with probability near one under random and chronological splits alike. Blocking helps only against short range dependence. The only real data check that widens the temporal gap (the buffered early to late arm) fails the prespecified rule in all three models, and the paper concedes it cannot say whether that is due to dependence, content change or drift. So the headline MultiPref result, "heterogeneity detected in all nine cells," is exactly what a persistent annotator offset would also produce, and the audit as proposed has no way to rule that out. The one design that would (repeated fixed content probes across independent sessions, Appendix F) is proposed but not run. This weakens the paper's positive claim and, more importantly, its claim that the audit gives a more reliable inference than user identification: in the main real dataset it returns an answer conditional on an assumption that the paper cannot verify.
Questions for authors
1/ Given the simulation showing near certain rejection under a persistent user offset, what evidence do you have that the MultiPref detection is not an annotator level nuisance effect? Is there any dataset with independent sessions on fixed content where you know the answer?
2/ For the utility conclusion, what effect size would your audit have detected with 80 percent power at K=133 and 64 judgments, and how does that compare with plausible gains from personalization?
3 comments
Part 2/2 (Q2)
(Continuation of response)
Q2. “For the utility conclusion, what effect size would your audit have detected with 80 percent power at K=133 and 64 judgments, and how does that compare with plausible gains from personalization?”
Regarding the detectable effect size, the post-hoc 80%-power thresholds are 1.05, 1.04, and 1.38 accuracy percentage points for Llama, Qwen, and Mistral when comparing cross-fitted shrinkage with leave-target-out pooling (Appendix S.3, Table 12). The observed gains are +0.17, −0.09, and +0.39 points. Accounting for shared test comparisons increases the approximate detectable thresholds to 1.6–2.2 points.
Regarding plausible gains, the original user-level procedure has 57–78% simulated power for a one-point improvement and 98–100% for a two-point improvement. Simulations matched to the observed signal scale yield power of about 0.38–0.39 (Appendix J). Thus, gains of the size we consider plausible, around one percentage point or less, are at or below what this study could reliably detect. The +1.23-point SFF/Qwen gain illustrates a possible effect size, although it does not predict the gain in MultiPref.
We have addressed the power question in our revised manuscript (Section 6 and Appendix S.3). Our conclusion is that the evaluated methods show no established average advantage over pooling at 64 judgments. Smaller or user-specific gains remain possible.
Overall, the additional analyses strengthen the distinction at the center of our paper. User identification, target detection, individual reproducibility, and downstream utility answer different questions. By evaluating them separately, Target-Aligned Validation determines which conclusions the available evidence supports and where further validation is needed. We have clarified these distinctions in the revised Abstract, Introduction, Section 6, and Appendices S.1–S.3.
Thank you for the careful response and the new analysis. The sensitivity results answer my question, though mostly in the direction I was worried about. Hence, I've decided to keep my score unchanged.
Part 1/2 (W1 and Q1)
We thank the reviewer for the careful reading. The review prompted us to re-examine MultiPref and clarify our claims in the revised manuscript. We address W1 and Q1 here; Q2 follows in a separate comment.
W1. Persistent annotator effects could explain the MultiPref detections, weakening the claim that the audit gives more reliable inference than user identification.
The nine MultiPref detections depend on Assumption 1. We have addressed this issue in our revised manuscript (Abstract, Introduction, and Section 6), where the detections are presented as conditional findings rather than the main human-data result.
However, we believe the audit still provides evidence that user identification alone cannot offer. Identification can detect differences in covariance or content exposure even when the named mean target is identical. In Figure 2, identification rejects in 60–75% of equal-mean trials, while the mean-target test rejects in at most 2%. On real MultiPref activation geometry with an imposed equal-mean null, three identification-style diagnostics also reported user signal in all 30 surrogate trials under their original thresholds (Appendix G.1). The result illustrates why detecting user differences does not, by itself, establish differences in a particular mean target.
The audit also evaluates how reproducible an individual estimate is. At 64 judgments, MultiPref's observed cross-half ratios are around 5% across three backbones and 4.6–5.4% across three Llama layers (Appendix L). The median ratios are 5.5–6.8% across 200 random partitions (Section 6; Appendix D). Even after model-based content adjustments, the ratios remain below 9%. If shared errors introduce positive bias after content-reference subtraction, the estimated signal would be too large under the model in S.1. We have not established the direction of that bias in MultiPref, so the reported ratios are not upper bounds on true reliability. The held-out utility analysis does not depend on Assumption 1.
Q1. “Given the simulation showing near certain rejection under a persistent user offset, what evidence do you have that the MultiPref detection is not an annotator level nuisance effect? Is there any dataset with independent sessions on fixed content where you know the answer?”
Regarding the first part, the current human data cannot rule out persistent annotator habits, such as consistently preferring longer responses or a particular candidate label. We investigated how measured response characteristics and shared session states affect the MultiPref results.
At 64 judgments, removing length-associated components reduces the cross-half increment by 52–54%. Joint adjustment for length, candidate slot, and source model reduces it by 66–74%, and none of the jointly adjusted tests passes the original Holm rule at this depth (S.6, Table 14). The adjustment was fitted on 3,113 separate comparisons without using the evaluation annotators' choices. However, some prespecified validation checks were not satisfied, so we report the findings as descriptive sensitivity evidence. A genuine preference for longer responses could also contribute to the observed mean, meaning the removed component is not necessarily nuisance.
The timestamp audit shows that 73% of users' chronological halves share a session under a 30-minute-gap definition. In simulations with dependence lasting up to seven days, a seven-day lag restriction reduces family-wise false positives from 29.8% to 8.1%, still above the nominal 5% rate (S.4). At 16 judgments per half, the buffered ratios remain positive at 1.95%, 2.58%, and 1.82%, compared with 3.77%, 4.22%, and 4.00% for the original split. However, the buffered confidence intervals include zero. Early–middle and middle–late splits also have the same median boundary gap but different increments. Temporal separation alone therefore does not explain the attenuation.
We have addressed both analyses in revised Section 6 and Appendices S.4 and S.6.
Regarding the second part, our human datasets do not contain independent sessions on identical fixed content with externally validated preferences. We added a controlled positive experiment using 48 sampling profiles of one LLM, fixed response pairs, four separately initialized sessions, and randomized writing-register instructions (S.7). The measured register score changes with the assigned preference in both directions. The experiment provides evidence that a specified representation property responds to a controlled preference manipulation, although it does not establish the semantic origin of the human MultiPref signal. We have clarified the scope of the MultiPref claims in the revised manuscript.