acceptodds
Under review as a conference paper at ICLR 2027

Beyond User Identification: Target-Aligned Validation of Personalized LLM Representations

Abstract

Personalized-LLM evaluations often treat successful user identification, representation separation, or predictive improvement as evidence that a learned user representation captures a named preference. That inference is not valid in general. Users can be distinguishable because of response variability or content exposure even when the target property, such as mean preference direction, is identical. We call this the target-identification problem: does the specific user-level property claimed for a representation actually vary across users? This question is distinct from whether users are identifiable or whether personalization is useful. We introduce Target-Aligned Validation, a practical audit that starts from a prespecified user-level property . It checks whether the observation design supports estimating that property, compares estimates from separate histories against a content-preserving reference, tests sensitivity to cross-history dependence, and reports finite-history reliability. Behavioural effects and predictive utility are evaluated separately. Controlled experiments show that broad user identification can succeed when every user has the same mean target. Across 133 MultiPref users and three LLMs, the mean-target test detects heterogeneity at all nine tested model-by-history combinations under chronological blocking, conditional on cross-history error assumptions. At the deepest tested history, estimated reliability is only about 5%, and none of the matched utility comparisons establishes a benefit over pooling. In a post-hoc wider-gap analysis, none of the three models passes the prespecified target decision, leaving the role of dependence unresolved. A partial audit of PerFit shows how the procedure transfers to a learned personalization module and which quantities that design cannot support. Target-Aligned Validation separates user identity, target heterogeneity, individual reliability, and downstream utility into distinct evaluation questions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
personalized language models, representation evaluation, target identification, user modeling, reliability, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HF55OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows that existing evaluations for personalized LLM representations conflate four separate questions: whether users can be told apart, whether a specific representation property (such as mean preference direction) varies across users, how reliably this property can be estimated for an individual user from limited history, and whether it improves downstream prediction. It proposes Target Aligned Validation, an audit built around a predefined user-level functional; this audit verifies sufficient shared user content for target estimation, calculates the target on two disjoint splits of each user’s history, applies cross-fitting to prevent sampling noise from being mistaken for user differences, removes a reference baseline made by permuting user labels within identical-content groups, and reports split-half reliability R(n) and utility separately. The statistic’s expectation matches the squared distance between true user means only when errors from the two splits are uncorrelated (Assumption 1). Controlled simulations show common user identification tests reject the null in 60–75% of trials when users share identical means but differ in covariance orientation, while the mean target test rejects at most 2% of the time. On 133 MultiPref users and three LLMs (Llama 3.1 8B, Qwen2.5 7B, Mistral 7B), the mean target test detects heterogeneity across all nine model-history combinations under chronological splitting, yet reliability is only around 5% given 64 judgments per user and no matched comparison outperforms the pooled preference direction; post-hoc buffered gap analysis weakens effects such that none of the three models satisfy the predefined rule, and SFF shows the reverse pattern with prediction gains despite no detectable mean heterogeneity. The work also provides a partial audit of PerFit’s learned shift on LaMP 2M and adds a within-user dispersion target in the appendix.

Critical strengths

First, the conceptual separation is correct, and the controlled experiments make it concrete. Being able to identify users, showing that a named property differs, being able to estimate that property per person, and getting a prediction gain are different claims, and the paper shows each can hold without the others. The equal mean, different covariance worlds, where identity tests fire in 60 to 75 percent of trials while the mean target test stays at or below 2 percent, are a clean demonstration. The real data contrast between MultiPref (target detected, no utility gain) and SFF (utility gain at 16 judgments with Qwen, target not detected) shows the dissociation in both directions. Personalization papers do often infer the wrong thing from identification or utility alone, so this is a useful message.

Secondly, the paper is unusually upfront about what its own procedure cannot do. Appendix I shows the false positive rate of the mean target test going from 0.034 to 0.980 at a cross half error correlation of only 0.05, and shows that chronological blocking does nothing against a persistent user specific offset (false positive rate 1.000 under both random and blocked splits). The buffered gap arm fails the prespecified rule in all three models and is reported rather than buried. Several analyses are labeled post hoc, the dispersion bias is documented with two attempted corrections, and the PerFit reliability ratio is explicitly declared not estimable as a target reliability. This level of disclosure is rare and makes the work easy to assess.

Critical weaknesses

One thing which I think is critial is the central real data claim cannot be separated from the confound the paper itself identifies. The mean target test is valid only if errors in the two histories are uncorrelated (Assumption 1), and the paper's own simulations show that a tiny violation, or any persistent user specific offset (for example annotator style or length preference that is not the target), produces rejection with probability near one under random and chronological splits alike. Blocking helps only against short range dependence. The only real data check that widens the temporal gap (the buffered early to late arm) fails the prespecified rule in all three models, and the paper concedes it cannot say whether that is due to dependence, content change or drift. So the headline MultiPref result, "heterogeneity detected in all nine cells," is exactly what a persistent annotator offset would also produce, and the audit as proposed has no way to rule that out. The one design that would (repeated fixed content probes across independent sessions, Appendix F) is proposed but not run. This weakens the paper's positive claim and, more importantly, its claim that the audit gives a more reliable inference than user identification: in the main real dataset it returns an answer conditional on an assumption that the paper cannot verify.

Questions for authors

1/ Given the simulation showing near certain rejection under a persistent user offset, what evidence do you have that the MultiPref detection is not an annotator level nuisance effect? Is there any dataset with independent sessions on fixed content where you know the answer?

2/ For the utility conclusion, what effect size would your audit have detected with 80 percent power at K=133 and 64 judgments, and how does that compare with plausible gains from personalization?

3 comments
AuthorsOpenReview ↗

Part 1/2 (W1 and Q1)

We thank the reviewer for the careful reading. The review prompted us to re-examine MultiPref and clarify our claims in the revised manuscript. We address W1 and Q1 here; Q2 follows in a separate comment.

W1. Persistent annotator effects could explain the MultiPref detections, weakening the claim that the audit gives more reliable inference than user identification.

The nine MultiPref detections depend on Assumption 1. We have addressed this issue in our revised manuscript (Abstract, Introduction, and Section 6), where the detections are presented as conditional findings rather than the main human-data result.

However, we believe the audit still provides evidence that user identification alone cannot offer. Identification can detect differences in covariance or content exposure even when the named mean target is identical. In Figure 2, identification rejects in 60–75% of equal-mean trials, while the mean-target test rejects in at most 2%. On real MultiPref activation geometry with an imposed equal-mean null, three identification-style diagnostics also reported user signal in all 30 surrogate trials under their original thresholds (Appendix G.1). The result illustrates why detecting user differences does not, by itself, establish differences in a particular mean target.

The audit also evaluates how reproducible an individual estimate is. At 64 judgments, MultiPref's observed cross-half ratios are around 5% across three backbones and 4.6–5.4% across three Llama layers (Appendix L). The median ratios are 5.5–6.8% across 200 random partitions (Section 6; Appendix D). Even after model-based content adjustments, the ratios remain below 9%. If shared errors introduce positive bias after content-reference subtraction, the estimated signal would be too large under the model in S.1. We have not established the direction of that bias in MultiPref, so the reported ratios are not upper bounds on true reliability. The held-out utility analysis does not depend on Assumption 1.

Q1. “Given the simulation showing near certain rejection under a persistent user offset, what evidence do you have that the MultiPref detection is not an annotator level nuisance effect? Is there any dataset with independent sessions on fixed content where you know the answer?”

Regarding the first part, the current human data cannot rule out persistent annotator habits, such as consistently preferring longer responses or a particular candidate label. We investigated how measured response characteristics and shared session states affect the MultiPref results.

At 64 judgments, removing length-associated components reduces the cross-half increment by 52–54%. Joint adjustment for length, candidate slot, and source model reduces it by 66–74%, and none of the jointly adjusted tests passes the original Holm rule at this depth (S.6, Table 14). The adjustment was fitted on 3,113 separate comparisons without using the evaluation annotators' choices. However, some prespecified validation checks were not satisfied, so we report the findings as descriptive sensitivity evidence. A genuine preference for longer responses could also contribute to the observed mean, meaning the removed component is not necessarily nuisance.

The timestamp audit shows that 73% of users' chronological halves share a session under a 30-minute-gap definition. In simulations with dependence lasting up to seven days, a seven-day lag restriction reduces family-wise false positives from 29.8% to 8.1%, still above the nominal 5% rate (S.4). At 16 judgments per half, the buffered ratios remain positive at 1.95%, 2.58%, and 1.82%, compared with 3.77%, 4.22%, and 4.00% for the original split. However, the buffered confidence intervals include zero. Early–middle and middle–late splits also have the same median boundary gap but different increments. Temporal separation alone therefore does not explain the attenuation.

We have addressed both analyses in revised Section 6 and Appendices S.4 and S.6.

Regarding the second part, our human datasets do not contain independent sessions on identical fixed content with externally validated preferences. We added a controlled positive experiment using 48 sampling profiles of one LLM, fixed response pairs, four separately initialized sessions, and randomized writing-register instructions (S.7). The measured register score changes with the assigned preference in both directions. The experiment provides evidence that a specified representation property responds to a controlled preference manipulation, although it does not establish the semantic origin of the human MultiPref signal. We have clarified the scope of the MultiPref claims in the revised manuscript.

AuthorsOpenReview ↗

Part 2/2 (Q2)

(Continuation of response)

Q2. “For the utility conclusion, what effect size would your audit have detected with 80 percent power at K=133 and 64 judgments, and how does that compare with plausible gains from personalization?”

Regarding the detectable effect size, the post-hoc 80%-power thresholds are 1.05, 1.04, and 1.38 accuracy percentage points for Llama, Qwen, and Mistral when comparing cross-fitted shrinkage with leave-target-out pooling (Appendix S.3, Table 12). The observed gains are +0.17, −0.09, and +0.39 points. Accounting for shared test comparisons increases the approximate detectable thresholds to 1.6–2.2 points.

Regarding plausible gains, the original user-level procedure has 57–78% simulated power for a one-point improvement and 98–100% for a two-point improvement. Simulations matched to the observed signal scale yield power of about 0.38–0.39 (Appendix J). Thus, gains of the size we consider plausible, around one percentage point or less, are at or below what this study could reliably detect. The +1.23-point SFF/Qwen gain illustrates a possible effect size, although it does not predict the gain in MultiPref.

We have addressed the power question in our revised manuscript (Section 6 and Appendix S.3). Our conclusion is that the evaluated methods show no established average advantage over pooling at 64 judgments. Smaller or user-specific gains remain possible.

Overall, the additional analyses strengthen the distinction at the center of our paper. User identification, target detection, individual reproducibility, and downstream utility answer different questions. By evaluating them separately, Target-Aligned Validation determines which conclusions the available evidence supports and where further validation is needed. We have clarified these distinctions in the revised Abstract, Introduction, Section 6, and Appendices S.1–S.3.

Thank you for the careful response and the new analysis. The sensitivity results answer my question, though mostly in the direction I was worried about. Hence, I've decided to keep my score unchanged.

Reviewer_KgY6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper argues that identifying users from personalized representations does not show which preference properties differ between them. It proposes an audit that defines a specific target, compares estimates from separate histories, and controls for differences in content exposure. It also separates detecting user differences from estimating individual preferences reliably and improving predictions. The main MultiPref experiment uses 133 users and three model backbones. It detects differences in mean preference directions under the stated assumptions. At 64 total judgments per user, the paper reports reliability of about 5%, with no established prediction gain over pooling other users’ preferences.

Critical strengths

The paper clearly explains why user identification does not validate a specific preference property. Figure 2 shows that users can be distinguishable even when their mean preference directions are identical. Differences in spread or covariance can produce identification signals instead. This gives a concrete reason to evaluate the intended property directly. The distinction is useful for interpreting personalization results and is the paper’s main contribution.

The paper separates statistical detection, individual reliability, and practical usefulness. Proposition 2 explains how population differences can be detectable while individual estimates remain noisy. The MultiPref experiments examine these questions on the same users. They find conditional evidence of mean differences without an established prediction gain over pooling. The paper also reports that wider temporal gaps weaken the target evidence. These results help prevent a positive statistical test from being treated as proof of useful personalization.

Critical weaknesses

The reliability interpretation needs more justification after content correction. Equation (7) adjusts the numerator using a content-preserving reference, but its denominator uses the unadjusted variance of the estimates. If persistent content-related variation remains in that denominator, the remaining variance cannot all be treated as finite-history noise. Appendix D derives reliability under a signal-plus-noise model, but does not explicitly account for this content component. Appendix Q.3 recognizes a similar issue for PerFit. The main activation analysis needs its own justification. This matters because the roughly 5% reliability estimate is a central empirical finding.

The PerFit target and its measurement do not clearly match. Section 8 defines the target as each user’s mean learned shift over the same 681 shared movie descriptions. Appendix Q.3 describes readouts over each user’s own item set, with 8–181 items per user. It is unclear how these different subsets estimate the same common-content target. Content-preserving permutations alone do not establish that link. The authors should clarify the readout procedure and any weighting used. This matters because the PerFit experiment is meant to show that the audit extends to learned personalization modules.

Questions for authors

How much content-related variation remains in the denominator of Equation (7) for the main activation experiments? Under what conditions does the corrected ratio measure target reliability rather than a descriptive signal share?

For PerFit, is each user’s mean shift evaluated over all 681 shared descriptions or only that user’s own subset? If subsets differ, what weighting or assumptions connect the estimates to the common-content target in Section 8?

1 comment
AuthorsOpenReview ↗

W1/Q1 and W2/Q2

We thank the reviewer for the careful reading and the constructive feedback. We have addressed both concerns in the revised manuscript through additional reliability analyses (Appendix S.1) and a complete common-content evaluation of PerFit (Section 8 and Appendix S.2). We respond to W1/Q1 and W2/Q2 below.

W1 / Q1. How much content-related variation remains in Equation (7)'s denominator, and under what conditions does the ratio measure target reliability?

Regarding the content contribution, we reproduced all 18 original MultiPref ratios and decomposed the denominator using a fitted additive model. We estimate that 32–37% of the original denominator is associated with content variation. We also found that the content-preserving reference attenuates the numerator by approximately 14% under the model's assumptions.

At 64 judgments, the original and jointly adjusted ratios are 4.56% and 7.94% for Llama, 5.14% and 8.95% for Qwen, and 4.47% and 7.86% for Mistral. All three adjusted ratios remain below 9%, so the conclusion of limited individual reproducibility is unchanged.

Regarding interpretation, the ratio can be treated as target reliability only if the additive variance model is appropriate, content incidence and co-annotation are adequately accounted for, and estimation errors across history halves are sufficiently uncorrelated. Otherwise, the ratio is a descriptive measure of cross-half signal relative to variation in the estimates.

We have addressed the interpretation and the assumptions behind the adjustments in Appendix S.1 of our revised manuscript. The denominator adjustment changes the reliability estimate but does not affect the target-detection statistic.

W2 / Q2. Was the PerFit mean shift evaluated over all 681 shared descriptions or each user's own subset? How does the weighting relate to the common-content target?

Regarding the original evaluation, we used each user's subset of 8–181 descriptions, giving equal weight to items within each user's subset and then equal weight to users. Because the subsets differed, the original readout did not directly estimate the stated uniform 681-item target without additional assumptions about how those items were selected.

To address the common-content target directly, we recomputed the PerFit readout for all 94 users on the complete set of 681 descriptions. The raw cross-half statistic is 6.882, compared with 6.902 on the original user-specific subsets. It is also 6.882 on a common 195-item panel excluded from all evaluated users' training histories. Owner-matching AUC changes only from 0.793 to 0.790. Approximately 99.9% of the learned-shift energy is item-invariant, which helps explain the small difference across evaluation panels.

We have updated Section 8 and Appendix S.2 of our revised manuscript to report the full 681-item evaluation. We also distinguish the raw cross-half statistic from the original reference-subtracted increment, since the content reference is approximately zero on a fully common panel. The complete-domain analysis directly addresses the target mismatch raised in the review, while the main empirical finding remains unchanged.

We hope these additional analyses resolve the reviewer's concerns about reliability interpretation and PerFit measurement consistency.

Reviewer_U7uJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work addresses an evaluation problem in LLM personalization and provides a systematic framework for distinguishing user identification, target heterogeneity, estimation reliability, and downstream utility. The experimental analysis is reasonably comprehensive and highlights several important limitations of existing evaluation practices.

Critical strengths
  1. This work addresses a relevant evaluation problem in LLM personalization by distinguishing user identification from the identification of specific preference-related properties. This distinction helps clarify what existing representation-based evaluations can and cannot establish.

  2. The proposed framework provides a structured evaluation protocol that considers content exposure, cross-history consistency, estimation reliability, and downstream utility. Although the individual techniques are established, their integration offers a systematic way to examine personalized representations from multiple perspectives.

  3. The experimental analysis is comprehensive, covering controlled simulations, and three LLM backbones. The authors also investigate important limitations, including cross-history dependence and insufficient content overlap, and appropriately qualify several conclusions based on these findings.

Critical weaknesses
  1. The proposed method critically relies on the assumption that estimation errors from two history splits are uncorrelated, which is difficult to guarantee in real-world user interactions. The authors' own simulations show that a cross-history error correlation of only 0.05 increases the false-positive rate from 3.4% to 98%. More importantly, when increasing the temporal separation between histories, none of the three models passes the prespecified target-identification test. Although this analysis also changes content composition, these results raise serious concerns about whether the detected user differences reflect genuine preference heterogeneity or persistent user-specific noise. The paper lacks sufficient empirical evidence to validate this central assumption.

  2. This work does not sufficiently establish that its predefined statistical targets correspond to meaningful user preferences. Specifically, the main experiments define the target as the mean direction of activation differences between preferred and rejected responses. However, reproducible differences in these activation means may arise from stable user-specific factors unrelated to the intended preference semantics. The framework primarily verifies whether a predefined representation statistic differs across users, rather than whether it captures the claimed preference attribute. Without independent preference annotations or target-specific behavioral validation, the connection between statistical target identification and genuine preference understanding remains unclear.

  3. The framework heavily depends on exact content overlap across users, which is often unavailable in realistic personalization scenarios. This requirement substantially reduces usable observations in several datasets, limiting the framework's generalizability and practical applicability.

  4. The methodological novelty is somewhat limited, as the framework primarily combines established statistical techniques, including cross-fitting, permutation-based controls, and reliability estimation. The paper does not clearly demonstrate what new capabilities this combination provides beyond existing approaches.

  5. The practical implications remain insufficiently explored. Although the paper distinguishes target identification, estimation reliability, and downstream utility, it does not demonstrate how the proposed framework can guide representation learning or improve personalization performance.

Questions for authors

Pls see the weakness

2 comments
AuthorsOpenReview ↗

Part 1/2 (W1–W3)

We thank the reviewer for the detailed comments and for recognizing the importance of the evaluation problem. The review raised valuable questions about the meaning of the target and the contribution beyond existing evaluations. We have addressed these concerns in our revised manuscript and respond to W1–W5 in two comments.

W1. The test assumes uncorrelated history errors, and the evidence weakens when the temporal gap increases.

The nine MultiPref detections remain conditional on Assumption 1. We have addressed the dependence concern in our revised manuscript (Section 6 and Appendix S.4) through a timestamp audit and additional simulations. At 64 judgments, 73% of users have history halves that share a session under a 30-minute-gap rule. In simulations with persistent states lasting up to seven days, a lag restriction reduces family-wise false positives from 29.8% to 8.1%, although the rate remains above the nominal 5%. We therefore do not use the restricted procedure for new MultiPref detection claims.

The buffered estimates remain positive, although their confidence intervals include zero. The early–middle and middle–late designs also have the same median boundary gap but different increments across all three models (Tables 4–5). Increasing the gap therefore does not isolate temporal dependence as the cause of the weaker result. Other split-history and owner-matching evaluations can also reflect persistent user-specific effects. Appendix I quantifies how correlated errors affect our cross-history test. The held-out utility and content-support evaluations do not require Assumption 1.

W2. Mean activation differences may reflect stable response habits rather than genuine preferences.

The available MultiPref observations cannot distinguish stable choice preferences from unrelated annotation habits. We have addressed the interpretation of the mean target in our revised manuscript (Section 6 and Appendices S.6–S.7) by examining which response characteristics contribute to the measured signal.

At 64 judgments, adjusting for response length reduces the cross-half increment by 52–54%. Joint adjustment for length, candidate slot, and source model reduces it by 66–74%, and none of the jointly adjusted tests passes the original Holm rule (Appendix S.6, Table 14). Because the adjustment did not satisfy every prespecified validation check, we report these reductions as descriptive findings. A preference for longer responses may itself be a genuine component of the observed choice-derived mean.

Testing the specified mean remains necessary because neither user identification nor predictive accuracy can establish whether that particular representation property differs across users. To demonstrate how the protocol evaluates a meaningful, predefined attribute, we added a controlled linguistic-register experiment. A direction learned from separate calibration text correctly distinguishes all 32 held-out formal/conversational pairs. The representation score also follows randomized preference reversal across four sessions in 48 stochastic sampling profiles of one LLM (Appendix S.7). The experiment demonstrates that the specified representation property responds to an assigned linguistic preference under controlled conditions, while the semantic interpretation of the human MultiPref signal remains conditional on further evidence.

W3. Exact-content overlap excludes too many observations and limits applicability.

Exact matching substantially reduces usable data in sparse datasets. Only 6.4% of Track B build observations and 0.8% of PRISM build observations satisfy the shared-content requirement. By comparison, shared build-content support is 94.6% in MultiPref and 100% in Community Alignment Track A and SFF, although these figures do not establish support for every held-out evaluation.

We chose exact matching to control differences in content exposure without introducing errors from a fitted content-effect model. For Track B and PRISM, researchers need additional shared-content judgments or a separately validated content adjustment before drawing conclusions about the specified target. The audit identifies insufficient observation support instead of drawing conclusions from comparisons that the available data cannot support.

We have clarified the applicability of exact matching and the possibility of alternative content-adjustment methods in Sections 3 and 6 of our revised manuscript.

AuthorsOpenReview ↗

Part 2/2 (W4–W5)

(Continuation of response)

W4. The framework combines established statistics. What new capability does it provide?

Our new known-truth benchmark demonstrates a failure that user identification or utility evaluations alone cannot reveal (Appendix S.5). In a synthetic setting where users have identical content-controlled mean targets, personalized prediction improves accuracy by an average of 2.73 percentage points. Across 500 replicates, the utility test detects an improvement in 75.8% of trials, while cross-fitting without content control incorrectly rejects equal mean targets in 84.2%. Our complete target-specific procedure rejects in only 1.8%. The improvement in predictive performance is genuine, yet attributing it to differences in the specified mean target would be incorrect. Figure 2 demonstrates a related problem when user identification is used to support a mean-target claim.

We developed Target-Aligned Validation to examine whether the evidence supports the particular representation property claimed by a personalization method. The protocol checks whether sufficient shared content is available, evaluates target differences while controlling for content exposure, measures individual reproducibility, and assesses downstream utility separately. We deliberately build on established statistical methods because their assumptions and results can be examined. However, none of the six representative evaluations compared in Appendix R implements the complete procedure.

The known-truth experiments demonstrate why such an evaluation is necessary, even when personalized predictions genuinely improve. We have strengthened the comparison with existing work in Section 9 and added the new experimental evidence in Appendix S.5 of our revised manuscript.

W5. The paper does not show how the audit guides representation learning or improves personalization.

The audit helps researchers assess whether a user-specific representation is reliable enough to justify personalization. In MultiPref, individual estimates show limited reproducibility at 64 judgments, and the held-out comparisons establish no average advantage over pooling.

A separate steering experiment reaches a consistent conclusion. Calibration on 40 users independently selects zero weight on the individual steering vector, and every nonzero weight has a negative calibration gain relative to pooling. On 93 held-out recipients, owner-minus-pooled steering is −0.000102, with a 95% confidence interval of [−0.000348, +0.000148] (Appendix P). Although the steering policy was selected independently of the audit, the result agrees with the limited-reproducibility finding.

The audit also helps identify what additional evidence would be useful. Under the fitted aggregation model, reaching 50% reliability for a single mean estimate would require approximately 590–684 judgments per user (Appendix D). This is a model-based projection rather than an experimentally established threshold. For Track B and PRISM, additional shared-content observations or a validated content-adjustment method would be needed before stronger target claims could be supported.

We have addressed these practical implications in Section 6 and Appendix P of our revised manuscript. The findings illustrate how the audit can inform decisions about individualization, data collection, and the strength of conclusions drawn from personalized representations.

We hope the additional experiments and manuscript revisions clarify the methodological contribution and address the reviewer's concerns about novelty, soundness, and practical relevance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.