Beyond User Identification: Target-Aligned Validation of Personalized LLM Representations

Personalized-LLM evaluations often treat successful user identification, representation separation, or predictive improvement as evidence that a learned user representation captures a named preference. That inference is not valid in general. Users can be distinguishable because of response variability or content exposure even when the target property, such as mean preference direction, is identical. We call this the target-identification problem: does the specific user-level property claimed for a representation actually vary across users? This question is distinct from whether users are identifiable or whether personalization is useful. We introduce Target-Aligned Validation, a practical audit that starts from a prespecified user-level property $T(P_u)$. It checks whether the observation design supports estimating that property, compares estimates from separate histories against a content-preserving reference, tests sensitivity to cross-history dependence, and reports finite-history reliability. Behavioural effects and predictive utility are evaluated separately. Controlled experiments show that broad user identification can succeed when every user has the same mean target. Across 133 MultiPref users and three LLMs, the mean-target test detects heterogeneity at all nine tested model-by-history combinations under chronological blocking, conditional on cross-history error assumptions. At the deepest tested history, estimated reliability is only about 5%, and none of the matched utility comparisons establishes a benefit over pooling. In a post-hoc wider-gap analysis, none of the three models passes the prespecified target decision, leaving the role of dependence unresolved. A partial audit of PerFit shows how the procedure transfers to a learned personalization module and which quantities that design cannot support. Target-Aligned Validation separates user identity, target heterogeneity, individual reliability, and downstream utility into distinct evaluation questions.