acceptodds
Under review as a conference paper at ICLR 2027

Is There an Imposter Among Us? Auditing User Simulators with Attribution Trials

Abstract

User simulators are commonly evaluated by their ability to steer toward the target user’s behavior, given the user’s profile or history. However, this similarity does not show whether the simulator captures what makes that individual unique, rather than behavior shared by many users. In interactive systems, this can lead to inaccurate simulations of users and thus, unreliable results. We identify this as the simulators’ lack of individual fidelity – the ability to capture behavior unique to the intended user. We therefore develop Attribution Trials (AT), a controlled audit for testing whether a simulator represents that individual rather than merely responding to patterns shared across users. AT compares the target user’s information (the treatment) with four controls: no user information, generic population information, matched-group information, and a matched imposter’s information. We apply AT to next-action prediction on real user trajectories, for a variety of simulators on three datasets: chess playing, knowledge tracing, and online shopping. Across 84 combinations of simulators, datasets, and predicted behaviors, none shows individual fidelity. Existing user simulation benchmarks barely change their scores when an imposter’s information replaces the target user’s. Finally, AT can guide simulator selection: on WildChat, simulators with larger AT gains tend to generate text that moves closer to the user’s real turn when given the target user’s profile instead of an imposter’s. Together, these findings show that apparent behavioral similarity can hide limited individual fidelity, and that AT can help choose the simulators that come closest to it.

Primary area:
datasets and benchmarks
Keywords:
User Simulation, User Modeling, Personalization, Large Language Models, Simulator Evaluation, Individual Fidelity

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_tYNAOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces the concept of individual fidelity, distinguishing it from steering in the context of user behavior simulation. The authors argue that existing work mostly focuses on steering, which measures how much a simulator’s behavior changes in response to supplied user information. In contrast, individual fidelity measures the degree to which a simulator preserves behavioral differences specific to the intended user. To evaluate this, the authors propose Attribution Trials, a method inspired by randomized controlled trials, to test how much the model learns from the specific user’s information. The results show that current user simulators do not exhibit individual fidelity, and that prompting-based simulation is ineffective.

Critical strengths

This paper discusses an important problem and, through very concrete experiments, proves that prompting LLMs does not work.

Critical weaknesses

Novelty of the prompting finding. The observation that prompted LLMs make limited use of user profiles is not new, including on the same dataset. The OPeRA benchmark already found that adding personas to the prompts of off-the-shelf LLMs does not consistently improve exact next-action accuracy, with gains limited to coarser action-type and click-type metrics, and concluded that current models have limited ability to integrate persona information into step-level decisions.

Questions for authors

N/A

1 comment

Thank you for the positive review and for pointing us to OPeRA [1]. Your one weakness credits us with OPeRA's finding, which is not ours; we sincerely hope you will re-evaluate our work in light of this difference.

Moreover, we have released the code of Attribution Trials at https://anonymous.4open.science/r/Attribution-Trials-ECC7/; it contains every experiment in the paper, including those added in this rebuttal, and the download steps for every dataset, and we ensure that all reported results are reproducible.


W1: the prompting finding is not new

Our finding is not OPeRA's. OPeRA compares a persona with no persona, which is our gain over no information. We ask whether a profile that helps does so because it is this user's: on our 22 frozen-prompt combinations, the profile beats no profile in 14, and a matched imposter's profile does almost as well in all 14 (Section 5.1). A persona-versus-none comparison cannot see this: it keeps 100.6% of its score when the imposter's profile replaces the target's (Section 5.2). Nor is the finding specific to prompting: none of the 84 combinations passes, including trained per-user models and released simulators. Revision: Section 5.1 cites OPeRA's finding.


References

[1] Wang et al. OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation. ACL 2026.

Reviewer_kSP7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a controlled auditing method to evaluate whether a simulated user is in fact faithful to the individual it is supposed to represent (individual fidelity), over and above population and group level patterns that makes the user plausible. They propose to test this with attribution trials (AT), which involves measuring the gain in information from the target user’s simulator against 4 control conditions, which include, no user information, population information, matched-group information, and a matched imposter's information. The trails ask whether the user simulator preserves individual fidelity in next action prediction across chess games, knowledge tracing, and online shopping. The study finds that across 84 combinations of simulator, dataset and predicted behaviour, no simulator passes the criteria. They also investigate how the AT can help select better simulators for text generation in Wildchat.

Critical strengths
  1. The paper discusses a useful distinction between steering and individual fidelity in user simulators. The auditing method which defines a treatment and 4 control conditions could serve as a diagnostic tool to distinguish between whether the user simulator is actually representing the individual characteristics it is supposed to be representing.
  2. The authors demonstrate their evaluation method across 3 datasets and also show how their method could potentially be used to select better simulators for text generation in Wildchat.
Critical weaknesses
  1. I am not convinced by the framing and motivation for individual fidelity as it is currently proposed in the paper. The problem of individual specific modelling is not new, and prior work already looks at how to evaluate individual behaviour prediction (e,g., McIlroy-Young et al., 2022). While the paper tries to offer a potentially useful evaluation, there is no significant conceptual insight that it adds beyond what the literature has established. Looking closely at how individual fidelity itself is defined, I find conceptual holes that are not fully addressed. For instance, to establish that some user information indeed contributes to individual fidelity, the method requires significant statistical gains over all 4 controls. However, how much of the behaviour in these controls can explain the individual behavior itself, and therefore act as a confound is not sufficiently explored.
  2. The experiment setup itself seems to be exposed to several confounds that make it hard to take the results at face value. First, of the 84 configurations that were tested, a majority of them seem to be on the OPeRA dataset with only 41 users. Second, across domains, the way the user profiles encode the information is quite different. These are often concise summaries that need not necessarily encode the individual behaviour that the method is in fact testing for. There is no sufficient analysis that discusses how much of the individual user level information presented to the model is affecting its performance. Another area I am confused by is the selection of the matched-group information. It seems that changing the granularity of the matched features on the imposter group substantially affects the results. It is therefore difficult to interpret and formulate what is appropriate to include in a trial based on this.
Questions for authors
  1. How does the novelty of the work compare to existing discussions on individual behaviour modelling found in the literature? IS the primary contribution this work the control conditions?
  2. If the control conditions are a major contribution, what justifies them as being necessary and sufficient conditions to show the absence of individual fidelity if they are not fully specified? What confounds on individual and group level behaviors being similar have authors considered, and how might this affect the interpretation of results?
  3. How does the profile itself being presented to the model affect individual fidelity? Could the authors try different ways of representing the profiles and document the robustness of their attribution protocol?
2 comments

Thank you for the detailed review. The contribution is the matched controls, and they reverse the conclusion of [1], the prior work W1 cites; the new experiments for W4 and W5 leave our conclusion unchanged. We believe our work does have significant value, and thus sincerely hope you will re-evaluate our work in light of these results.

Moreover, we have released the code of Attribution Trials at https://anonymous.4open.science/r/Attribution-Trials-ECC7/; it contains every experiment in the paper, including those added in this rebuttal, and the download steps for every dataset, and we ensure that all reported results are reproducible.


W1 + Q1: novelty relative to individual behavior modelling [1]

Yes, the contribution is the matched controls, and they reverse [1]'s conclusion. Our static-embedding family is [1]'s per-player model (Section 4). Against unmatched controls, [1] concludes that it captures "human decision-making at an individual level". Under the matched imposter, this family never beats no user information, and a per-user base rate matches all 5 of its combinations significant after correction (Section 5.1). An unmatched control cannot tell the target from a similar user: a random unrelated profile [2] still finds 3 of 5 significant when the imposter's profile replaces the target's (Table 1). Revision: Section 2 places [1], [3], [4].


W2 + Q2: necessary and sufficient; group versus individual confounds

Showing absence needs only necessity, and each of the four gains is necessary. A simulator that does not beat one of the four controls is not using what is specific to the user, and every one of the 84 combinations fails at least one. Separating group from individual behavior is what the controls are built for; Section 6 tests it on synthetic users:

Synthetic populationTarget significantly better than
Users differ only by groupno information 85%, population 100%, imposter 3%
Individual share λ = 00 of 200 populations

When users differ only by group, the imposter control fires in 3% of populations, so group behavior is not mistaken for individual behavior.


W3: most combinations are on OPeRA (41 users)

The verdict does not rest on OPeRA. On chess and knowledge tracing alone (18 combinations; 299 and 500 users), none passes, and all 6 combinations significant against the imposter after correction are there (Table 1). On OPeRA, 41 users suffice to detect the profile against no profile (27 of 66) but not against the imposter (0 of 66). Revision: Section 5.1.


W4 + Q3: other profile representations

We ran it: a richer representation does not help. We replaced the summary profile with the user's own recent actions (up to 1,000 tokens):

Frozen prompt, gain over imposterSummaryOwn actions
Qwen3-1.7B, KT response+0.056+0.004
Qwen3-8B, KT response+0.049−0.011
Qwen3-1.7B, chess timing+0.003+0.000
Qwen3-8B, chess timing+0.011+0.001

The model does read it (Qwen3-1.7B on KT: gain over population −0.13 → +0.20), but what it takes is not user-specific. Revision: Appendix F.7.


W5: matching granularity

Yes, by design: the closer the imposter, the smaller the gain, and the stricter the test.

Gain over imposterRandom userSame third (ours)Same ninthNearest neighbor
Static, chess timing+0.071*+0.062*+0.055*+0.005*
Static, KT response+0.053*+0.011*+0.006*+0.002
Recurrent, KT response+0.046*+0.016*+0.013*+0.006
Frozen 8B, KT response+0.070*+0.049*+0.031*+0.024

* significant; all 11 combinations in Appendix F.8. Once lost, significance never returns at a finer level. Rule for a trial: match on the attribute that drives behavior (rating, accuracy, spend), fix it in advance, and report a finer matcher as a stricter bound. Revision: Section 3.1, Appendix F.8. Running this, we found that the nearest-neighbor matcher for KT timing read a constant column; corrected, 2 rather than 3 of 6 stay significant (Section 6, Table 4, F.3), and the verdict is unchanged.


References

[1] McIlroy-Young et al. Learning Models of Individual Behavior in Chess. KDD 2022.

[2] Yang et al. BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces. arXiv:2606.02798, 2026.

[3] Piech et al. Deep Knowledge Tracing. NeurIPS 2015.

[4] Tan et al. Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning. EMNLP 2024.

Thank you to the authors for responding to my comments and running new experiments. However, I remain unconvinced that the paper, in its present state, presents the conceptual novelty and methodological rigour required to meet the bar for ICLR. I therefore intend to maintain my score.

Reviewer_LBdsOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Attribution Trails, an approach to measure individual fidelity of user simulators, defined as the ability to capture behaviors unique to an individual user, instead of a general population of users. The paper evaluates a range of LLM-based simulators on multiple tasks using their approach.

Critical strengths

I believe the goal of being able to simulate specific users instead of a general population very important and a good motivation, but I find many issues with the execution of this idea in this paper, which are summarized in the weaknesses section below.

Critical weaknesses
  • The approach seems to be just based on changes in surprisal values by the simulator at next token prediction when conditioning for a specific user profile compared to the general population. This is a bit overly simplistic, and doesn't give any further insights into whether the simulator is capturing any specific behaviors. I think the metric may also be confused by the authors with perplexity since it is at the level of the dataset, (instead of surprisal), so this needs to be revisited.

  • The tasks are not defined clearly or given any particular motivation/background, so it is difficult to make sense of the presented results. Further, I find the presentation of the results to be unclear. For example, Figure 2 is apparently showing the results of 84 models on 3 different tasks, with multiple user configurations, but it is not clear what the takeaways from those results are, or what are the differences between models, despite carefully looking at the plot. Overall, I do find a lot of clarity issues with the writing of the paper and presentation of the results.

Questions for authors

No particular questions, please see weaknesses section for feedback.

1 comment

Thank you for the review. W1 and W2 describe a method that is not ours, and the revision fixes W3 and W4. As the main concerns rest on W1 and W2, we sincerely hope you will re-evaluate our work in light of these clarifications.

Moreover, we have released the code of Attribution Trials at https://anonymous.4open.science/r/Attribution-Trials-ECC7/; it contains every experiment in the paper, including those added in this rebuttal, and the download steps for every dataset, and we ensure that all reported results are reproducible.


W1: surprisal at next-token prediction, compared with the population

This is factually incorrect on both counts. Section 3 defines a simulator that "predicts a given user uu's next action": a chess move, a knowledge-tracing answer or a shopping action, each with the time before it as a continuous duration (Section 4). Three audited families are not language models at all. The population is one of four controls, and the controls give the insight you ask for: where user information helps, generic population information recovers a median 97% of the gain (Section 5.1), so the gain is generic, not specific to the user. Revision: Section 3 opening.


W2: confusion with dataset-level perplexity

There is no confusion: the metric is per-user log-perplexity, and the paper shows that dataset-level perplexity fails. Equation 1 averages surprisal over one user's held-out actions, which for discrete actions is log⁡PPLu\log \mathrm{PPL}_u. Equation 2 takes the difference per user before averaging, so each gain is the mean over users of log⁡(PPLu(control)/PPLu(target))\log(\mathrm{PPL}_u(\text{control}) / \mathrm{PPL}_u(\text{target})). Dataset-level perplexity changes by a median 0.1% when the imposter's information replaces the target's (Section 5.2). Revision: Section 3.2 names this link.


W3: tasks not defined or motivated

Agreed, and fixed. Section 4 now has Table R1: dataset, users, the two behavior channels, what the target's profile states, and why each dataset carries individual differences (skill and time use in chess, prior knowledge and speed in knowledge tracing, browsing and pace in shopping).


W4: takeaway of Figure 2 and differences between models

Agreed, and fixed. The takeaway: no combination is significant on both axes of all three panels, so none shows individual fidelity. By method family (Table R2):

FamilyCombinationsBeats no information: uncorrected (after correction)Beats imposter: uncorrected (after correction)
Static and recurrent embedding906 (6)
Structured memory100
User-profile LoRA1604 (0)
Frozen prompt2214 (12)2 (0)
Released simulators3615 (12)0

Per-user embeddings beat the imposter but never no information; prompted and released language models beat no information but not the imposter. No family does both. Revision: Figure 2 caption, Table R2 in Section 5.1.

Reviewer_9RXXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper noticed the limitation of existing user simulator evaluations: they focus on if simulated behaviors are changed or steered to wards a more realistic direction at a population level, rather than individual fidelity that preserves the behavioral differences unique to the intended target user. The author designed the controlled RCT-style audit (AT) against four progressively tighter controls from no information to matched imposters, with 84 baselines from user simulation literatures. Shockingly, no one passed the AT, making us questioning the alignment fundamentals of user simulation research (e.g., what prior about the user should be used in training, what audit metrics and what level of alignment should be evaluated, etc).

Critical strengths

The author noticed the difference of user simulation behaviors between Steering and Individual Fidelity and proposed AT. All these are novel and inspiring. The findings that no existing user simulation baseline can stably pass AT is also surprising, making this paper a good material to inspire more future works in user simulation research.

The AT experiments are also very comprehensive -- 84 combinations of building user simulators are evaluated as baselines, covering 3 different cases (Chess games, knowledge tracing, and online shopping).

The design of AT has a good scientific motivation: follows and motivates by controlled RCT-style audit and implemented against four progressively tighter controls rather than a set of isolated dimensions.

Critical weaknesses

I am concerned about the robustness of the '0 of 84 pass' conclusion, because the trained baselines use fixed, unregularized training recipes with no validation split, hyperparameter tuning, or early stopping (e.g., training for a fixed 15 epochs at lr = 0.01 with latent dimension 16 and no weight decay). E.g., for static embeddings, fitting 16 free parameters per user on a limited 70% history without any regularization may easily overfits relative to a pooled model.

Moreover, my major concerns are about the findings in Sec 5.3. For example, Section 5.3 shows cosine-distance metric favors their AT. However, it could be due to the identical "Target − Imposter "contrast structure. The "ground-truth" target metric in Sec 5.3 (Content Gain) follows the same structure as the paired difference between the exact same target and imposter profiles used in Δindiv\Delta indiv. Instead, most other selection rules in Table 3 (Absolute similarity, Population distance, Judged fidelity) do not have the imposter baseline. So naturally, a selector built as a form of "𝑓(𝐼_imposter) −𝑓(𝐼_target)" aligns naturally better with an evaluation target built as "ℎ(𝐼_imposter) − ℎ(𝐼_target)​" on the exact same imposter pairs.

Also They did not test $ \Delta pop$ or Δgroup\Delta group in Section 5.3. Would they work just as well as $ \Delta indiv$? If a simulator treats any generic profile similarly, i.e, S(𝐼pop)∼𝑆(𝐼group)∼S(𝐼imposter)S(𝐼 pop) \sim 𝑆(𝐼 group) \sim S(𝐼 imposter) as Section 5.1 showed, then subtracting 𝑆(𝐼target)​𝑆(𝐼_target)​ gives the similar Δ\Delta too, meaning a simple population or group baseline could rank simulators just as well without needing matched imposters.

Questions for authors

There seems a contradiction between Sec 5.1 (No LLM has fidelity) and Sec 5.3 (All 7 LLMs separate target from imposter). So is the bottleneck really the simulator architecture, or is it data/task modality or profile richness (e.g., 6 numeric stats in Chess/Shopping vs. 1200 tokens of raw language in WildChat)?

2 comments

Thank you for the careful review. We ran every test you proposed, and none changes the conclusion. As each soundness concern is now tested and the results hold, we sincerely hope you will re-evaluate our work.

Moreover, we have released the code of Attribution Trials at https://anonymous.4open.science/r/Attribution-Trials-ECC7/; it contains every experiment in the paper, including those added in this rebuttal, and the download steps for every dataset, and we ensure that all reported results are reproducible.


W1: fixed, untuned recipe

Tuning does not change the verdict: 0 of 9 pass, as before. The recipe was also regularized: AdamW with weight decay 0.01, which Appendix B failed to state. We tuned the 9 combinations with per-user parameters (static and recurrent embedding): validation split, early stopping, and a grid over weight decay, latent dimension and learning rate.

9 combinationsOriginalTuned
Pass (all four gains after correction)00
Target better than imposter after correction64

A better fit removes target–imposter separation rather than creating it: on chess moves, surprisal falls from 2.931 to 2.750 and the gain over the imposter from +0.018 to +0.001. Revision: Appendix B.3, F.6.


W2: selector and endpoint share target–imposter pairs

The shared pairs raise τ but do not produce the result: with an independently drawn imposter, the trial still picks HumanLike-7B, the simulator with the largest content gain. τ falls from +0.43 to +0.24 (avg@8) and from +0.62 to +0.43 (best@8), still above the profile-versus-no-profile rule (+0.14, +0.33). Revision: Table 3 row; Section 5.3 now claims the top pick and a positive correlation.


W3: would the population or group gain select as well?

No. We ran both, and each picks a worse simulator.

Gain used to selectτ avg@8τ best@8Top pick (content-gain rank)
Over the imposter (ours)+0.43+0.62HumanLike-7B (1st)
Over the group+0.43+0.43Qwen3-8B (3rd)
Over the population+0.14+0.33HumanLM-opinion (6th)

The premise does not hold on WildChat: for HumanLike-7B the gains over population, group and imposter are +0.66, +0.54 and +0.97, not similar. Revision: Table 3 rows.


Q1: what is the bottleneck?

There is no contradiction: Sections 5.1 and 5.3 test different tasks, and the bottleneck is the task, not the architecture. On WildChat, with population and group profiles added, all seven simulators pass all four gains (Holm over 28; weakest +0.14 [+0.02, +0.25]); five of the same simulators fail on next actions in Section 5.1. Profile richness is not the cause: with the user's own recent actions (up to 1,000 tokens) in place of the summary, the target beats the imposter in 0 of 4 frozen-prompt combinations (Qwen3-1.7B, KT: +0.056 → +0.004). Revision: Section 5.3, Appendix E.2, F.7.

Thank you for provide the additional experiments. My main concern on the identical "Target − Imposter "contrast structure in development and evaluation still holds. I lean to maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.