Is There an Imposter Among Us? Auditing User Simulators with Attribution Trials

User simulators are commonly evaluated by their ability to steer toward the target user’s behavior, given the user’s profile or history. However, this similarity does not show whether the simulator captures what makes that individual unique, rather than behavior shared by many users. In interactive systems, this can lead to inaccurate simulations of users and thus, unreliable results. We identify this as the simulators’ lack of individual fidelity – the ability to capture behavior unique to the intended user. We therefore develop Attribution Trials (AT), a controlled audit for testing whether a simulator represents that individual rather than merely responding to patterns shared across users. AT compares the target user’s information (the treatment) with four controls: no user information, generic population information, matched-group information, and a matched imposter’s information. We apply AT to next-action prediction on real user trajectories, for a variety of simulators on three datasets: chess playing, knowledge tracing, and online shopping. Across 84 combinations of simulators, datasets, and predicted behaviors, none shows individual fidelity. Existing user simulation benchmarks barely change their scores when an imposter’s information replaces the target user’s. Finally, AT can guide simulator selection: on WildChat, simulators with larger AT gains tend to generate text that moves closer to the user’s real turn when given the target user’s profile instead of an imposter’s. Together, these findings show that apparent behavioral similarity can hide limited individual fidelity, and that AT can help choose the simulators that come closest to it.