acceptodds
Under review as a conference paper at ICLR 2027

Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents

30 views

Abstract

Large Language Model (LLM) agents are increasingly deployed in settings where they interact with diverse users, including those who are unclear, impatient, or reluctant to share information. However, collecting real interaction data at scale remains expensive. The field has turned to LLM-based user simulators as stand-ins, but these simulators inherit the behavior of their underlying models: cooperative and homogeneous. As a result, agents that appear strong in simulation often fail in real human interactions. To narrow this gap, we introduce Persona Policies (PPol), a plug-and-play control layer that induces realistic behavioral variation in user simulators while preserving original task goals. Rather than hand-crafting personas, we employ an evolutionary coding agent to discover persona generation programs optimized for human-likeness and behavioral coverage over real user conversations. The evolved program generates diverse, human-like personas for any task in the domain. Across 4 benchmarks—including -bench Retail and Airline, ColBench, and WildChat—evolved PPol yield 28-72% absolute gains in fitness score over the baseline simulator. In blinded evaluations, annotators judged PPol users as 'human' 80.4% of the time, nearly 2x more than the baseline simulators. Training agents with PPol also improves real-world performance: our user study with live human–agent interactions showed that fine-tuning with our method boosted task success by % over default baselines. PPol thus offers a novel approach to strengthen simulator-based evaluation and training without changing underlying tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
User Simulators, LLM Agents, Persona Generation, LLM Evaluation, Large Language Models, Human-AI Interaction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_iL8bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PPol, a method that generates and iteratively optimizes personas to control user simulators. It uses behavioral features from real user conversations to calibrate simulated interactions and evaluates whether the resulting data improves agents’ ability to handle diverse user behaviors. However, persona injection is already a common control strategy for user simulators in industry. The paper’s contribution therefore depends on whether its personas better reflect real users, whether simulators consistently follow them, and whether they deliver greater downstream benefits than alternative persona methods. The experiments provide some encouraging evidence, but do not fully establish these three points.

Critical strengths

The paper connects user simulation to downstream agent performance. Beyond presenting generated personas, it incorporates behavioral features from real conversations into optimization and conducts both agent-training experiments and human interaction studies. This extends the evaluation beyond the quality of generated text and provides preliminary evidence of practical utility.

Critical weaknesses
  1. The realism of generated personas remains insufficiently validated. LLM-generated personas may reflect the model’s assumptions about how people behave rather than actual human preferences and reactions. Behavioral statistics and human judgments provide some supporting evidence, but do not sufficiently establish alignment with real users in concrete decisions, preference trade-offs, and context-dependent responses.

  2. Injecting a persona does not guarantee behavioral adherence. The method uses prompting and interaction feedback to guide behavior, but lacks an independent, systematic evaluation of persona adherence. It remains unclear whether specified behaviors appear when relevant and remain consistent throughout extended interactions.

  3. The downstream comparisons do not sufficiently establish the specific advantage of PPol. The paper compares training on base-simulator data with training on data augmented by PPol, but does not include equivalent downstream training comparisons using simple persona prompting, comparable persona-generation methods, or relevant state-of-the-art approaches. Consequently, the results do not adequately distinguish the benefits of introducing persona diversity from those of PPol’s specific optimization mechanism.

  4. Downstream generalization is evaluated primarily within the same task environments. Although training and evaluation tasks are separated, both largely remain within the Retail and Airline environments of τ², while the challenging evaluations primarily vary user behavior. These results do not establish transfer to new task types, tools, or domains, and cannot rule out gains that depend on adaptation to the benchmark’s particular distribution.

Questions for authors
  1. How can the authors establish that the generated personas better represent real users, beyond matching behavioral statistics? Could real and simulated users be evaluated under identical scenarios, comparing their preferences, decisions, and reactions, with simple persona-generation methods included as baselines?

  2. How is actual persona adherence verified? Could the authors measure whether specified behaviors occur under relevant conditions and whether they remain consistent over long conversations?

  3. Does PPol provide greater downstream training benefits than alternative persona methods? Could the authors compare agents trained using personas produced by simple prompting, comparable generation methods, and relevant state-of-the-art approaches against those trained using PPol, while controlling for training tasks, data volume, and training budget?

  4. Do the gains transfer beyond the training distribution? Could the authors evaluate unseen task types, tools, or domains, and clarify how tasks in the human evaluation are separated from those used for training?

7 comments
AuthorsOpenReview ↗

Addressing W1, Q1

Thank you for sharing your constructive feedback on our paper. We appreciate you recognizing that our work goes beyond evaluating text quality by directly linking user simulation to downstream agent performance through behavioral features, agent-training experiments, and human interaction studies. We address your concerns below:


W1. First, we would like to emphasize that making simulated users communicate more like real humans is our central focus. PPol's optimization objective explicitly rewards persona-generation programs for human-likeness and behavioral coverage, both measured against real conversations from the target domain. Our results provide evidence that goes beyond surface statistics and covers several independent views of human-likeness:

  • With reference to the target domain's human dialogs, PPol outperforms baselines on human-likeness across every dataset (Table 1, Tables 3–4).
  • PPol covers a much broader human distribution (79% overlap vs. 56% for the base simulator; Figure 4, Table 7).
  • Human annotators identified PPol users as 'human' about 2× as often as baseline users (Figure 5).
  • Agents trained with PPol perform better with real people (Table 2).

We agree that it is interesting to look at it from another lens. To answer Q1, we test whether simulated users make the same decisions and reactions as real people when placed in the identical situation, that is, same task, same conversation so far, same agent message.


  • Real users. We use two independent sets of real human-agent conversations on 60 τ2\tau^2-bench test tasks, neither of which was used to build or optimize PPol: (1) our live user study and (2) the τ2\tau^2-bench human conversations.

  • Same conversation state. From each real conversation we pick up to 5 random user turns, giving 1,706 points in total. At each point, every simulated user receives the exact conversation up to the agent's last message and writes the next user message. We then compare it with what the real person wrote at that moment.

  • Simulated users. At each point, we generate the next message with 10 simulated users from each of three methods: PPol (Evolved), PPol:Initial (the seed generator before evolution), and DP Personas (personas from directly prompting an LLM). We repeat this with two user LLMs: Qwen3-Next-80B and GPT-5.4-Mini.

  • We describe each message with the paper's behavioral features, grouped by the dimensions in Table 1:
    • Decisions (D3): pushing back or refusing, asking clarifying questions, seeking information, and committing with certainty vs. hedging.
    • Reactions when things go wrong (D4): expressing emotion, blaming the agent, or switching strategy.
    • What the user reveals (D2): sharing identifying information (IDs, emails, order numbers).

  • Comparison Metric. A simulated message matches the real one on a dimension if it shows the same behaviors (e.g., both push back, or neither does). We report the share of simulated messages that match, averaged over tasks. To test whether PPol matches more often than a baseline, we use a paired Wilcoxon test over tasks (significant at p < 0.05). With 2 baselines and 2 user LLMs, that gives 4 comparisons per dimension.

Table: Share of simulated next messages (%) that match the real user's next message (higher is better; range over two user LLMs)

PPol:EvolvedPPol:InitialDP personasPPol significantly better
Decisions85.0–85.873.2–79.673.8–80.64 / 4 comparisons (all p < 10⁻⁶)
Reactions when things go wrong94.6–95.592.2–93.387.8–90.24 / 4 comparisons (all p < 10⁻⁴)
What the user reveals83.6–86.583.2–83.581.1–82.12 / 4 comparisons (highest in all 4)

Findings: Given the exact same conversation, PPol's next message makes the same decision as the real user 85–86% of the time, compared with 73–81% for the baselines, and reacts like them more often when things go wrong (95% vs. 88–93%); both gaps are significant in all 4 comparisons, and the decision gap holds for each set of real users separately.

Overall, PPol's realism extends to how users decide and react in the same situation. We will add this analysis to the paper with example conversations, and hope that it makes the paper's central claim stronger.

AuthorsOpenReview ↗

Addressing W2, Q2

W2. We agree that it is important to consider how faithfully the PPol assigned persona produces the corresponding behaviours in its trajectory.

First, we would note that persona adherence is intentionally built into how PPol is optimized. Our objective is computed on the realized behavior of the persona policy. Both the human-likeness score and the coverage metric are measured from behavioral fingerprints extracted from the actual generated rollouts (Section 3.3). A persona whose specified behavior never surfaces therefore scores poorly (on combined score) and is selected against during the program evolution. So the method rewards personas precisely when the intended behavior appears in the dialogue.

To directly verify that PPol personas exhibit the assigned behaviours, we perform a new experiment with two independent LLM judges (GPT-5.4-mini and LLama-3.3-70b) that read the user turns of a conversation (tau-bench Retail setting from Table 1) and decide whether the behavior is followed as per the assigned behavioral axis. The judge is grounded in the persona's own roleplay instructions but is agnostic to the 19-feature fingerprint. As a control against bias, we also tested whether the judges find these behaviors in a cooperative base-simulator dialogue for the same task.

  • Across all evolved personas, the models successfully show their assigned behaviors 87% of the time, compared to 22% for the baseline control (noting that the baseline isn't zero, as standard simulators naturally exhibit some basic conversational realism). Both judges agreed 77% of the time, showing moderate-to-strong inter-judge reliability.
  • The behaviour gap to the baseline is largest on the friction behaviours that default simulators lack (e.g., incremental disclosure 0.83 vs 0.12). This confirms the intended behaviours are revealed in the dialogue.

We will add this behavioural faithfulness evaluation. It also supports our motivation to use PPol to train agents against diverse user behaviours.

AuthorsOpenReview ↗

Addressing W3, Q3

W3. Thank you for suggesting this. We added two additional baselines to distinguish the downstream improvements of training with PPol that go beyond simply introducing persona diversity

  1. Direct Prompting (DP): LLM is directly prompted to generate 10 diverse personas for each task (Sec 4: Baselines).
  2. PersonaHub: We sample 10 profiles per task from the PersonaHub (Ge et al., 2025) persona collection (a widely used repository of synthetic personas) and use an LLM (Gemini-3-Flash) to convert them into task-specific user instructions.

Both methods use the same persona-generation model (Gemini-3-Flash), rollout agent (Gemma-4-31B), user simulator (Qwen3-Next-80B), and training tasks as PPol. We collect successful trajectories and match the number of training examples used in the original PPol experiments. We use the same nominal LoRA hyperparameters and training steps (48 for both).

We evaluate the resulting agents on the τ2\tau^2-bench test split, using the default simulator and four challenging tau τ\tau-trait simulators (Sec 5.5) featuring the prompts from He et al. (2026). Results below show the Mean Success Rate (%) below:

DomainTraining personasDefault-userChallenging-user
RetailDP Personas70.020.6
PersonaHub57.516.3
PPol75.025.0
AirlineDP Personas65.038.8
PersonaHub60.037.5
PPol65.041.3

PPol achieves the high default-user success rate in both domains, outperforming DP Personas by 5pp in Retail, and PersonaHub by 17.5pp and 5pp, respectively. These results also hold across the challenging user case. We further examined the generated trajectories to understand these differences. In Retail, PersonaHub-trained agents sometimes generated fabricated tool responses instead of making actual tool calls (~20% of tasks across evaluation settings).

These findings suggest that generating diverse personas alone may not produce equally useful training data. The optimization for PPol in producing realistic, task-relevant user behaviors helps agents handle challenging interactions while avoiding these failure modes. We will include these baselines in our discussion of the benefits of PPol-based training. We would also be happy to run any additional state-of-the-art baselines you believe would strengthen the differentiation of our work.

Reviewer_x9DXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework called PPol for generating human-like personas for LLM-based user simulation. It uses evolutionary search to optimize persona generation based on human-likeness grounded with real human user conversations. Experiments on multiple benchmarks show improvements over baseline simulators. They also conduct human evaluations and show that agents trained with conversations generated by Ppol has better performance when interacting with human users.

Critical strengths
  1. Using evolutionary can potentially generate more diverse and realistic user behaviors.
  2. The paper evaluates across multiple benchmarks and model backbones.
  3. The human evaluation and downstream training experiments shows that the generated personas can improve agent robustness.
Critical weaknesses
  1. It is unclear whether the personas generated with Ppol preserve the original user goals and constraints. The framework optimizes human-likeness and behavioral coverage, but does not directly evaluate task fidelity. Moreoever, some examples in Appendix M prompt the simulator to forget certain requirements (or provide incorrect information,, e.g., an incorrect email address). In such cases, the simulated users may no longer follow the original tasks.
  2. The human study only involves 36 participants. It is unclear whether the reported improvement (29.5% to 36.4%) is statistically significant, given that the same participants completed multiple interactions.
Questions for authors
  1. How often do personas genrated with PPol comply with/violate the original task goals?
4 comments
AuthorsOpenReview ↗

Addressing W1, Q1

Thank you for writing a thoughtful review of our paper and recognizing the core innovation of our method, including the diversity and realism of personas tested through multiple experiments across benchmarks and models. We appreciate that you noted the human evaluation and training experiments as a critical strength of our work. We address your concerns below:


W1, Q1: Thank you for highlighting this. We agree that maintaining task fidelity is an important requirement of evaluating agents with PPol. Based on your suggestion, we conducted an additional evaluation to measure whether PPol personas preserve the original task goals and constraints.

For all τ2\tau^2-bench tasks (Retail setting from Table 1), we extracted the benchmark's ground-truth specifications, covering user identity, required actions, and information the user must request from the agent. We used 2 independent LLM judges (GPT-5.4-mini and Llama-3.3-70B) to evaluate 800 PPol and default simulator (as a control) conversations. Each judge assessed whether the user's behavior was consistent with each requirement, using only the conversation transcript without access to the persona instructions. Judges also provided supporting excerpts from the dialogue.

  • Table values denote: Requirements preserved (%) | Judge: | GPT-5.4-mini | Llama-3.3-70B | | :--- | :--- | :--- | | PPol | 91% | 94% | | Default | 90% | 92% |

PPol preserves 91-94% of evaluated requirements, compared to 90-92% for the default simulator, with overlapping task-level 95% bootstrap confidence intervals. Hence, we do not observe a reduction in task fidelity relative to the default simulator. The judges agreed on 85% of requirement-level assessments (Cohen's κ = 0.69).

In Appendix M, instructions introduce friction but don't aim to change the user's goals. We find that personas instructed to forget or mistype information still preserve 92-98% of task requirements, as they subsequently end up correcting the information they provided. These results suggest that PPol introduces behavioral variation without compromising task fidelity. We will add this analysis to the paper and believe that it will strengthen the results.

AuthorsOpenReview ↗

Addressing W2

W2. Thank you for asking about the live user study. We would like to clarify the design choice and significance.

When comparing two agents (one trained on PPol personas versus the default simulator) using human interactions, there are primarily two confounding factors that can potentially skew results: participant capability and task difficulty. We rule out these concerns through a rigorous experimental design:

  • No Repeated Tasks (preventing learning bias): Participants never perform the same task twice (e.g., interacting with Agent 1 and then repeating it with Agent 2), which would introduce a learning bias since they already know the task. Instead, each participant completes 10 distinct tasks.
  • Blind Evaluation: Participants never know which agent they are interacting with during any given session.
  • Balanced Participant & Task Distribution: Each task is evaluated independently by multiple participants (an average of 10.8 distinct participants per task across the baseline and PPol conditions).

Once these factors are balanced, any remaining performance difference can be attributed directly to the agent itself. Having each participant complete multiple interactions is both a practical necessity (recruiting suitable, high-quality participants is expensive) and our method’s strength, as it allows a single person to contribute cleanly to both sides of the comparison.

We also scaled our live user study for better testing. The study is now 46 participants that completed 330 chats (162 baseline / 168 PPol). Our findings show that the improvements still hold:

MetricBase-trainedPPo-trainedChange
Task success24.8%30.0%+21%
Perceived completion59.6%63.7%+7%
Rated easy43.4%44.5%+3%
Unwanted agent actions19.7%10.4%−47%
Tool calls / chat7.66.0−21%
Turns / chat13.214.4+9%

Since individual participants completed multiple chats, standard statistical tests would improperly treat those sessions as independent and overstate our sample size. To account for this, we used a cluster bootstrap at the participant level (Cameron et al., 2007), which resamples entire participants rather than individual interactions to preserve the within-user correlation. We also paired agent performance within each task before averaging, ensuring that variations in task difficulty never skew the aggregate metrics. Under this rigorous evaluation, gains in task success are significant ($p < 0.05$, $n = 46$), in line with the gain we originally reported. We will include the updated numbers in the paper and add examples that illustrate the comparison.


Ref.: Cameron, A. Colin and Gelbach, Jonah B. and Miller, Douglas L., Bootstrap-Based Improvements for Inference with Clustered Errors (2007)

Dear reviewer,

Thank you for taking the time to provide your feedback. We have responded to your suggestions above. As the discussion phase is ending soon, we would love to hear whether we addressed your concerns or if there is something more we can do to strengthen the paper.

Sincerely,

Authors

Reviewer_x9DXOpenReview ↗

Response to rebuttal

Thank you for the additional experiments and clarification. Overall, the additional evidence addresses most of my concerns. I will raise my rating from 2 (weak rejection) to 3 (weak acceptance). Please include these additional results in the revised paper.

Reviewer_ZekSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors tackle the challenging problem of developing user simulators with LLMs. Collecting real user interactions can be expensive or impractical. Instead, emulating users with LLMs is an emerging direction. The problem is that any simulated user powered by an LLM also inherits the cooperative and homogenous qualities of the LLM, again creating a performance gap between an agents trained on LLM-simulated users and how the agent performs in the real world. If user simulators can be even more closely aligned with real-world users, in theory this performance gap should close. The authors present Persona Policies, a layer for introducing realistic behaviors in LLM simulators. It includes an evolutionary coding agent that discovers persona generating programs, generating human-like personas for any domain. Experiments show that Persona Policies generate more human-like interactions and training agents with Persona Policies produces agents that perform better.

Critical strengths

The idea of evolved personas via evolutionary program search seems novel - it goes beyond clever prompting and seems to generate unique, realistic personas at scale.

The experiment results validate the approach - compared to other simulator approaches, Persona Policies generated personas score highest on human-qualities. When using PPol for generating SFT training data, models trained on this data performed better in live human interactions.

The results include human evaluation as well, and the results show that even humans cannot tell the difference between Persona Policies generated interactions and real human interactions.

Critical weaknesses

None

Questions for authors

N/A

2 comments

Thank you for writing a positive review of our paper! We appreciate that you recognized various strengths of our method, including (1) the novel idea of using evolutionary program search for generating personas, (2) an extensive set of experiments validating our approach, and (3) live human interaction studies that confirm the realism of our personas and the benefits of using PPol as training data.

Highlighting some of the key contributions of our work –

  • To the best of our knowledge, PPol is the first method to ground user simulation in real human conversations from the target domain and jointly optimize for diversity and human-likeness through evolutionary program search.
  • We validate PPol across multi-domain benchmarks and models. Our ablation studies show that program evolution leads to improved personas, and our approach is robust to alternative feature spaces and initialization configs.
  • We conducted user studies to confirm the realism of PPol personas and the practical advantage of using it for training agents, both showing favorable results.

Our main objective is to help the agentic AI community move past overly cooperative LLM users and toward simulators that reflect how a wide range of real people behave.

If there are any remaining concerns you have, we would be happy to provide further clarification or evidence to address them. We hope this discussion can help strengthen your recommendation for our paper.

Dear reviewer,

Thank you for taking the time to provide your feedback! As the discussion phase is ending soon, we would love to know whether you have any remaining concerns that we can address to strengthen the paper. We look forward to hearing from you.

Sincerely,

Authors

Comments on the paper

1 comment

Dear reviewers and AC,

Thank you for sharing your feedback on our paper! We are encouraged that all reviewers found PPol to be a clear, well-evaluated approach that moves user simulation beyond cooperative, homogeneous LLM users toward realistic human behavior. Specifically, they appreciated:

1. Novel method (ZekS, x9DX): evolutionary program search for persona generation “goes beyond clever prompting” and produces “unique, realistic personas at scale”.

2. Grounding in real users (x9DX, iL8b): behavioral features from real human conversations are built into the optimization objective.

3. Broad evaluation (ZekS, x9DX, iL8b): experiments span multiple benchmarks and models, go beyond text quality to show usefulness as training data for agents.

4. Multiple human studies (ZekS, x9DX, iL8b): annotators found that PPol dialogs look almost the same as real humans. Live user studies also show our method’s practical utility.

5. Real downstream value (ZekS, x9DX, iL8b): agents trained with PPol perform better in live interactions with real people, which ties simulation directly to agent performance.

6. Presentation, Informativeness, Soundness (ZekS, x9DX, iL8b): We are glad that all reviewers found our paper clear and easy to follow. Our approach was also rated as interesting and useful by everyone, alongside no major concerns regarding its soundness.


Based on the reviewers’ feedback, our rebuttal presents the following findings:

1. PPol makes decisions similar to real users (iL8b): We gave simulated and real users the identical task and conversation so far (1,706 points, 60 held-out tasks). PPol's next message makes the same decision as the real user 85–86% of the time. It also reacts more like the real user when things go wrong. Our gains over baselines are statistically significant. We added this in Sec 5.4 and Appendix K.

2. Task fidelity and persona adherence (x9DX, iL8b): PPol preserves 91–94% of the ground-truth task requirements, on par with the default simulator. Personas show their assigned behaviors 87% of the time, compared with 22% for a cooperative baseline. We added details in Sec 5.4 and Appendices L,M.

3. Advantage over other persona methods and transfer (iL8b): With matched data and training budget, agents trained on PPol beat agents trained on Direct-Prompting and PersonaHub personas, with both default and challenging users. A program evolved only on Retail also transfers to the Airline domain (human-likeness 0.92 vs. <=0.47 for the baselines). We updated Section 5.5 and Appendix I.

4. A larger, statistically tested user study (x9DX): We expanded the study to 46 participants and 330 chats. Task success rises by +21% relatively (p<.05), and unwanted agent actions fall by 47%. Updated Section 5.5 and Appendix J.2.

We have added the above analyses and results to our paper (please refer to blue-colored text in the revised PDF).

We were able to address the concerns of Reviewer x9DX, and they decided to raise their score. Reviewer iL8b also acknowledged our rebuttal and confirmed that they have no further questions. Reviewer ZekS expressed no concerns, and we hope that this discussion period has strengthened their positive review of our work. Thanks a lot again.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.