acceptodds
Under review as a conference paper at ICLR 2027

Uncovering Robustness Weaknesses in LLM Agents via Persona Evolution with Task-Integrity Constraints

Abstract

Interactive-agent evaluations commonly rely on standardized LLM-based user simulators that exhibit broadly cooperative behavior, potentially overlooking failures that arise under diverse but realistic user interactions. This can lead to overly optimistic estimates of agent robustness and provide limited failure signals for training agents that generalize across users. Existing approaches largely rely on predefined user behaviors, which may miss agent- or domain-specific weaknesses, while failure-oriented search can reduce task success by invalidating the underlying task rather than exposing genuine robustness failures. To fill this gap, we introduce Tiper, a task-integrity-preserving persona evolution method that automatically discovers agent-specific user policies exposing robustness weaknesses while keeping the underlying task valid and solvable. Tiper evolves persistent user personas through reflection-guided behavioral mutations and task-wise Pareto selection under a task-integrity constraint. Across four interactive-agent benchmarks, Tiper yields lower held-out avg@3 scores in all 12 GPT-5.4 settings, with observed reductions of 5.8-33.7 percentage points, and in all 28 agent–domain pairs spanning seven agents, with a median observed reduction of 22.9 points. Further analysis yields a behavior-to-failure taxonomy that connects discovered user behaviors to recurring agent failure modes. Together, these results show that Tiper enables more diagnostic robustness evaluation of interactive agents and provides informative failure analysis for future agent robustness improvement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, interactive agent evaluation, user simulation, automated stress testing, persona evolution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9ghKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TIPER, a framework that evolves user personas to expose robustness failures in interactive LLM agents while preserving task integrity. It combines reflection-guided persona mutation, trajectory-level integrity checks, and task-wise Pareto selection. Experiments across four benchmarks and seven agent models show consistent reductions in task success compared to default user simulators, along with analyses connecting challenging user behaviors to specific agent failure modes.

Critical strengths
  • I find the distinction between agent failure and task invalidation particularly important. Simply making users more adversarial can trivially lower task success without revealing meaningful weaknesses. TIPER explicitly addresses this, making its robustness evaluation more convincing.
  • The empirical evidence is compelling. The consistent failure exposure across different agents and domains, together with comparisons against predefined behaviors and objective-matched persona optimization, supports the practical value of adaptive persona discovery.
Critical weaknesses
  • I'm not fully convinced that the task-wise Pareto mechanism is necessary. The final objective is still to select a single persona with the lowest average score, and Table 4 does not isolate the contribution of this Pareto selection. It also remains unclear whether the gains come primarily from reflection-based search and integrity filtering rather than maintaining a diverse Pareto frontier.
  • The relationship between task integrity and evaluation reliability deserves more attention. The search-time gate measures integrity specifically among failed trajectories, while the formal objective concerns overall integrity. Since the final scores also condition on valid trajectories, I would like to understand whether this filtering changes which successes and failures contribute to the reported performance gap.
Questions for authors
  • How would TIPER perform if task-wise Pareto selection were replaced with simple top-K selection by average rank-set performance? Would the discovered failure modes become less diverse?
  • Have you examined cross-agent transfer of discovered personas? For example, would a persona optimized against GPT-5.4 expose similar failures in other agents? This would help distinguish genuinely agent-specific weaknesses from universally difficult user behaviors.
  • Could you report overall trajectory-integrity rates, including successful rollouts, and compare the results on a common set of integrity-valid trajectories?
3 comments

Thank you for the thoughtful review. To address your concerns, we ran the following experiments: (1) replacing Pareto selection with top-K selection and measuring the diversity of what each discovers, (2) evaluating the personas searched against GPT-5.4 on other agents, and (3) auditing all held-out trajectories, including successes.

[W1 & Q1] Task-wise Pareto selection vs. top-K.

We compared Pareto selection with top-K selection on VitaBench Delivery, together with an extra ablation of the reflection module, keeping all other settings fixed, including the search budget:

  • Top-K: replaces Pareto selection; the active set keeps the K personas with the best mean rank-set score.
  • No reflection: the mutation model sees only the parent persona, its scores, and the text-check and gate-rejection feedback.

[Table R1]. Selection and reflection ablations on VitaBench Delivery. Reduction in avg@3 points.

AgentTIPERTop-3Top-1No reflection
GPT-5.425.025.015.88.3
MiniMax-M327.518.318.311.7

Pareto selection is at least as good as top-3 and better than top-1. Top-3 ties with Pareto selection on GPT-5.4 and is 9.2 points lower on MiniMax-M3; top-1 is 9.2 points lower on both agents. By design, top-K keeps the personas with the best average score, so it can drop a persona that makes the agent fail tasks no other persona does. Pareto selection keeps such a persona, and the search can build on it. This is consistent with GEPA’s comparison of Pareto selection with top-K selection (Agrawal et al., 2026).

Pareto selection also keeps the discovered failures more diverse. We measured two quantities in each search: task coverage, the number of rank-set tasks on which at least one persona in the final active set makes the agent fail, and behavior categories, the number of user-behavior categories covered by all candidates that pass the gate. The categories are the failure-causing behavior types of Section 5.2, and the Pareto and top-3 searches use identical settings.

[Table R2]. Diversity under Pareto and top-3 selection. Task coverage counts rank-set tasks (30 in each setting).

SettingTask coverage (Pareto)Task coverage (Top-3)Behavior categories (Pareto)Behavior categories (Top-3)
Delivery, GPT-5.4212074
Delivery, MiniMax-M3181542
ColBench, GPT-5.4191253

Pareto selection covers more tasks and more behavior categories in every setting, and on Delivery the gap follows Table R1. With GPT-5.4, where top-3 ties with Pareto selection, the coverage is also similar (21 vs. 20), although top-3 explores fewer behavior categories (4 vs. 7). On MiniMax-M3, where top-3 is 9.2 points lower, it covers 3 fewer tasks and half as many behavior categories (2 vs. 4). This is consistent with the mechanism above: top-K drops the personas that make the agent fail tasks no other persona does.

Reflection contributes the most, and integrity filtering is also needed. Without reflection, the reduction falls by about 16 points for both agents (25.0 → 8.3; 27.5 → 11.7), more than the 9.2 points from replacing Pareto selection. Without the integrity gate, the reduction falls from 25.3 to 10.5 on Telecom (Table 4(a) of the paper). As discussed in Section 5.1, we hypothesize that the gate also acts as a regularizer: it keeps proposed candidates from overfitting the search tasks through task-invalidating shortcuts that do not transfer to held-out tasks.

We will add these ablations to Table 4 of the paper, together with the diversity analysis.

References

  • Agrawal et al. GEPA: Reflective prompt evolution can outperform reinforcement learning. arXiv:2507.19457, 2026.

[Q2] Cross-agent transfer experiments.

To further study how well the searched personas transfer and generalize, we froze the personas searched against GPT-5.4 on Telecom and Delivery and evaluated them on two other agents, with all other settings fixed.

[Table R3]. Cross-agent transfer. Reduction in avg@3 points. DS-V4.1-Flash replaces DS-V4-Flash (no longer available), so its numbers are not from Table 2 of the paper.

BenchmarkAgentFrozen GPT-5.4 personaPersona searched against this agent
TelecomDS-V4.1-Flash29.017.9
TelecomMiniMax-M331.537.0
DeliveryDS-V4.1-Flash24.223.3
DeliveryMiniMax-M327.527.5

The personas largely transfer. The frozen personas reduce both agents by 24.2–31.5 points, and on MiniMax-M3 they are comparable to an agent-specific search (31.5 vs. 37.0; 27.5 vs. 27.5). This indicates that the personas expose weaknesses shared across agents, while some weaknesses remain agent-specific. Another Telecom persona transfers fully to MiniMax-M3 (20.4 vs. 19.8 on GPT-5.4) but barely to DS-V4.1-Flash (1.9), so per-agent search remains useful. We will add these transfer results to the paper.

[W2 & Q3] Integrity rates and evaluation reliability.

Thanks for raising these. Below we answer three questions: (1) why the search-time gate checks only failures, (2) whether filtering changes which rollouts count, and (3) how the results look on a common set of valid tasks.

The search-time gate is an efficient approximation, and all reported results are based on auditing every rollout. During search, we check integrity only on failures to save audit cost, since only a failed rollout can turn a simulator error into an apparent agent failure. Invalid successes are rare (1 of 371 TIPER successes on τ³-bench), so the gate also keeps the overall invalid rate close to its 5% threshold. The final evaluation then audits every rollout, including successes.

Filtering removes only a few trajectories, equally in both conditions. We audited all 1,290 held-out τ³-bench trajectories behind the paper's Table 1, using the paper's two-judge setup:

[Table R4]. Overall integrity on the held-out τ³-bench trajectories.

ConditionOverall integritySuccesses removedFailures removed
Default98.9%2 of 4465 of 199
TIPER98.9%1 of 3716 of 274

The reductions barely change, whether we remove invalid trajectories or keep only common-valid tasks. Common-valid tasks are those with three valid trials in both conditions, as in Appendix B.4. On LiC and ColBench, the reductions change by at most 1.1 points.

[Table R5]. Change in the reduction (avg@3 points) relative to using all trajectories.

DomainRemove invalid trajectoriesCommon-valid tasks
Telecom0.00.0
Airline0.0+0.1
Retail−1.6−1.2
Banking+0.7+2.8

We will clarify the search-time gate in Section 3.2 and report the overall integrity rates and the common-valid results in the paper.

Thank you again for the constructive feedback. We hope these results address your questions, and we are happy to discuss further.

Thank you for your response. Most of my concerns are addressed and I would like to maintain my original positive score.

Reviewer_cgAPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the TIPER, which automatically searches for user behavior strategies to identify weaknesses in the robustness of interactive agents. It uses interaction traces to guide personas through reflection and revision, and combines task completeness checks with Pareto-based selection based on performance across tasks to retain candidate solutions. Through manual review and failure analysis, they examined whether these failures occurred in valid tasks and which specific agent errors they corresponded to.

Critical strengths
  • It incorporates task integrity checks into the search and final evaluation processes addressing the issue of failure attribution caused by simulated users disrupting tasks. The authors conducted a blind manual review of 150 automatically accepted failure trajectories, of which 148 were deemed to have maintained task integrity, and 143 were found to pinpoint specific agent errors.

  • The experiments covered 4 benchmarks and included evaluations of 7 target agents, with distinctions made between search, candidate selection, and testing tasks. In addition to the default simulated user, the paper also compared strategies with fixed user behavior, as well as a variant of PPol that balances optimization objectives and search budgets. It provides empirical insights with an analysis of failure patterns such as unverified user information, omitted follow-up requirements, and premature execution.

Critical weaknesses
  • The ablations do not isolate the contributions of reflection and Pareto selection, leaving it unclear how much additional benefit these components provide over simpler search methods under the same search budget.

  • With the persona fixed, changing the user-simulator backbone reduces the performance drop on Telecom from 25.3 percentage points to 4.3 points, suggesting substantial dependence on the simulator. It remains unclear whether this reflects adaptation to the original simulator or differences in how faithfully the models follow the persona.

  • The paper defines the optimization objective as reducing the task score in valid interactions while constraining the overall validity rate of all interactions; however, the actual search uses task scores that have not been filtered for completeness, and the completeness threshold applies only to failed interactions. The relationship between these two aspects requires further clarification.

Questions for authors

See weaknesses above.

3 comments

Thank you for the thoughtful review. During the rebuttal period, we ablated reflection and Pareto selection, tested how faithfully other simulators perform the personas, and checked whether the search-time approximation changes the selected persona.

[W1] Ablation of reflection and Pareto selection.

We removed each component in turn on the VitaBench Delivery domain, keeping all other settings fixed, including the search budget:

  • Top-K: replaces Pareto selection; the active set keeps the K personas with the best mean rank-set score.
  • No reflection: the mutation model sees only the parent persona, its scores, and the text-check and gate-rejection feedback.

[Table R1]. Selection and reflection ablations on VitaBench Delivery. Reduction in avg@3 points.

AgentTIPERTop-3Top-1No reflection
GPT-5.425.025.015.88.3
MiniMax-M327.518.318.311.7

Reflection is important for both agents, and Pareto selection is at least as good as selecting by mean score. Without reflection, the reduction falls by about 16 points for both agents. Against top-3, Pareto selection ties on GPT-5.4 and is 9.2 points better on MiniMax-M3. Against top-1, it is 9.2 points better for both agents. We will add these ablations to Table 4 of the paper.

[W2] Simulator dependence analysis.

The two explanations make different predictions. If the persona were adapted to GPT-5.4, other simulators would harm the agent less even when they perform the same behavior. If fidelity were the cause, they would harm the agent less only because they perform the behavior less often. We selected two Telecom personas searched against GPT-5.4 (A and B) and ran one replay experiment and two interventions, with the agent fixed to GPT-5.4:

  1. Replay under other simulators, measuring execution: the share of rollouts in which the simulator actually performs the persona's key behavior.
  2. Removing the behavior: we delete only the key-behavior instructions from the persona and keep the rest.
  3. Raising execution: to increase the occurrence of key behaviors, we add a short prompt (the reminder) before each simulator turn that asks the simulator whether the current situation fits the key behavior and, if so, instructs it to perform the behavior (Shim et al., 2026).

[Table R2]. Replay and reminder intervention. Each cell is the reduction in avg@3 points, with execution in parentheses. As in Table 4(c) of the paper, each simulator is compared with its own default. DS-V4.1-Flash replaces DS-V4-Flash, which was no longer available during the rebuttal.

SimulatorPersona AA + reminderPersona BB + reminder
DS-V4.1-Flash16.7 (20%)24.7 (31%)46.9 (65%)51.9 (81%)
GPT-5.23.7 (2%)12.3 (15%)29.6 (65%)35.8 (76%)
GPT-5.4-mini−1.2 (3%)10.5 (23%)41.4 (70%)43.2 (74%)
Sonnet-54.9 (4%)32.7 (33%)——
GPT-4.1——19.1 (44%)38.3 (94%)

[Table R3]. Behavior-removal intervention. Reduction in avg@3 points.

PersonaSimulatorWith behaviorBehavior removed
AGPT-5.425.37.4
ADS-V4.1-Flash16.70.0
BGPT-5.419.8−7.4
BGPT-5.4-mini41.42.5

On Telecom, the smaller drop on other simulators reflects persona fidelity, not adaptation to GPT-5.4. Simulators that rarely perform A’s behavior (2–4% of rollouts) show little reduction. B’s behavior, which all simulators perform often (44–75%), reduces GPT-5.4 by 19.1–46.9 under the other simulators, comparable to or larger than the 19.8 under GPT-5.4 itself, with 75% execution. This contradicts what adaptation predicts. Both interventions confirm the causal link: the reduction largely disappears when the behavior is removed (A: 25.3 → 7.4; B: 19.8 → −7.4), and the reminder raises it in all eight simulator–persona pairs (r = 0.77 between the gains in execution and in reduction). This also explains the small drop under Sonnet-5 in Table 4(c) of the paper: Sonnet-5 performs A’s behavior in only 4% of rollouts, and with the reminder (33%) the reduction rises from 4.9 to 32.7, above the 25.3 under GPT-5.4. DS-V4-Flash, which gives the 4.3 you cite, was no longer available during the rebuttal, so we could not measure its execution, but every simulator we could test follows the same pattern.

We will add these analyses to the paper and report execution rates alongside the cross-simulator results.

[W3] Gap between the search-time objective and Eq. (2).

The search uses an approximation to Eq. (2) that only affects which persona is selected; the gap is small, and applying Eq. (2) exactly selects the same persona. All reported results already follow Eq. (2), because the final evaluation checks every rollout. Only the search uses the approximation, to save audit cost.

The approximation differs from Eq. (2) in two places, and both gaps are small:

  1. It does not check successes. The only invalid rollouts it can miss are invalid successes, and these are rare. After a thorough check, only 5 of 221 successes in the paper’s Telecom search and 1 of 371 TIPER successes on the τ³-bench test sets are invalid. A persona that passes the gate therefore has an overall invalid rate of at most 5% plus this small share, close to the constraint in Eq. (2).
  2. It does not filter the score. The unfiltered score can count an invalid failure as an agent failure, but the gate caps these at 5% of failures. The failure rate is thus overstated by at most 5% of itself, e.g., by at most 2 points when the failure rate is 40%. Conversely, invalid successes can only make a persona look weaker.

Both gaps are small relative to the differences between candidates. Empirically, re-ranking the 7 candidates of the paper’s Telecom search with filtered scores and the validity check on all rollouts, as in Eq. (2), keeps the paper’s persona first. We will add this explanation to Section 3.2.

Thank you again for the constructive feedback. We hope these experiments and clarifications address your concerns, and we are happy to discuss further.

I thank the authors for the additional results and the detailed rebuttal. I will maintain my original score.

Reviewer_UP9uOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TIPER, a framework for discovering difficult user personas to evaluate agents in interactive settings. The framework performs persona evolution to discover user policies that can cause agents to fail, while still keeping the task itself valid, which the users term as task-integrity. The personas evolved by TIPER happen through reflection guided mutation and task-wise Pareto selection with a task-integrity constraint. TIPER leads to lower agent scores in 12 GPT-5.4 settings while also enabling more diagnostic evaluating of interactive agents.

Critical strengths
  1. The paper presents an important framing in separating agent weaknesses from underlying task feasibility when user simulators are involved. Even when users can be challenging, sometimes the task itself is unsolvable, and agent failures, which may increase, are generally invalid. Recognizing this through maintaining task integrity is a useful objective.
  2. The interactive agent evaluation is broad, over 12 settings, 7 agents and 4 benchmarks, which allows interpreting the results to draw robustness related conclusions about interactive agents.
Critical weaknesses
  1. It is unclear whether the simulated user behaviours maintain realism while challenging agent failures and maintaining task integrity. The search objective is adversarial in nature, so I wonder if there is a possibility of reward hacking, where the personas preserve task integrity without representing behaviours that are actually likely to occur in realistic interactions. While the USI metrics are reported, these metrics only capture surface level regex features that may not be completely reliable in distinguishing useful simulator behaviour that maintains task integrity. Seeing human evaluation and further analysis of potential reward hacking in the personas would strengthen the contribution.

  2. There seem to be robustness issues with the simulator backbone. For instance, “On Telecom the reduction nearly vanishes”. This makes me question whether the reduction in agent performance is an artifact of how a specific simulator presents a persona, and not a fundamental common failure mode associated with the agent itself. Similarly, how would personas evolved against one agent perform against other agents? This ends up raising questions about how we interpret the findings of how user behaviours translate to agent failures. Right now, it is being reported as a general finding, but the robustness and transferability issues make it hard to interpret as a genuine finding about agent behaviours.

Questions for authors
  1. In the Integrity-Audit System Prompts, the judge is encouraged to be conservative and when in doubt, judge the output as a genuine failure rather than an invalid trajectory. Could the authors clarify why this decision was taken? For the blinded human audit on failure trajectories, could we also see recall numbers, where humans judge trajectories that both did and did not pass, and qualitatively assess where simulators are leading to invalid results?

  2. Please see weaknesses for remaining questions

6 comments

We thank Reviewer UP9u for the valuable feedback! During the rebuttal period, we added human evaluations of user realism and of the key behaviors, a broader audit of the integrity judge, and experiments on persona execution and cross-agent transfer. Below we explain how these results address your concerns and clarify the scope of our findings.

[W1] Realism and reward hacking analysis.

Thanks for raising this. Following your suggestion, we ran a blinded human evaluation, compared the simulated users with real users at the behavior level, and tested the failure-causing behaviors for reward hacking.

TIPER's users are as realistic as the default users, both in human ratings and in behavior-level distance to real users.

  1. Human evaluation. We sampled 60 matched pairs from τ³-bench, each with a default and a TIPER conversation on the same task. Two experienced annotators, blinded to the condition, scored the realism of every conversation from 1 to 5 (5 = indistinguishable from a real customer) and flagged any suspicious behavior or exploitation of the simulation. The two annotators agree on 98% of their labels.
  2. Behavior-level distance to real users. Following Mehri et al. (2026), we described the user’s behavior in each conversation along six facets, embedded these descriptions, and clustered them into behavior categories. We then computed the Jensen–Shannon divergence (JSD) between the category distribution of each condition and that of real users (Zhou et al., 2026).

[Table R1]. Human evaluation of matched pairs.

MetricDefaultTIPER
Human realism rating (1–5)4.034.06
Suspicious87
Exploiting the simulation00

[Table R2]. JSD to real users (lower is closer). Invalid = personas rejected by the integrity gate during search.

MetricDefaultTIPERInvalid
JSD (Retail)0.3520.3270.567
JSD (Airline)0.3760.4190.621

Human ratings show no significant difference in realism (paired difference +0.03, 95% CI [−0.17, 0.23]), and default and TIPER users are comparably far from real users in JSD, while Invalid personas are much farther. The measure thus detects unrealistic users, and the remaining gap to real users comes from LLM simulation itself, not from TIPER.

We find no sign of reward hacking: the behaviors TIPER discovers are plausible though uncommon, and the weaknesses they expose are still worth finding.

Average realism could hide problems in exactly the conversations that cause failures, so one annotator also rated the failure-causing behavior in 60 Telecom conversations from two searched personas. They used the same 1–5 scale and marked whether the user seemed mechanical and whether the agent could have handled the behavior. The two behaviors received mean ratings of 3.9 and 4.5. Only 4 of the 60 conversations seemed mechanical, and in 58 the agent could have handled the behavior. So the behaviors are plausible, and the agent had a fair chance to succeed.

These behavior types also appear in real deployments. We sampled 10,000 redacted, proprietary conversations between real customers and AI agents from four domains similar to those of τ³-bench (retail, telecom, airline and banking; 2,500 each). Two LLM judges independently labeled whether the user shows one of the failure-causing behavior types of Section 5.2; 156 conversations were flagged by both, and a manual check of all of them confirmed 153 (1.5% of the sample). We cannot release the conversations, but we will add the aggregate statistics and the judge prompts to the appendix.

Moreover, as you note, the search is adversarial in nature, and adversarial evaluation judges a test case by whether it is valid, rather than by how often it occurs. Adversarial examples are rare in natural data, yet they are valid inputs on which models fail, and they reveal brittleness that average-case accuracy hides (Szegedy et al., 2014; Goodfellow et al., 2015; Madry et al., 2018). TIPER applies this logic to interactive agents. Text-level and trajectory-level integrity checks keep the task solvable, human annotation confirms plausibility, and the failures reveal agent weaknesses that the default user misses.

We will state this scope in the main text, revise the introduction to claim comparable realism, present USI as one of several measures, and add these evaluations to the paper.

References

  • Goodfellow et al. Explaining and harnessing adversarial examples. ICLR 2015.
  • Madry et al. Towards deep learning models resistant to adversarial attacks. ICLR 2018.
  • Mehri et al. Measuring and mitigating the distributional gap between real and simulated user behaviors. arXiv:2605.07847, 2026.
  • Szegedy et al. Intriguing properties of neural networks. ICLR 2014.
  • Zhou et al. Mind the sim2real gap in user simulation for agentic tasks. arXiv:2603.11245, 2026.

[Q1] Impact of the judge instructions.

Thanks for the question. The current judge instruction comes from an earlier calibration against human labels, where a stricter prompt also rejected difficult but legitimate user behaviors. We agree that it might let simulator errors count as agent failures, so we tested whether a stricter prompt changes the results, and ran the two-sided human audit as you suggested.

A stricter LLM judge gives nearly the same results. We re-audited all 1,290 held-out τ³-bench trajectories behind the paper's Table 1, in both conditions and including successes, with both the paper's prompt and the strict prompt. The strict prompt also flags ordinary customer friction, such as terse, vague or skeptical replies. It flags slightly more trajectories, but the reductions barely change.

[Table R3]. Paper's prompt vs. strict prompt on all 1,290 held-out τ³-bench trajectories.

MetricPaper's promptStrict prompt
Trajectories flagged invalid (default / TIPER)1.1% / 1.1%2.8% / 1.6%
Telecom reduction after filtering25.325.0
Largest change in any domain's reduction (vs. no filtering)1.61.9

The human audit agrees with the judge in both directions. Two experienced annotators checked 94 held-out trajectories that did and did not pass the judge, from both conditions and all four τ³-bench domains, without seeing the judge's labels.

[Table R4]. Two-sided human audit of the integrity judge.

Judge's decisionTrajectories checkedAnnotators agree with the judge
Rejected14 (all)14
Accepted80 (random sample, including successes)80

All 14 rejections are correct, and none of the 80 acceptances is a missed invalid trajectory. In the paper, a separate audit of 150 accepted TIPER failures found 148 valid (Appendix E.3).

In our observation, invalid results come mostly from simulator errors, not from the persona instructions. This agrees with prior work showing that LLM-simulated users are unreliable proxies for human users (Seshadri et al., 2026). Qualitatively, we found two cases in which the simulator makes a test invalid.

  • Case 1: long or multi-step tasks. The simulator forgets part of the task or confirms an incomplete item list, much as LLMs get lost in long multi-turn conversations (Laban et al., 2026). All 14 trajectories the judge rejected fall into this case. It occurs equally often with default and TIPER personas (1.1% of the held-out τ³-bench trajectories in each condition), and the integrity check removes these trajectories.
  • Case 2: complex behavioral instructions. One TIPER persona instructs the user to turn a setting back on, but to answer truthfully whenever asked about it. The simulator sometimes loses track of the device state: in 5 of the 36 rollouts with this behavior, it told the agent that the setting was still off. The judge can miss such cases. Treating these rollouts as invalid lowers the Telecom reduction from 25.3 to 24.4.

In both cases, “invalid” does not mean that the persona’s behavior breaks the task. The instructions keep the task solvable, and the error lies in how the simulator carries them out. These failures should therefore be attributed to the simulator, not to the agent, which is why the integrity check aims to exclude them from the reductions.

We will explain the choice of the judge instruction and add the strict-prompt comparison and the two-sided audit to the paper.

References

  • Laban et al. LLMs get lost in multi-turn conversation. ICLR 2026.
  • Seshadri et al. Lost in simulation: LLM-simulated users are unreliable proxies for human users in agentic evaluations. arXiv:2601.17087, 2026.

[W2] Simulator dependence and cross-agent transfer.

Thank you for pressing on this point. We agree that the reduction depends on how the simulator presents the persona, and we believe this is expected. What TIPER discovers is a user behavior that exposes an agent weakness, and the persona is the instruction that induces it. A simulator that does not perform the behavior cannot expose the weakness. LLM simulators tend to default to cooperative behavior (Naous et al., 2026), so they differ in how faithfully they perform it.

Causal analysis of behavior occurrence and performance reduction. We selected two Telecom personas searched against GPT-5.4 (A and B) and ran one replay experiment and two interventions, with the agent fixed to GPT-5.4 (54 test tasks × 3):

  1. Replay under other simulators, measuring execution: the share of rollouts in which the simulator actually performs the persona's key behavior.
  2. Removing the behavior: we delete only the key-behavior instructions from the persona and keep the rest.
  3. Raising execution: to increase the occurrence of key behaviors, we add a short prompt (the reminder) before each simulator turn that asks the simulator whether the current situation fits the key behavior and, if so, instructs it to perform the behavior (Shim et al., 2026).

[Table R5]. Replay and reminder intervention. Each cell is the reduction in avg@3 points, with execution in parentheses. As in Table 4(c) of the paper, each simulator is compared with its own default. DS-V4.1-Flash replaces DS-V4-Flash, which was no longer available during the rebuttal.

SimulatorPersona AA + reminderPersona BB + reminder
DS-V4.1-Flash16.7 (20%)24.7 (31%)46.9 (65%)51.9 (81%)
GPT-5.23.7 (2%)12.3 (15%)29.6 (65%)35.8 (76%)
GPT-5.4-mini−1.2 (3%)10.5 (23%)41.4 (70%)43.2 (74%)
Sonnet-54.9 (4%)32.7 (33%)——
GPT-4.1——19.1 (44%)38.3 (94%)

[Table R6]. Behavior-removal intervention. Reduction in avg@3 points.

PersonaSimulatorWith behaviorBehavior removed
AGPT-5.425.37.4
ADS-V4.1-Flash16.70.0
BGPT-5.419.8−7.4
BGPT-5.4-mini41.42.5

On Telecom, the performance reduction comes from the behaviors the personas induce, not from a specific simulator. Simulators that rarely perform A’s behavior (2–4% of rollouts) show little reduction, while B’s behavior, which all simulators perform often (44–75%), reduces GPT-5.4 by 19.1–46.9 under the other simulators (19.8 under GPT-5.4 itself, with 75% execution). Both interventions confirm the causal link: removing the behavior largely removes the reduction (A: 25.3 → 7.4; B: 19.8 → −7.4), and the reminder raises it in all eight simulator–persona pairs (r = 0.77 between the gains in execution and in reduction). This explains why the Telecom reduction “nearly vanishes” under Sonnet-5 in Table 4(c) of the paper: Sonnet-5 performs A’s behavior in only 4% of rollouts, and with the reminder (33%) the reduction rises from 4.9 to 32.7, above the 25.3 under GPT-5.4. DS-V4-Flash, the other backbone in Table 4(c) (4.3), was no longer available during the rebuttal, so we could not measure its execution. Every simulator we could test follows the same pattern: low execution comes with a small reduction, and raising execution raises the reduction.

The weaknesses can transfer across agents. To test how personas searched against one agent perform against other agents, we froze the personas searched against GPT-5.4 for Telecom and Delivery and evaluated them on DS-V4.1-Flash and MiniMax-M3 with the same simulator. They reduce these agents by 24.2–31.5 points. On MiniMax-M3, this is comparable to the reduction from a persona searched against MiniMax-M3 itself.

[Table R7]. Cross-agent transfer. Reduction in avg@3 points. DS-V4.1-Flash replaces DS-V4-Flash (no longer available), so its numbers are not from Table 2 of the paper.

BenchmarkAgentFrozen GPT-5.4 personaPersona searched against this agent
TelecomDS-V4.1-Flash29.017.9
TelecomMiniMax-M331.537.0
DeliveryDS-V4.1-Flash24.223.3
DeliveryMiniMax-M327.527.5

We will add these analyses to the paper, report execution rates alongside the cross-simulator results, and clarify in the main text that each reduction reflects an agent weakness exposed by a behavior the simulator actually performs.

Thank you again for the constructive feedback. We hope these analyses address your concerns, and we are happy to discuss further.

References

  • Naous et al. Flipping the dialogue: Training and evaluating user language models. ICLR 2026.
  • Shim et al. Non-collaborative user simulators for tool agents. arXiv:2509.23124, 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Discussion Summary

We sincerely thank all reviewers for their thoughtful reviews and constructive comments. After the rebuttal, Reviewers 9ghK and cgAP kept their positive scores, and we addressed the two remaining points from Reviewer UP9u in our final response. Below is a summary of our discussion and final remarks:

Strengths agreed by the reviewers.

Reviewers recognized the strengths of this work, including the important framing of separating genuine agent failures from task invalidation (Reviewer UP9u, cgAP, 9ghK), broad evaluation across 4 benchmarks and 7 agents (Reviewer UP9u, cgAP), consistent failure exposure compared with predefined user behaviors and an objective-matched search baseline (Reviewer cgAP, 9ghK), and a human-audited analysis of agent failure patterns (Reviewer cgAP).

Concerns raised by the reviewers & how we addressed them

1. Whether the reduction reflects agent weaknesses or simulator artifacts. (Reviewer UP9u, cgAP)

In rebuttal, we measured how often each simulator actually performs the persona's key behavior, and observed that the reduction tracks this execution rate rather than the specific simulator. Removing the key behavior largely removes the reduction (25.3 → 7.4), while raising its execution increases the reduction in all 8 simulator–persona pairs (e.g., 4.9 → 32.7 under Sonnet-5). The search simulator performs the key behaviors in 44% and 75% of all rollouts, and TIPER keeps only personas that lower the score in actual rollouts, so it optimizes the realized effect rather than adherence. Each persona is searched for a given simulator, and we will report execution rates alongside all cross-simulator results in the final version.

2. Reliability of the integrity judge. (Reviewer UP9u)

We clarified that the conservative judge instruction came from calibration against human labels, and showed that a stricter prompt changes the reductions by at most 1.9 points. We also added a two-sided human audit of 94 trajectories that did and did not pass the judge, in which annotators agreed with every judge decision.

3. Search-time gate vs. the overall integrity objective. (Reviewer cgAP, 9ghK)

We clarified that the failure-only gate is a cost-saving approximation used only during search, and that all reported results audit every rollout. Auditing all 1,290 held-out τ³-bench trajectories, including successes, gives 98.9% overall integrity in both conditions, and restricting to common-valid tasks changes the reductions by at most 2.8 points. On Telecom, the exact objective in Eq. (2) selects the same persona.

4. Realism of the simulated users and potential reward hacking. (Reviewer UP9u)

We added a blinded human evaluation and observed that TIPER users are as realistic as default users (4.06 vs. 4.03), with no exploitation of the simulation. We also showed that the failure-causing behaviors are rated plausible and appear in 1.5% of 10,000 real customer–AI-agent conversations. As the search is adversarial, we judge these behaviors by validity and plausibility, not frequency; the remaining gap to real users comes from LLM simulation and is shared by default users. We will revise our realism claim from "stronger" to "comparable."

5. Contribution of reflection and Pareto selection. (Reviewer cgAP, 9ghK)

We added ablations under the same search budget. Removing reflection lowers the reduction by about 16 points. Replacing Pareto selection with top-K lowers it by up to 9.2 points and covers fewer failed tasks and behavior categories.

6. Cross-agent transfer of the discovered personas. (Reviewer UP9u, 9ghK)

We evaluated personas searched against GPT-5.4 on two other agents, and observed reductions of 24.2–31.5 points. On MiniMax-M3, this is comparable to a search run against that agent directly.

We truly appreciate the reviewers' and ACs' time and effort, and will include these discussions in the final version, together with a clear statement of scope: TIPER finds valid and plausible user behaviors that cause genuine agent failures under a given simulator. Our task-integrity-preserving persona evolution framework lowers held-out scores in all 12 GPT-5.4 settings and all 28 agent–domain pairs, with a median reduction of 22.9 points across pairs, and connects discovered user behaviors to recurring agent failure modes, laying foundations for more diagnostic evaluation and robustness improvement of interactive agents.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.