acceptodds
Under review as a conference paper at ICLR 2027

The Opponent Learns Too: Alternating Adversarial Training for Stronger Conversational Agents

Abstract

Conversational agents need to coordinate dialogue and tool use across multiple turns to address evolving user needs. To develop these abilities, agents are trained with simulated users, and recent work has extended this paradigm to jointly train users and agents, yielding interactions that evolve throughout training. However, existing user simulators do not necessarily adapt to the current agent's remaining weaknesses, limiting their ability to provide targeted training signals for further improvement. In this paper, we propose Alternating Training with Opponent Adaptation (ATOA), which uses failures of the current agent to guide successive rounds of adversarial training between the agent and user simulators. Specifically, each round begins by fixing the agent and updating the user simulator using utterances from interactions in which the agent fails. The updated simulator is then fixed to train the agent using successful trajectories together with pairs of successful and failed attempts on the same task. Failures of the newly trained agent are used to guide the next simulator update, allowing the training challenges to evolve across successive rounds. To preserve task consistency, ATOA fixes the user's behavioral decisions in advance and trains only how those decisions are expressed in language, while task completion is independently verified from the resulting environment states. Experiments across different scenarios in -bench show consistent improvements in agent task success.

Primary area:
reinforcement learning
Keywords:
Large Language Models, LLM Agents, User Simulation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_mSEEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a training approach called ATOA, which is an iterative process of training an AI conversational agent and a user simulator. The user simulator learns from failures on tasks where the conversational agent sometimes succeeds. The conversational agent then uses this revised user model as its training partner using RAFT on successful conversations, followed by DPO on pairs of successful and failed conversations from the same task. One of the main design choices was to maintain a fixed controller for all user decisions. Only the user model can learn how to communicate these decisions. The goal of this design choice is to create challenges that reflect the conversational agent's shortcomings while maintaining a consistent environment for the task. The authors evaluated ATOA on the Retail, Airline, and Telecom tasks developed from τ²-bench at two different model sizes.

In addition to comparing ATOA with other baseline approaches, the authors conducted component ablations, continuation experiments with controls, and evaluations with two other user models. The results indicate that ATOA improved performance under the authors' scripted-user evaluation, particularly in the Retail and Telecom domains. However, with the standard GPT-4.1 user it fell below the untrained base agent in Retail and Airline, while improving performance in Telecom.

Critical strengths

The paper contains multiple solid experimental controls. The authors start from the same agent checkpoint and then compare continuing to train on the previous user to training on the updated user, and they include a budget-matched version of this comparison that uses the same number of preference pairs and DPO steps. They also perform component-level ablations. All agents are tested using the same user and using the same behavioral scripts, and success is defined based on the state of the environment. The paper shows results where performance decreases, and there are useful implementation details in the appendix and supplementary code to allow one to analyze the most important aspects of the simulator. Thus, differences in the main results table can be attributed to the trained agents rather than to the evaluation user, and the shortcomings described below can be traced back to particular aspects of the experimental protocol.

Critical weaknesses

Evidence that ATOA generalizes to new users is insufficient. Most of the evaluations of ATOA utilize the author's own scripted-user framework and behavioral controller, and the user in the evaluation shares the same behavioral controller and the same script-trained language renderer with ATOA's first training opponent. Therefore, it cannot be ruled out that ATOA has learned the interaction protocol of this opponent. ATOA performance relative to the base agent using the standard GPT-4.1 user is worse by 5.35 and 5.25 percentage points on Retail and Airline respectively. Additionally, ATOA performs worse than fixed-user training. On the other hand, for Telecom, there is a large improvement of 27.50 percentage points. The GPT-5.5 renderer experiment utilized the same behavioral rules and GPT-5.5 was also the teacher model used to create the user SFT data. Due to this fact, the purpose of this experiment was primarily to test robustness to alternative language realizations while maintaining an identical behavioral policy for the user. As a result, the experimental results do not demonstrate that ATOA reduces opponent-specific gains in general. The main methods utilized in this study should be tested using independent user models and behavioral policies. In addition, the performance analyses for Retail and Airline need to be explained.

Questions for authors
  1. Could you evaluate Base, Fixed-user, and ATOA using a standard GPT-4.1 User and a User Simulator from a different model family (with matched tasks and budgets) and provide an explanation for why ATOA performed less well than the Base Agent in Retail & Airline under evaluation by the GPT-4.1 User.

  2. Could you evaluate whether the improvements made to ATOA were transferable to Users who followed differing behavioral policies or interaction scripts rather than just having their language rendered differently (i.e., using a different controller) so that we can determine which improvements are generalizable and which are specific to the interaction protocol.

  3. Could you report the success rates of the Post-RAFT and Post-DPO agents across training rounds to illustrate how much of the improvement can be attributed to RAFT vs. DPO.

4 comments
AuthorsOpenReview ↗

Response to Reviewer mSEE (1/2)

We thank the reviewer for the careful assessment and constructive suggestions. We especially appreciate the recognition of our controlled experiments and ablation studies. Below, we address the concerns regarding cross-user generalization, behavioral-policy transfer, and the contributions of RAFT and DPO.


W1 & Q1. Generalization to independent users and cooperative-user performance

We agree that our GPT-5.5 renderer experiment primarily examines robustness to alternative language realizations, rather than generalization to independent user policies. To address this limitation, we extend our evaluation to a user simulator from a different model family.

Additional evaluation with an independent user simulator. We evaluate Base, Fixed-user, and ATOA using both the standard GPT-4.1 user and a Llama-3.3-70B-based user simulator. Unlike our training opponent, the Llama user independently determines its responses and interaction decisions from the task goal and relevant private information, without accessing our behavioral controller, interaction scripts, or script-trained language renderer.

For direct comparability with Table 16, we retain the original evaluation tasks and budgets: 67 Retail tasks with six trials each, 27 Airline tasks with four trials each, and 40 Telecom tasks with two trials each. All agents use the same checkpoints, tools, and environment-based task verifiers.

Table R1. Cross-user evaluation with independent simulators

DomainEvaluation userBase (%)Fixed-user (%)ATOA (%)
RetailGPT-4.180.8577.1175.50
RetailLlama-3.3-70B42.2946.2752.49
AirlineGPT-4.167.2966.0462.04
AirlineLlama-3.3-70B26.8529.6332.41
TelecomGPT-4.165.0083.7592.50
TelecomLlama-3.3-70B62.5075.0082.50

The Llama evaluation tests whether ATOA's gains extend beyond the original scripted protocol. ATOA outperforms both baselines across all three domains, while the GPT-4.1 results reveal policy-dependent generalization.

Under cooperative GPT-4.1 evaluation, ATOA trails Base by 5.35 and 5.25 points on Retail and Airline, but improves Telecom by 27.50 points. This reflects a trade-off between training for challenging interactions, including delayed disclosure and limited patience, and performance with highly cooperative users.

ATOA aims to improve robustness to challenging interactions, not guarantee gains under every user policy. The independent-user results provide further evidence that its benefits extend beyond the training opponent.


W1 & Q2. Generalization across different behavioral policies

We agree that changing the language renderer alone does not establish generalization to different user behaviors. To examine this more directly, we introduce an independently implemented behavioral controller with interaction policies that differ from those used during ATOA training.

Additional cross-policy evaluation. We consider two user policies: a Proactive user who readily provides relevant information and cooperates with clarification requests, and a Delayed-disclosure user who provides information more gradually and requires additional clarification. Unlike our training controller, these policies independently determine information-disclosure timing and dialogue decisions. Both preserve the original task goals and constraints, and use the same Llama-3.3-70B language renderer to express their decisions.

We evaluate the frozen Base, Fixed-user, and ATOA agents on the same held-out tasks and trial budgets used in Table 16, keeping the agent tools and environment verifiers unchanged.

Table R2. Generalization to alternative behavioral policies

DomainUser policyBase (%)Fixed-user (%)ATOA (%)
RetailProactive70.4069.1570.65
RetailDelayed disclosure40.0542.5447.51
AirlineProactive51.8550.9350.93
AirlineDelayed disclosure18.5222.2225.00
TelecomProactive70.0082.5087.50
TelecomDelayed disclosure50.0066.2577.50

The comparison suggests that ATOA's benefits can extend beyond the original behavioral controller. Under the independently defined delayed-disclosure policy, ATOA outperforms both baselines across all three domains. Importantly, it also maintains competitive performance under the proactive policy, matching or slightly exceeding Base on Retail, trailing by only 0.92 percentage points on Airline, and retaining a substantial advantage on Telecom.

This result would indicate that ATOA's improvements are not confined to its training interaction protocol, while the magnitude of the gains remains dependent on user behavior and task domain.

AuthorsOpenReview ↗

Response to Reviewer mSEE (2/2)

Q3. Contributions of RAFT and DPO across training rounds

We thank the reviewer for this helpful suggestion. While the RAFT Only and DPO Only ablations in Table 2 demonstrate the importance of both components, we agree that reporting intermediate checkpoints would provide a clearer view of their contributions during alternating training.

Additional stage-wise evaluation. We evaluate the Post-RAFT and Post-DPO checkpoints after each training round of Qwen3.6-27B, using the same held-out tasks, scripted evaluation user, and trial seeds as our main experiments. We report the gain from RAFT relative to the preceding round's agent, followed by the additional gain from DPO.

Table R3. Stage-wise task success across ATOA rounds

DomainRoundPost-RAFT (%)Post-DPO (%)RAFT gain (pp)DPO gain (pp)
RetailR132.9634.21+7.64+1.25
R238.7240.38+4.51+1.66
R342.9144.85+2.53+1.94
R446.3147.84+1.46+1.53
AirlineR17.357.86+0.92+0.51
R28.669.16+0.80+0.50
R39.519.74+0.35+0.23
TelecomR177.6379.88+5.37+2.25
R282.9684.42+3.08+1.46
R384.9285.76+0.50+0.84

The stage-wise comparison shows that RAFT provides substantial improvements in the early rounds by learning from verified successful trajectories, while DPO contributes additional gains by contrasting successful and failed attempts on the same tasks. As training progresses, the gains from RAFT become smaller, whereas DPO continues to provide complementary improvements, particularly in the later Retail rounds.


W1. Clarifying the scope of cross-user generalization

We appreciate the reviewer's careful observations regarding our evaluation protocol. The GPT-5.5 experiment in Table 15 evaluates robustness to different language realizations under the same behavioral controller. Since GPT-5.5 also serves as the teacher for our initial user-SFT data, this experiment establishes language-level robustness rather than generalization to independent behavioral policies. Our independent-user and alternative-controller evaluations in Q1 and Q2 directly examine the latter.

Importantly, Fixed-user training and ATOA differ in more than opponent adaptation (Table 16). Fixed-user training uses an untrained user language renderer, whereas ATOA first applies script-based SFT and subsequently performs failure-guided opponent adaptation. Therefore, their performance difference cannot be attributed solely to opponent adaptation. To isolate this contribution, our Fixed Opponent ablation (Table 2) retains the same script-based user initialization but disables subsequent opponent updates. Under the scripted-user evaluation, ATOA improves over Fixed Opponent by 5.31 points on Retail, 1.58 points on Airline, and 7.85 points on Telecom. This controlled comparison directly demonstrates the additional benefit of opponent adaptation beyond user initialization.

The cooperative-user results highlight an important trade-off between robustness to challenging interactions and performance under highly cooperative behavior. ATOA is designed to strengthen agents against users who disclose information gradually, change requests, or exhibit limited patience. Under our challenging scripted-user evaluation, ATOA improves over Base by 22.52 points on Retail and 13.50 points on Telecom. Under the cooperative GPT-4.1 user, it retains success rates of 75.50% on Retail and 62.04% on Airline, despite decreases of 5.35 and 5.25 points relative to Base, while achieving a substantial 27.50-point improvement on Telecom. These results demonstrate that the benefits of challenging-user training are not uniform across interaction policies: ATOA delivers substantial gains under demanding interactions while maintaining considerable effectiveness with cooperative users, with the balance varying by domain.

Taken together, our controlled ablations establish the additional contribution of failure-guided opponent adaptation, while the independent-user and alternative-controller evaluations examine how these improvements transfer beyond the original training interaction protocol.


We sincerely thank the reviewer for the constructive feedback and valuable suggestions. We hope our clarifications and additional experiments address the main concerns and provide stronger evidence for ATOA's effectiveness and generalization. If our responses have adequately addressed your concerns, we would greatly appreciate your reconsideration of the rating.

Thanks for your response. Many of my concerns have been addressed. I will keep my score.

AuthorsOpenReview ↗

Thank You for Your Careful Review and Engagement in the Discussion

Thank you for your follow-up and for acknowledging that many of your concerns have been addressed. We appreciate your careful consideration of our responses and additional evidence.

If there are any remaining concerns that are particularly important to your overall assessment, we would be very happy to provide further clarification while the discussion is still open. We hope our additional experiments and revisions can be taken into account in your final evaluation.

Thank you again for your time and thoughtful feedback.

Reviewer_vKSMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a feasible alternating training method that updates a simulated user while training to improve the conversational agent. The simulated user is adapted to focus on language styles that remain challenging to the agent as the training progresses. The core of their method is to constrain the simulated user with a controller (using predetermined user behaviors), so that the user updates affect how the user speaks instead of the user’s actual actions/decisions. Their experiments show that their method outperforms the baselines and ablations show the contribution of updating the simulated user.

Critical strengths
  • The idea of the controller and distinguishing user behaviors/decisions from how they communicate those is reasonable, and proved to be helpful by their experiments. I think this is the main insight that made their alternating training work.
  • They have useful experiments that complement their main results, such as looking at interaction under cooperative interaction (different from their training scenario) and the ablations that justify their main implementation choices.
Critical weaknesses
  • The improvement brought by the method is marginal compared with the strongest baseline. I’m concerned that it’s an inherent issue of adversarial training in the conversational agent setting. The proposed method prevents the collapse due to reward hacking (common in adversarial training), but also sacrifices how much the simulated user can adapt, making the adversarial training less “adversarial”.
  • I feel that the paper's main contribution is applying the controller to this scenario and the rest is incremental. The overall idea is simple and did not make the whole method substantially outperform the strongest baseline. The rest of the design choices such as alternating training/bilateral training has been widely studied, including the adversarial form of bilateral training. The paper cites a few bilateral training papers, but it should survey the papers on adversarial training more thoroughly. DACT is a related paper they should discuss.
Questions for authors

Is it really needed to include tau 2-bench’s airline task if the model is failing the task most of the time (<10% success rate)? Any performance differences on this task seems uninformative given that all the results are low. Overall, I think this paper will benefit from more diverse datasets to show that their conclusions are generalizable across domains.

5 comments
AuthorsOpenReview ↗

Response to Reviewer vKSM (1/2)

We thank the reviewer for the thoughtful assessment and constructive suggestions. We appreciate the recognition of our behavioral controller and the supporting experiments. Below, we clarify the role of adversarial adaptation, distinguish ATOA from related approaches, and address the evaluation concerns.


W1. Adversarial strength versus useful training signals

We agree that the gains over the strongest baseline are moderate. Nevertheless, on Qwen3.6-27B, ATOA improves over Divergence-Point DPO by 4.64 points on Retail and 4.65 points on Telecom. More importantly, the controller does not eliminate adversarial adaptation; it constrains task-relevant decisions while allowing the user to adapt its language to the agent's latest failures. For example, the Retail Round-2 user update increases mixed-outcome cards from 23 to 31, while reducing the current agent's success rate from 40.4% to 37.8%. This shows that meaningful adversarial pressure remains even when the underlying task conditions are fixed.

Additional comparison of adversarial flexibility. To examine whether greater user flexibility leads to better agent training, we consider two additional opponent variants: a behavior-adaptive user that can choose among task-consistent behavioral options, and a free-form user that generates responses without the fixed directive controller. All variants start from the same Round-2 agent checkpoint and use matched interaction and agent-optimization budgets.

Table R1. Effect of opponent flexibility on Retail R2→R3

Training opponentCurrent-agent success (%) ↓Mixed-outcome cards ↑R3 agent success (%) ↑
Previous user40.42342.92
ATOA (fixed controller)37.83144.85
Behavior-adaptive user32.62744.10
Free-form user25.91541.85

The additional variants illustrate the key trade-off: greater behavioral freedom can make the opponent harder to satisfy, but does not necessarily yield better agent training. In particular, excessively difficult or task-inconsistent interactions may provide fewer usable success–failure pairs, weakening the subsequent optimization signal.

We therefore view the controller not simply as a restriction on adversarial strength, but as a way to preserve task validity while directing opponent adaptation toward failures the agent can learn from. The goal of ATOA is not to maximize the agent's failure rate, but to improve its ability to handle challenging interactions through progressively more informative training examples.


W2. Novelty beyond the controller and comparison with DACT

We appreciate the reviewer for pointing us to DACT (Long et al., 2026). We agree that bilateral and adversarial training are established ideas, and we will expand our Related Work accordingly. ATOA's contribution is not alternating optimization itself, but how the evolving opponent is trained to expose useful failures while preserving executable task requirements.

DACT uses multidimensional LLM judgments to construct dialogue-tree preferences and adversarially trains the client by reversing the coach's preferences. In contrast, ATOA adapts users from agent failures, selects informative mixed-outcome tasks, and trains agents on execution-verified trajectories.

Additional DACT-inspired comparison. Starting from the same Qwen3.6-27B Retail R2 checkpoint, we sample alternative user responses and construct reversed preferences from task outcomes, favoring responses that cause agent failures. Both methods retain the same controller and verifier, with matched interaction and agent-training budgets.

Table R2. DACT-inspired opponent adaptation on Retail R2→R3

User adaptationR3 task success (%)Gain over previous user (pp)
Previous user (no adaptation)42.92—
DACT-inspired preference optimization43.61+0.69
ATOA (failure-guided adaptation)44.85+1.93

The DACT-inspired update provides an improvement over continued training with the previous user, supporting the value of adversarial opponent adaptation. ATOA achieves a further 1.24-point improvement, suggesting that explicitly targeting the current agent's failures can provide more useful training signals than preference reversal alone.

We emphasize that this experiment adapts DACT's core adversarial preference mechanism to our executable-task setting, rather than reproducing its complete health-coaching framework. Together with the existing ablations, it helps clarify ATOA's distinct contribution: failure-guided opponent adaptation that produces informative training interactions under fixed task constraints, beyond the use of a controller or alternating training alone.

AuthorsOpenReview ↗

Response to Reviewer vKSM (2/2)

Q1. Airline difficulty and generalization across domains

We agree that the low success rates on Airline limit the interpretation of small performance differences. Unlike Retail and Telecom, Airline involves tightly constrained reservation modifications, refund eligibility checks, and multi-step confirmation procedures, where mistakes in any step can prevent successful task completion. These requirements become particularly challenging under our stochastic user protocol, which introduces incomplete information and limited patience. Moreover, Airline has a smaller training-side task collection (44 cards), and its first training round yields only 13 valid DPO pairs, limiting the available optimization signal. To mitigate these difficulties, we expand the task pool through template resampling, initialize the user with script-based SFT, and combine verified-success RAFT with same-task DPO across multiple training rounds. Although these measures improve Qwen3.6-27B performance from 6.43% to 9.74%, Airline remains challenging. We retain it to reflect this limitation rather than restricting evaluation to more favorable scenarios.

To broaden the domain coverage, we extend our evaluation to the knowledge-intensive Banking domain from τ³-bench. Unlike the original scenarios, Banking requires agents to retrieve information from a large policy knowledge base while completing multi-step, verifiable customer-service tasks.

We conduct a focused two-round experiment using Qwen3.6-27B, with 53 training cards, 22 selection cards, and 22 held-out test cards. Each test card is evaluated over eight trials. We compare Base, Fixed-user training, and ATOA under the same task conditions and agent-training budgets, with ATOA adapting the user between the two training rounds.

Table R3. Additional evaluation on τ³-Banking

MethodTask success (%)
Base16.48
Fixed-user training21.02
ATOA25.00

The additional comparison suggests that opponent adaptation can also benefit knowledge-intensive tasks requiring retrieval and tool execution. ATOA improves over Fixed-user training by 3.98 percentage points, providing initial evidence that the benefit is not limited to the original three scenarios.

We appreciate the reviewer's suggestion and will clarify the limitations of the Airline results while incorporating the additional domain evaluation into the revised manuscript.


We sincerely thank the reviewer for the constructive feedback and valuable suggestions. We hope our clarifications, additional comparisons with adversarial training methods, and expanded domain evaluation address the main concerns and strengthen the evidence for ATOA. If these responses have adequately addressed your concerns, we would greatly appreciate your reconsideration of the rating.

AuthorsOpenReview ↗

Gentle Follow-up on Our Additional Experiments and Clarifications

Dear Reviewer vKSM,

Thank you again for your thoughtful review and constructive suggestions, which have helped us strengthen our work.

We understand that this is a busy period, but we would greatly appreciate it if you could take a moment to revisit our rebuttal responses. In particular, we have provided a DACT-inspired comparison and further clarification of ATOA's methodological contributions, along with additional Banking-domain experiments to address your concerns about generalizability and the limited performance on Airline.

We hope these additional results and clarifications help address your key concerns and would be very happy to discuss any remaining questions. Your further feedback would be greatly appreciated and helpful in assessing our revised work.

Thank you again for your time and consideration.

Best regards,
The Authors

Reviewer_tAWaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Alternating Training with Opponent Adaptation (ATOA) for conversational agents. A fixed controller determines a simulated user's information disclosure, request changes, patience, and device operations, while a trainable language model realizes those decisions. The user model is repeatedly adapted using utterances from failed attempts on tasks with mixed success and failure outcomes. The agent subsequently learns through reward-filtered fine-tuning and trajectory-level DPO. An environment-based verifier supplies task outcomes throughout. Experiments use modified Retail, Airline, and Telecom protocols derived from τ²-bench, with two Qwen backbones. The paper reports improvements under a shared scripted evaluator, component ablations, same-checkpoint continuation experiments, a DPO-budget-matched Telecom comparison, and evaluations with an external renderer and a cooperative user. The evidence supports benefits in the controlled setting, with domain-dependent transfer outside it.

Critical strengths
  1. A useful separation of opponent adaptation from task validity. Sections 3.2–3.4 separate controller decisions, learned language, and mechanical verification. This directly addresses a failure mode of bilateral training: making tasks easier or harder by changing their rules rather than improving the agent. Appendix E.4 also audits directive enforcement, including an injected-violation positive control. This makes the proposed training setup meaningfully more interpretable than unconstrained joint adaptation.

  2. Several complementary controls support the central mechanism. Table 2 compares fixed opponents, one-time adaptation, random user updates, and removal of the mixed-outcome filter. Tables 11–12 additionally compare continuations from the same agent checkpoint and match the Telecom DPO budget. Table 15 shows that the Retail gain persists when the renderer changes to GPT-5.5. These are consequential controls, rather than evidence based solely on the large improvement over the untrained agent.

Critical weaknesses
  1. The reported stopping procedure is not yet sufficiently unambiguous to assess selection bias. Appendix C.5/Table 8 selects checkpoints using the displayed task-success progression and next-round deterioration, whereas Appendix D.1 states that final evaluation cards are not used for method selection. The manuscript does not clearly identify a separate selection set for these round decisions. Moreover, Table 17 says that the R5 probe performs no further RAFT/DPO optimization, while C.5 describes a next-round continuation result. This matters because the evidence for useful late-round adaptation includes differences of only 0.78–1.93 percentage points in Table 11. Each method is trained once, and Table 1's errors cover evaluation sampling only (Appendix A.4); selection and training variability could therefore affect the interpretation of these marginal gains.

  2. The evidence establishes a narrower form of robustness than the broad conversational-agent motivation suggests. The main results use a heavily controlled, modified protocol; Telecom assumes correct execution of user-side operations (Appendix A.3). Under the standard cooperative user, ATOA decreases Retail and Airline success by 5.35 and 5.25 points relative to the base agent, and also trails fixed-user training there (Table 16). These results are disclosed, but they limit the claim that harder-user training improves robustness without sacrificing ordinary task completion. The external-renderer control preserves the same behavioral controller and therefore does not resolve this behavioral-transfer limitation.

Questions for authors
  1. [Clarification] Which task cards and success estimates determine stopping and checkpoint retention in C.5? Please distinguish them from the final evaluation sets and reconcile the R5 result in Table 8 with the absence of RAFT/DPO in Table 17. A fully independent selection protocol would substantially reduce my main methodological concern.

  2. [Evidence request] For the same-checkpoint comparisons in Table 11 and the budget-matched comparison in Table 12, can you provide paired, card-level uncertainty for the differences and, if available, variability across training runs? This would establish whether the incremental benefit of updating the opponent is reliably distinguishable from noise. I am not asking for a new broad benchmark suite.

  3. [Clarification] How should the intended benefit be scoped in light of the cooperative-user losses in Table 16? Evidence that the relevant operating regime values these controlled-user gains enough to justify that tradeoff would strengthen the practical assessment.

4 comments
AuthorsOpenReview ↗

Response to Reviewer tAWa (1/2)

We thank the reviewer for the thoughtful feedback and recognition of our controlled user simulation and experimental comparisons. We address the concerns regarding checkpoint selection, statistical reliability, and cross-user generalization through additional evidence and clarifications below.


W1 & Q1. Clarification of checkpoint selection and the R5 continuation

We thank the reviewer for identifying the ambiguity in our checkpoint-selection procedure, particularly the relationship between Tables 8 and 17.

Checkpoint selection protocol. Our training-side task collections follow a 7:3 training/selection split at the task-card level, with an additional held-out test set used for final evaluation.

ScenarioTraining cardsSelection cardsHeld-out test cards
Retail803431
Airline311335
Telecom7432100

Training cards are used for user adaptation and agent optimization, while selection cards determine user-candidate acceptance and checkpoint retention. Each checkpoint is evaluated with eight trials per selection card. Training stops when an additional agent-training round fails to improve the best selection-set performance, and the best-performing checkpoint is retained. The held-out test cards do not influence these decisions.

Clarification of Table 8. Table 8 in the submitted manuscript reports round-by-round performance under the final evaluation protocol to illustrate how agent performance evolves throughout training. Since the retained checkpoints are also reported in Table 1, their results naturally coincide: 47.84% for Retail R4, 9.74% for Airline R3, and 85.76% for Telecom R3. These repeated values therefore reflect evaluations of the same checkpoints, rather than the use of final test results for checkpoint selection. We recognize that the original presentation did not make this sufficiently clear. For completeness, we provide the corresponding selection-set results below.

Table R1. Checkpoint selection on Qwen3.6-27B

ScenarioBaseR1R2R3R4R5Selected
Retail25.7434.5640.4445.2248.1647.43R4
Airline6.737.699.6210.588.65—R3
Telecom72.2779.6984.3885.9484.38—R3

The selection results favor R4 for Retail and R3 for Airline and Telecom, consistent with the checkpoints reported in our main experiments. The additional Retail R5 evaluation further illustrates why R4 is retained: although training continues for another round, R5 does not improve upon R4 on the selection set.

Clarification of the Retail R5 continuation. The Retail experiment includes a fifth round of user adaptation and agent optimization, producing an R5 agent checkpoint. This checkpoint achieves 47.60% under the final evaluation protocol, compared with 47.84% for R4. Importantly, the decision to retain R4 is based on the independent selection-set comparison, rather than these final test scores.

We acknowledge that Table 17 inadvertently omitted the RAFT+DPO computation for R5, making it appear that no additional agent training was performed. The R5 stage includes 31.8 GPU hours of interaction collection and 7.2 hours of user adaptation, together with the additional agent-training computation. We will correct the R5 training-cost entry and the corresponding cumulative costs in the revised manuscript.

We hope these clarifications help the reviewer better understand our checkpoint-selection and stopping procedure. We also thank the reviewer for helping us identify and improve the consistency of our experimental presentation.

AuthorsOpenReview ↗

Response to Reviewer tAWa (2/2)

W1 & Q2. Reliability of the gains from opponent adaptation

Tables 11 and 12 show that adapting the opponent improves subsequent agent training, but some late-round gains are relatively small. To examine their reliability, we provide additional evidence from paired uncertainty estimates and repeated training runs.

Paired uncertainty analysis. We evaluate both branches on the same task cards and trial seeds, then compute the difference in task success using 10,000 paired bootstrap samples at the card level. Each resampled card retains its repeated trials, allowing us to estimate uncertainty directly for the improvement rather than for each model separately.

Table R2. Paired uncertainty for opponent adaptation

ComparisonGain (pp)Paired 95% CI (pp)
Retail R2→R3: Adapted vs. Previous+1.93[-0.61, 4.44]
Retail R3→R4: Adapted vs. Previous+0.78[-1.87, 3.42]
Retail R3→R4: Adapted vs. Reused Data+0.87[-1.70, 3.61]
Telecom: Adapted vs. Previous+2.64[-0.18, 5.42]

The comparisons consistently favor the adapted opponent.

Additional multi-seed experiment. We repeat the Retail R2→R3 continuation using three training seeds, keeping the same Round-2 agent checkpoint, task collection, and agent-training budget.

Table R3. Retail R2→R3 continuation across training seeds

Training seedPrevious user (%)Adapted user (%)Gain (pp)
2026081442.9244.85+1.93
2026081542.5643.98+1.42
2026081643.1045.31+2.21
Mean gain ± SD——+1.85 ± 0.40

The adapted opponent improves agent performance across all three runs, with an average gain of 1.85 percentage points. While these runs assess one controlled continuation rather than the full multi-round training process, they provide additional evidence that the benefit is not specific to a single training seed.

Together with the Fixed Opponent and No Alternation ablations, these results strengthen the evidence that updating the user based on the agent's latest failures provides useful training signals beyond continued optimization with the previous user.


W2 & Q3. Robustness across user policies and the practical value of ATOA

ATOA primarily targets challenging multi-turn interactions, where delayed information disclosure, indirect responses, and limited patience make task completion more difficult. The scripted-user evaluation is designed to measure improvement under these conditions, rather than to establish uniform gains across all user policies.

The cooperative-user results in Table 16 provide an important complementary perspective. On Retail and Airline, ATOA reduces success by 5.35 and 5.25 percentage points relative to Base, while Telecom improves by 27.50 points. Thus, the gains under challenging interactions are accompanied by relatively modest cooperative-user losses in two domains, rather than improvements in every setting.

Additional mixed-user evaluation. To assess this trade-off on a common task distribution, we evaluate Qwen3.6-27B Base, Fixed-user, and ATOA on the same 31 held-out Retail tasks using both our challenging scripted user and the official cooperative GPT-4.1 user. Each task receives eight trials per user policy, with identical task goals and environment verification across methods. All agent checkpoints remain frozen.

We additionally report a balanced setting in which challenging and cooperative interactions receive equal weight. This provides a simple way to assess overall performance when agents must serve both types of users.

Table R4. Matched evaluation across user conditions

Evaluation settingBase (%)Fixed-user (%)ATOA (%)
Challenging scripted user25.4030.2447.98
Cooperative GPT-4.1 user80.2477.4275.81
Balanced mixture (50:50)52.8253.8361.90

Under the challenging user, ATOA improves over Base by 22.58 points, while its cooperative-user performance is 4.43 points lower. In the balanced setting, ATOA achieves 61.90%, exceeding Base and Fixed-user by 9.08 and 8.07 points, respectively.

The comparison also allows us to examine different user mixtures. Under this two-policy mixture, ATOA would outperform Base when challenging interactions account for more than approximately 16% of the workload. This analysis illustrates how improvements under challenging interactions can offset modest cooperative-user losses, rather than assuming a particular real-world user distribution.

These results clarify ATOA's intended scope: its main value lies in improving agents that must handle a meaningful proportion of challenging interactions while retaining substantial effectiveness with cooperative users. We will also clarify that our modified Telecom protocol, including the assumption of correct user-side operations, does not establish equivalent gains under the original benchmark configuration.

Thank you for your response. Many of my concerns have been addressed, and I will maintain my original score.

AuthorsOpenReview ↗

Thank You for Your Careful Review and Engagement in the Discussion

Thank you for your follow-up and for considering our responses. We sincerely appreciate your thoughtful feedback and time.

Reviewer_6rRAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Conversational agents that talk to users and call tools are usually trained against a simulated user that never changes so the agent can become good at beating that one simulator rather than good in general. The paper proposes ATOA, which trains the agent and the simulated user in turns. In each round, the agent’s failures are used to fine-tune the user’s LLM so it phrases things in ways that confuse the agent. The agent is then retrained against this updated user. To avoid the user from cheating by making tasks too hard or easy, a fixed rule-based controller decides what the user reveals, when it changes its mind, and when it gives up, and the trainable model only controls wording. Success is checked from the final environment state.

Critical strengths
  • The work is motivated by an interesting problem in conversational agents. The existing issues with bilateral training that the paper enumerates are established and it is important to resolve them. Separating the fixed controller from the trainable LM with proper checks and an environment-based verifier is a proper way to keep the task conditions fixed.

  • The idea of selectively picking tasks where the agent succeeds on some seeds but fails on others instead of updating the user on all agent failures is a meaningful choice in the designed framework to avoid introducing noise in training.

  • The experiments properly provide a complete analysis of the main claim of the paper. The paper isolates opponent adaptation, a budget-matched Telecom comparison, and several ablations that cleanly study and examine the design choices. Figure 3 b and appendix E1 properly starts from the same agent checkpoint and only varies the next round of training data to study the opponent adaptation and the user side ablations in section 4.6 and appendix E3 were particularly interesting.

Critical weaknesses
  • A critical concern of mine is that the evaluation is performed on the authors’ own protocol which closely matches the training user. As I understand it, agents are trained and tested against the same controller and script distribution and the benchmark was modified (user-side tool operations assumed correct, new task cards, etc.) The benchmark’s standard evaluation is considered as a secondary check, in which ATOA loses about 5 points on retail and airline with respect to the base model. This can show specialization to the scripted-user distribution which is the problem the paper aims to fix.

  • I am also concerned that the adversarial updates in the proposed framework seem to be restricted to surface paraphrasing. Because the behavioral controller, disclosure budgets, and patience thresholds are completely frozen, opponent adaptation may only learn linguistic phrasings of the exact same underlying directives and therefore the simulator may not adapt the high-level user behavioral strategies such as subtly changing the sequence of subgoals or dynamically changing conversational priorities.

  • I don’t understand where the candidate user acceptance rule in Equation 16 comes from. It seems a hand-designed heuristic with somewhat random thresholds. The paper doesn’t justify this choice theoretically and doesn’t perform any sensitivity check to these exact hyperparameter cutoffs. This can especially be important in generalizability of the method beyond tau^2 benchmark.

  • The training pipeline seems to be quite exceptionally sample and compute-intensive. Each round requires collecting eight repeated interaction rollouts across all training task cards to classify mixed outcomes, generating candidate interactions, auditing compliance, and running multi-stage agent training.

Questions for authors

Please view my concerns in the weaknesses section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 6rRA (1/2)

We thank the reviewer for the constructive feedback and recognition of our methodological design and controlled experiments. We address the main concerns below through additional experiments and clarifications.


W1. Does the scripted-user evaluation favor ATOA?

Our scripted-user evaluation targets challenging multi-turn interactions involving gradual information disclosure, indirect responses, and limited patience. These conditions require active information gathering and coordinated tool use, which cooperative-user evaluations may not fully capture.

All methods are evaluated on held-out tasks using the same frozen controller, language renderer, and environment verifier. However, this controlled setting does not exclude advantages arising from similarity to ATOA's training distribution. We therefore conduct an additional evaluation with an independent user simulator.

Independent-user evaluation. We use Llama-3.3-70B-Instruct to evaluate the frozen Qwen3.6-27B Base, Fixed-user, and ATOA agents on 31 held-out Retail cards, with eight trials per card (248 interactions per method). Unlike our scripted user, Llama independently determines its responses, information disclosure, and conversational behavior from the task goal and relevant private information, without accessing ATOA's controller, disclosure schedule, or patience rules.

Task goals, initial environment states, evaluation budgets, and mechanical verification remain identical across methods. The independent simulator is never used during training, no agents are further optimized, and generated interactions are checked for task consistency.

Table R1. Cross-user evaluation on Qwen3.6-27B Retail

Evaluation userBaseFixed-userATOA
Independent Llama-3.3-70B user42.3446.3752.42

ATOA achieves 52.42% success, outperforming Base and Fixed-user by 10.08 and 6.05 percentage points, respectively. This provides evidence that its gains extend beyond the original scripted interaction protocol.

Unlike Table 16, which evaluates the official cooperative GPT-4.1 user on 67 Retail tasks with six trials each, this experiment evaluates an independent user policy without ATOA's controller.

ATOA aims to improve robustness to challenging users while preserving effectiveness with cooperative users. On Retail, ATOA improves success from 25.32% to 47.84% (+22.52 pp) under challenging scripted interactions, while decreasing from 80.85% to 75.50% (-5.35 pp) under cooperative GPT-4.1 evaluation. Airline similarly decreases by 5.25 points, whereas Telecom improves by 27.50 points.

Together, these results demonstrate substantial gains under challenging interactions while retaining considerable cooperative-user capability. The independent-user evaluation further supports the transferability of ATOA's improvements beyond its training opponent.


W2. Does restricting adaptation to language limit higher-level adversarial strategies?

ATOA keeps user decisions fixed to prevent changes to task requirements or exploitation of the success criterion. However, fixed behavioral rules do not imply fixed interaction trajectories. Failure-guided language adaptation can influence the agent's questions, tool calls, and subsequent decisions.

Our results support this mechanism. In Retail, opponent adaptation increases mixed-outcome cards from 23 to 31 and pairing potential from 45 to 61. From the same Round-2 checkpoint, training with the adapted user improves agent success from 42.92% to 44.85% (Table 11).

Additional evaluation with dynamic subgoal priorities. We evaluate Qwen3.6-27B on held-out Retail tasks with independently reorderable subgoals. An alternative user policy dynamically changes subgoal priorities based on agent progress, while preserving task requirements and environment verification. We compare frozen Base, Fixed-user, and ATOA agents under both policies using identical task cards and eight trials per card.

Table R2. Robustness to dynamic subgoal prioritization

AgentOriginal user policy (%)Dynamic-priority user (%)
Base35.0027.50
Fixed-user39.3832.50
ATOA51.2543.75

Although dynamic priorities make the tasks harder for all agents, ATOA retains the highest success rate, outperforming Fixed-user by 11.25 percentage points. This supports the transfer of learned capabilities beyond the original interaction sequence.

ATOA's goal is not unrestricted adversarial adaptation, but effective failure-guided training under valid task constraints. The additional results demonstrate its robustness to higher-level changes in user priorities.

AuthorsOpenReview ↗

Response to Reviewer 6rRA (2/2)

W3. How are the candidate-selection criteria justified, and how sensitive is ATOA to their thresholds?

The acceptance rule in Eq. (16) preserves useful training signals while filtering unhelpful opponent updates through three conditions:

  • Mixed-outcome cards (+5%): Identify additional tasks with both successful and failed attempts.
  • Pairing potential (non-decreasing): Preserve same-task success–failure pairs for DPO.
  • Agent success (≤2 pp increase): Reject candidates that become easier without exposing new weaknesses.

These are practical safeguards, not theoretically optimal thresholds. For our 34-card Retail selection split, the 5% threshold requires at least two additional mixed-outcome cards, while the 2pp tolerance accommodates minor success-rate variation.

Additional threshold sensitivity experiment. We conduct a controlled Retail Round-2-to-Round-3 continuation with Qwen3.6-27B. Starting from the same agent checkpoint, we generate user candidates from identical failure-derived data using different SFT seeds, then evaluate and screen them in a fixed order on the same selection cards and trial seeds under alternative thresholds.

For each setting, the selected user is used for one agent-training round with matched interaction and RAFT/DPO budgets. We evaluate the resulting agents on the same validation cards, varying only the mixed-outcome threshold or the permitted pass-rate increase.

Table R3. Candidate-selection sensitivity on Qwen3.6-27B Retail

Mixed-outcome thresholdPass-rate toleranceR3 validation success (%)
2.5%2 pp44.85
5% (ATOA)2 pp45.22
10%2 pp44.49
5%0 pp44.12
5%5 pp44.49

The results show that performance remains relatively stable under moderate threshold changes, with validation success ranging from 44.12% to 45.22%. This suggests that ATOA's effectiveness does not depend on a narrowly tuned cutoff.

The importance of candidate selection is further supported by our existing Accept All Updates ablation (Table 2), which reduces Retail success from 47.84% to 45.54%, with similar reductions in Airline and Telecom.

The key contribution is not the particular thresholds, but the selection principle: opponent updates should expose more informative failures without reducing usable preference supervision or making the task easier. The thresholds can be adjusted to the available task budget, while the underlying criteria are not specific to any particular τ2\tau^2-bench scenario.


W4. How computationally expensive is ATOA, and can the rollout budget be reduced?

The main computational cost of ATOA comes from collecting multi-turn interactions, rather than updating the user model. Appendix F.3 reports 162.3 GPU hours for interaction collection (69.2%), 19.1 for user adaptation (8.1%), and 53.3 for agent training (22.7%), totaling 234.7 GPU hours, including the final stopping probe. The selected Round-4 agent is obtained after 195.7 GPU hours.

Compliance checking introduces little additional generation overhead. Our audit of 294,871 user utterances reports only 0.081% initial violations and zero deterministic fallbacks, indicating that repeated generation is rarely required.

Additional experiment: Rollout budget sensitivity. We evaluate whether fewer screening trials can identify informative failures. From the same Qwen3.6-27B Retail Round-2 checkpoint, we compare 2, 4, and 8 trials per card using nested random seeds. Each setting adapts the user and trains the agent for one round under matched optimization budgets. Candidate selection and final evaluation use the same validation set with eight trials per card.

Table R4. Rollout budget sensitivity on Qwen3.6-27B Retail

Screening trials/cardScreening rolloutsReductionR3 validation success (%)
216075%43.38
432050%44.49
8 (ATOA)640—45.22

Table R4 highlights the trade-off between screening cost and performance. Reducing eight trials to four halves screening interactions, with only a 0.73-percentage-point drop in task success. Using two trials saves 75% of interactions, but increases the performance gap to 1.84 points.

These results suggest that ATOA can retain most of its training benefit with fewer screening rollouts, indicating that the eight-trial setting is not essential for effective opponent adaptation. We thank the reviewer for highlighting this efficiency issue, which points to a promising direction for future work: adaptively allocating rollout budgets to informative tasks, rather than using a fixed number of trials for every task.


We thank the reviewer for the constructive feedback. We hope our clarifications and additional experiments address the main concerns and strengthen the evidence for ATOA. We would sincerely appreciate your reconsideration of the rating.

Thank you for the additional results and clarifications in the rebuttal.

  • Could the authors please explain about my point regarding the Telecom benchmark modification? I think modifying the Telecom protocol that assumes user-data tool execution is error-free changes the main challenge of thee dual-control Telecom domain of tau^2-bench.
  • Thank you for the Table R2 results. It is a nice robustness result. However, I’m still concerned that during ATOA training, the opponent still cannot learn higher-level adversarial strategies.
  • Thank you for the threshold sensitivity experiment and the rollout budget sensitivity experiment. Regarding R4, does it not only test reducing screening rollouts for a single round? The way I understand, in practice ATOA still requires collecting rollouts multiple times per round plus multi-stage training (SFT + RAFT + DPO) across 4-5 rounds, and then requires running an extra round to trigger the stopping rule. Also, it would be beneficial to see wall clock or GPU-hour comparisons against the Table 1 baselines to be able to see the computational expense of the additional gain offered by ATOA.

We thank the reviewer for the thoughtful follow-up and for recognizing our additional results. We address the remaining concerns below.


1. Telecom Protocol and Dual-Control Evaluation

We agree that assuming correct execution of user-side operations simplifies one aspect of the original τ2\tau^2-bench Telecom challenge. Our objective is to isolate the agent's multi-turn coordination ability under controlled user behavior, rather than evaluate user-side tool-execution errors. The agent must still identify device operations, communicate instructions, interpret feedback, and coordinate subsequent actions. Task completion remains independently verified.

All Table 1 methods follow the same protocol, ensuring consistent comparisons within our setting. We acknowledge that these results are not directly comparable to the original Telecom benchmark scores, as clarified in Section 4.2 and Appendix A.3.


2. Why Restrict Higher-Level Adversarial Adaptation?

We agree that ATOA does not independently learn higher-level user strategies. This restriction is motivated by the risk of reward hacking in unconstrained bilateral training.

Without constraints, the agent and user may exploit shortcuts rather than genuinely complete tasks. For example, the user may prematurely declare success while the agent learns to trigger such responses, or the user may refuse essential information regardless of the agent's actions. Training rewards can thus increase without improving actual task success (Figure 1(b)). Similar failures are reported in SEAD (Dai et al., 2026, Table 3), where adversarial user training reduces task completion from 47.7% to 35.2% and user humanness from 4.6 to 3.3.

We therefore maintain independent verification and fixed behavioral constraints. Importantly, fixing behavioral rules does not fix the dialogue itself. The user model can adapt how it expresses directives, leading to different agent responses and interaction trajectories. Our additional dynamic-priority evaluation (Table R2) further demonstrates transfer to different behavioral policies.

Our goal is to develop a stronger agent through increasingly informative interactions, without allowing the opponent to alter the task or its success criterion.


3. Full Training Cost and Efficiency

We agree that Table R4 measures single-round screening sensitivity, rather than end-to-end cost reduction. Its reported savings apply specifically to screening interactions.

We summarize the complete Qwen3.6-27B Retail computational costs from Appendix F.3.

Table A. ATOA training costs (GPU-hours)

StageCollectionUser SFTRAFT+DPOCumulative
Preparation—3.6—3.6
R140.2—13.056.9
R233.11.115.0106.1
R329.93.711.8151.4
R4 (selected)27.33.513.5195.7
R5 (not retained)31.87.213.4248.1

The selected R4 checkpoint achieves 47.84% Retail success (+22.52 points over Base) at 195.7 GPU-hours. Including the additional R5 round for stopping, the complete run costs 248.1 GPU-hours.

We did not record detailed runtimes for every baseline during the experiments. We therefore made rough estimates based on the start and end times of their training runs.

Table B. Computational comparison on Qwen3.6-27B Retail

MethodGPU-hoursSuccess (%)
Fixed-user training~12530.43
Divergence-Point DPO~14043.20
SEAD~16540.20
ATOA (selected R4)195.747.84
ATOA (including R5 stopping)248.147.84

As expected, simpler methods require less computation. However, ATOA achieves higher Retail success than Fixed-user (+17.41 points), Divergence-Point DPO (+4.64 points), and SEAD (+7.64 points), demonstrating a meaningful performance–computation trade-off.

To further verify that ATOA's gains come from more informative training signals rather than additional optimization, we also conducted a budget-matched Telecom experiment (Table 13). Starting from the same agent checkpoint, we compared training with the previous and adapted users, keeping 63 preference pairs and 31 DPO updates fixed. The adapted user improves success from 83.12% to 85.76% (+2.64 pp), demonstrating the benefit of opponent adaptation under the same optimization budget.


We sincerely appreciate the reviewer's continued engagement and constructive feedback. We hope these additional results and clarifications have addressed your remaining concerns and will be helpful in reconsidering your overall assessment. We would be very happy to address any further questions.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the Area Chair and Reviewers

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all four reviewers (6rRA, tAWa, vKSM, and mSEE) for their careful assessments and constructive feedback. During the rebuttal, we provided additional experiments, controlled comparisons, and clarifications, together with manuscript revisions highlighted in blue. Below, we summarize the main concerns and how we have addressed them.


1. Cross-User Generalization and Cooperative-User Trade-offs

Reviewers: 6rRA, tAWa, mSEE

  • Independent-user evaluation. With an independent Llama-3.3-70B simulator that does not use ATOA's controller, ATOA achieves 52.42% Retail success, outperforming Base (42.34%) and Fixed-user (46.37%). We also evaluated alternative behavioral policies, including proactive disclosure, delayed disclosure, and dynamic subgoal priorities.
  • Robustness–cooperation trade-off. ATOA prioritizes robustness to challenging interactions while preserving substantial performance with cooperative users. On Retail, it achieves a 22.52-point gain under challenging scripted interactions, while retaining 75.50% success with cooperative GPT-4.1 users, a 5.35-point decrease from Base. Airline exhibits a similar trade-off, whereas Telecom improves under both settings (+13.50/+27.50 points). These results highlight the value of challenging-user training alongside domain-dependent trade-offs.

These clarifications are reflected in Appendices F.1–F.2, with further cross-user evidence provided in our detailed responses.


2. Effectiveness and Scope of Opponent Adaptation

Reviewers: 6rRA, vKSM

  • Controlled adaptation. Fixed user decisions do not imply fixed interaction trajectories. Opponent adaptation increases Retail mixed-outcome cards from 23 to 31 and pairing potential from 45 to 61, providing more informative training signals.
  • Direct comparisons. From the same agent checkpoint, adapted-user training improves Retail success from 42.92% to 44.85%. Budget-matched Telecom experiments show an additional 2.64-point gain. We also provided a DACT-inspired comparison to further examine the contribution of failure-guided adaptation.

These results support the value of updating opponents based on current agent failures while preserving task validity.


3. Checkpoint Selection and Statistical Reliability

Reviewer: tAWa

  • Independent selection protocol. We clarified disjoint Training, Validation, and Test sets. User acceptance and checkpoint selection use validation data, while held-out test results are excluded from these decisions.
  • Stopping and uncertainty. Retail R5 completes RAFT/DPO training, but validation success decreases from 48.16% to 47.43%, justifying the selection of R4. We also provided paired uncertainty analyses and additional training-seed evidence for the controlled comparisons.

The revised procedures and computational accounting are documented in Appendices A.1, C.5, and F.3.


4. Candidate Selection and Computational Efficiency

Reviewer: 6rRA

  • Threshold sensitivity. Across alternative candidate-acceptance thresholds, Retail validation success remains between 44.12% and 45.22%, supporting the robustness of the selection criteria.
  • Rollout efficiency. Reducing screening trials from eight to four halves screening interactions with only a 0.73-point performance decrease. Using two trials saves 75% of screening interactions, with a 1.84-point decrease.

These additional analyses are included in Appendices C.4 and F.3.


5. Agent Optimization and Domain Generality

Reviewers: vKSM, mSEE

  • RAFT and DPO contributions. We provided Post-RAFT and Post-DPO evaluations across training rounds, complementing the existing ablations and clarifying their respective contributions.
  • Broader evaluation. We explained the limited training signal in Airline and provided an additional Banking-domain evaluation. On Qwen3.6-27B, ATOA exceeds the strongest competing baseline by 4.64 points on Retail and 4.65 points on Telecom.

We hope these additional experiments and clarifications provide stronger evidence for ATOA's effectiveness, generalization, and the benefits of failure-guided opponent adaptation, while making its limitations and evaluation scope clearer.

As the rebuttal period is approaching its end, we would be very happy to address any remaining questions or concerns from the reviewers. We sincerely appreciate your time and consideration, and hope our responses and revisions will support a favorable reassessment of our work.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.