acceptodds
Under review as a conference paper at ICLR 2027

ARGUS: An Agenda-Driven Framework for Reliable User Simulation

Abstract

High-fidelity user simulators are essential for evaluating and training agents in interactive environments. However, current LLM-based simulators struggle with reliability issues, such as goal omission and policy non-compliance. We propose ARGUS (Agenda-dRiven Goal-oriented User Simulator), a neuro-symbolic framework that maintains the user's goal state using a dynamic agenda stack manipulated via symbolic operations. By rigorously decoupling state tracking from natural language generation, ARGUS facilitates strict goal progression and constraint-aligned interactions. For cost-efficient deployment, we distill ARGUS into compact models using a unified pipeline: evaluator-guided SFT followed by online RL with dense step-level rewards. To rigorously evaluate behavioral fidelity, we introduce a rubric-based metric Composite Behavioral Score (CBS). Evaluations on -Bench and MultiWOZ demonstrate the significant superiority of the ARGUS architecture and the efficacy of our training pipeline. Code and data are available at https://anonymous.4open.science/r/ARGUS-3BB7.

Primary area:
foundation or frontier models, including LLMs
Keywords:
User Simulator, Task-Oriented Dialogue, Neuro-Symbolic Agent, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_eevMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies reliability failures of LLM-based user simulators — goal omission, constraint violations, and premature termination in long-horizon interactive tasks. It proposes ARGUS, a neuro-symbolic user simulator that decouples state management from response generation: the user's goals are maintained in an explicit LIFO agenda stack manipulated through constrained symbolic operations (PUSH, POP, MODIFY, WAIT) emitted as tool calls by a state controller, while a separate generation LLM produces the user utterance conditioned on the updated agenda. To enable cost-efficient deployment, the authors distill prompt-based ARGUS into compact 4B/8B models via a two-stage pipeline — evaluator-guided SFT (a step-level evaluator scoring Soundness, Faithfulness, and Adherence filters teacher rollouts) followed by online RL with dense step-level rewards (an adapted DAPO variant with balanced Stack/NLG groups). They also introduce the Composite Behavioral Score (CBS), a rubric-based metric that exposes procedural violations missed by outcome-centric metrics. Validation on τ³-Bench (Retail, Airline) and MultiWOZ across six backbones shows substantial CBS gains (e.g., +6.7 on Retail over the best baseline with Qwen3-8B), robustness to controlled input representation and prompting constraints (Table 10), a human agreement study, an alternative-judge analysis, and appendix studies on goal extractors and data filtering. The authors additionally analyze "illusory successes" — task-successful dialogues that still contain simulator violations (Section 6.2).

Critical strengths
  1. The motivating insight is correct and important: task success alone is not evidence of simulation quality. The paper articulates clearly that a simulator can help an agent succeed by leaking information, relaxing constraints, or omitting goals, so outcome-centric metrics can certify a false evaluation signal. This reframing justifies the whole neuro-symbolic agenda and the CBS metric, and it is consequential because it challenges how the community currently evaluates user simulators — a simulator that "helps too much" is worse than an obviously weak one. The human agreement study and alternative-judge analysis lend credibility to CBS as more than a self-defined metric.
  2. The empirical case is unusually well-controlled for this literature. The improvements on Retail are large and consistent across six backbones, and — unlike most prompt-strategy comparisons — the authors control for structured inputs, prompting constraints, goal extractors, and data filtering in the appendix, so the gains cannot be dismissed as better prompting. The training pipeline (evaluator-guided SFT + online RL) that distills the architecture into 4B/8B models while advancing the cost-quality Pareto frontier is a practical contribution with clear deployment value. These controls are what make the central claim (that explicit agenda state tracking improves simulator reliability) believable, and they are the main reason I find the paper promising despite my concerns.
Critical weaknesses
  1. The gains are not attributed to the LIFO stack, which is the paper's core architectural claim. ARGUS bundles several changes at once — two-stage generation, external state, local update operations, and LIFO scheduling — and none of the experiments isolates the stack from the rest. A controller with the same external state and local operations but unrestricted selection among pending tasks might achieve similar results, in which case the contribution lies in constrained state updates rather than stack structure. This also has an applicability dimension: real task-oriented conversations interleave transactions and agents ask about goals not at the top of the stack, so the necessity of strict LIFO is an empirical question the paper never tests. Since the stack is the headline contribution, the absence of a non-LIFO control with other components held fixed leaves the paper's central design choice unsupported by its own evidence.
  2. The evaluation interpretation overreaches: CBS measures procedural compliance, but is treated as evidence of behavioral realism and of distorted agent evaluation. Section 6.2 labels every successful dialogue with any CBS violation an "illusory success," yet the violations include things like an incorrectly formatted termination marker that plausibly did not help the agent at all — the analysis establishes that successful dialogues can contain violations, not that those violations caused or enabled success. Similarly, the Table 14 example penalizes a user for not restating an expensive-hotel preference before choosing the cheaper option — a script-adherence failure at most, not evidence of unrealistic behavior. Relatedly, RL trades errors rather than uniformly improving reliability: Goal Completion drops from 80.1 (SFT, 20k) to 74.9 (RL) (Appendix K), which deserves prominent discussion given that reducing goal omission is a central motivation. These issues are consequential because the paper's framing stakes its contribution on improving "behavioral fidelity" and agent evaluation quality; without separating violation types by their causal impact on task difficulty, the strength of that claim is not established.
Questions for authors

Prompt 9 instructs the controller to use modify_nearest_main_task, while Prompt 3 (the evaluator) checks modify_top_task and marks modifications to a non-top frame as incorrect. These are not equivalent when a temporary subtask sits on top of its main task. This may be an outdated appendix entry, but it determines which transitions are kept for SFT and rewarded during RL. Which tool definitions and evaluator prompts were actually used in the reported experiments, and could you provide a complete worked example of a temporary subtask with a conditional update to its main task? If this is only a documentation issue, a correction suffices; if not, the training signal itself needs re-examination.

Would a variant of ARGUS with the same external state and local update operations but unrestricted (non-LIFO) selection among pending frames achieve similar results? A single such control would establish whether the contribution lies in the stack structure itself or in constrained state updates more broadly. If the stack is essential, a small set of task-interleaving tests (where the agent asks about a goal that is not on top) would also demonstrate robustness of the strict top-frame priority to realistic dialogue patterns. Answers here directly determine whether the paper's headline architectural claim is supported.

RL improves Faithfulness and Policy Adherence but lowers Goal Completion from 80.1 to 74.9. Could you report premature-termination rates and the proportion of dialogues expressing all required goals for SFT vs. RL from the same checkpoint, with uncertainty estimates? Additionally, UGST-RL trains on pre-sampled interaction states while ARGUS-RL collects states from a live environment — is the reported difference attributable to the architecture or to the state-distribution/training-condition difference?

Reviewer_wBZZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the problem of behavioral fidelity in user simulators. Current user simulators struggle to maintain goal progression and adhering to their goal. They build upon prior works that introduce user goal states to help track progress and train better simulators.

Prior works frame state transitions as a text generation task, which can be noisy. Instead, the authors propose using a neuro-symbolic framework that tracks the user's goal state using a dynamic agenda stack manipulated via symbolic operations Using this framework, they present ARGUS: Agenda-Driven Goal-oriented User Simulation. They also present a training pipeline to distill the ARGUS framework into a single model, which involves SFT and RL.

They use task success rate metrics and a rubric-based metric to measure behavioral fidelity in user simulators, and demonstrate that their approach leads to better behavioral fidelity for simulators for tau^3 bench and MultiWOZ.

Critical strengths

The paper provides a clear description of their method, and it is well motivated. Moreover, they present a thorough evaluation of user simulators, with a good amount of baselines and evaluation metrics. Moreover, the authors present additional analyses such as simulator error rates across different dimensions, cost-quality trade-off and specific error patterns.

Critical weaknesses

It seems like the paper’s main argument is that goal state tracking should be done in a neuro-symbolic manner rather than in a textual generation manner, since it helps reduce the semantic drift and parsing errors. While there are end-to-end evaluations of user simulators, there are no evaluations of the goal state tracking with the different methods. Such an analysis would be beneficial for directly proving the author’s main claim. For example, the authors can evaluate the goal state tracking directly, instead of evaluating the simulator, and show that ARGUS goal state tracking has fewer errors compared to UGST.

In the training pipeline, SFT data is collected by collecting responses that achieve a perfect score on three dimensions. However, it is not clear whether this same approach was used to generate data for the NAIVE-SFT and UGST-SFT baselines. It may not be a fair comparison if it wasn’t, and conducting such ablations would be useful.

Questions for authors

Are prompts for the baselines provided anywhere? This information, along with more details about how baselines are implemented would be helpful for understanding the paper's experiments.

Reviewer_ePUEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper is about a framwork called ARGUS for LLM-based user simulation. There’s a stack of user goals with state updates through natural language. They also show how SFT and RL can improve user simulator models. In addition, they proposes evaluation metric “CBS” for measuing the quality of the user simulators in following user goals/constraints.

Critical strengths
  1. Separating state management from response generation using an explicit agenda stack is intuitive and reasonable for ensuring adhering to user goal and constraints.
  2. The evaluation is extensive. The paper evaluates multiple model backbones and sizes, domains, and the ablation studies are comprehensive.
Critical weaknesses
  1. The results are mostly based on “CBS”, the metric newly proposed in this paper. Although human validation is provided, the agreement is relatively low (Table 7). Given that the training pipeline also uses the same CBS metrics, it is unclear how much the improvements reflect better user simulation rather than better alignment with the evaluation metric.
  2. It is unclear how much improvement comes from the LIFO stack rather than the additional reasoning and state management steps because ARGUS introduces several changes at the same time (e.g., structured goal representation, two-step generation, symbolic stack operations). A comprehensive ablation study would be helpful.
Questions for authors
  1. Since LLM-judged CBS metric has low human–LLM agreement rate, how reliable are the reported improvements in CBS? If you use human evaluation instead, would it show similar improvements?
4 comments
AuthorsOpenReview ↗

[1/2] Re to Reviewer ePUE

We sincerely thank the reviewer for the positive assessment of our work and the constructive feedback. We address the two concerns below.

Re to W1: Reliability of CBS

First, we would like to clarify that our training and evaluation do not use the same evaluator or metric. During SFT and RL, we use GPT-5.1 as a step-level evaluator, which assesses individual stack operations and response generation steps along three dimensions: Soundness, Faithfulness, and Adherence. In contrast, the final CBS evaluation uses Qwen3-MoE-235B to assess complete dialogues across 12 behavioral subcriteria. Although both are designed to encourage reliable user behavior, they differ in the judge model, evaluation granularity, and scoring criteria. Therefore, our training does not directly optimize the test-time CBS metric.

Regarding the reliability of CBS, we provide human validation results in Appendix E.2 (Table 7). We evaluated 90 dialogues with five expert annotators and found strong overall human–human agreement (Gwet's AC2 = 0.861) and human–Qwen3-MoE-235B agreement (Gwet's AC1 = 0.812). The overall results suggest that our primary judge aligns well with human annotations. We also tested an alternative judge, DeepSeek-V4.1-Flash, on 24 method–backbone combinations in Retail. The two judges show strong score correlations (Pearson = 0.873, Spearman = 0.886) and agree on 91.67% of method-pair rankings, with an aggregate Kendall's τ of 0.966. This suggests that our main conclusions are not particularly sensitive to the choice of judge.

We agree that a full-scale human evaluation would provide further evidence, but annotating all experimental dialogues would be very costly, especially given the large number of interactive rollouts. Our current human validation and cross-judge analysis provide complementary evidence for the reliability of CBS, although they cannot fully replace a direct human comparison of all methods.

Re to W2: Contribution of the LIFO Agenda Stack

We agree that ARGUS combines several design choices, and it is important to understand where the improvements come from. First, we have already included controlled experiments in Appendix G.2 (Table 10) to examine the effects of structured goal representations and additional prompt constraints. We augment Vanilla and ReAct with the same parsed goals used by ARGUS, matched behavioral instructions, or both. These changes improve the baselines in several settings, but ARGUS still achieves the highest CBS across all four backbone–domain combinations. These results suggest that structured inputs and more detailed prompts alone cannot explain ARGUS's improvements. However, they do not isolate the contribution of the LIFO constraint itself.

To investigate this more directly, we conducted an additional LIFO vs. Non-LIFO experiment on Retail using four Qwen3 backbones. The Non-LIFO variant retains the structured goal representation, two-stage generation, and symbolic state updates, but removes the restriction that operations must target the top task. Instead, the controller can modify or remove tasks at arbitrary positions and must explicitly select the current task focus. We kept the other implementation details, including the prompt structure and examples, as comparable as possible.

The results are shown below (Δ = Non-LIFO − LIFO).

User ModelLIFO TSRNon-LIFO TSRΔ TSRLIFO CBSNon-LIFO CBSΔ CBS
Qwen3-8B51.5848.95−2.6372.5074.30+1.80
Qwen3-30B-A3B61.0558.77−2.2885.6282.21−3.41
Qwen3-32B57.5461.23+3.6983.9884.36+0.38
Qwen3-235B-A22B64.3963.16−1.2391.3088.56−2.74
Macro Average58.6458.03−0.6183.3582.36−0.99

Overall, the two variants achieve comparable performance, with LIFO showing a small average advantage in CBS and TSR. The overall CBS difference is not statistically significant (the bootstrap estimate gives a 95% CI of [−2.15, +0.17] pp). Non-LIFO performs better on some backbones, so we do not view LIFO as universally superior. We believe one advantage of LIFO is that the top frame naturally serves as the current task focus, allowing the model to track task progression without explicitly deciding which pending task to handle at each turn. In contrast, Non-LIFO provides more flexibility but also requires more complex task selection and lifecycle management. In our case analysis, we observed instances where Non-LIFO prematurely removed unfinished tasks or left completed tasks in the agenda, leading to repeated interactions. This may partly explain its lower Goal Completion score (87.72 vs. 89.30) and Format & Protocol score (91.80 vs. 94.30). The difference in Format & Protocol is statistically significant (−2.50 pp, 95% CI [−3.95, −1.05]).

AuthorsOpenReview ↗

[2/2] Re to Reviewer ePUE

We also looked more closely at cases where LIFO could be limiting. Strict LIFO may be less flexible when an agent handles pending tasks out of order. However, this situation is relatively uncommon in our current evaluation setting, since user simulators generally reveal tasks sequentially, and the agent typically cannot act on later tasks before they are disclosed. Among the 114 Retail tasks, only 11 explicitly require multiple goals to be disclosed together, and 7 of these represent the goals within a single main-task frame. We inspected all 220 original LIFO dialogues for these 11 tasks (4 models × 5 trials). Only four tasks contain multiple main-task frames, corresponding to 80 dialogues. Among these, 19 dialogues involved a later task being completed before an earlier one, and we identified 6 incorrect POP operations caused by task-order mismatches.

In the original LIFO implementation, pop_task always removes the top frame, even if the controller's reasoning refers to another task. When a later task is completed first, the controller therefore needs to preserve it until the preceding frames can be safely removed. This sometimes works, but can also cause incorrect task removal. Interestingly, in the corresponding Non-LIFO runs, five of these six cases correctly removed the intended tasks, while one prematurely removed all tasks. This confirms that Non-LIFO can indeed offer more flexibility when task completion happens out of order.

Overall, we do not claim that LIFO is strictly necessary or always better than Non-LIFO scheduling. Our experiments suggest that ARGUS's main advantage comes from explicit agenda state management and constrained symbolic updates, while LIFO provides a simple and useful scheduling bias for the mostly sequential workflows studied here. Non-LIFO achieves comparable overall performance and may be more suitable for settings with frequent task interleaving. We will clarify this distinction and discuss the limitations of strict LIFO in the revised paper.

We thank the reviewer again for the helpful suggestions, which have helped us better understand and clarify the contributions of our framework.

Reviewer_ePUEOpenReview ↗

Response to rebuttal

Thank you for the clarification and additional experiments. They address my concern about directly optimizing CBS. The human agreement results and the additional judge comparison are also helpful. A direct human comparison between methods would still strengthen the evidence, but my concern is reduced.

The LIFO vs. Non-LIFO experiment addresses my second concern. Since the overall performance is similar, I agree with the more limited claim about the role of LIFO. Please include this experiment and the discussion of its limitations in the revised paper.

Overall, the response addresses most of my concerns. I will keep my positive rating at 3 (weak acceptance).

Thank you for taking the time to review our response and for your encouraging feedback! We're glad that the additional experiments have addressed most of your concerns. We will incorporate the LIFO vs. Non-LIFO results and clarify the limitations in the revised paper, as suggested. We really appreciate your thoughtful comments throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.