acceptodds
Under review as a conference paper at ICLR 2027

GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus

Abstract

Training LLM agents for policy-constrained tool environments, such as an airline backend with booking, refund, and cancellation policies, requires tasks that exercise the environment's rules individually and in combination. Existing synthesis pipelines let a model propose tasks and retain only those it immediately solves, biasing training data toward easier behaviors. We introduce Gradus, which decouples what a task should teach from the concrete task and solution used to teach it. Without seed tasks, Gradus treats a fixed environment as a latent syllabus, extracting guarded behavioral operators that capture how actions depend on current conditions and how their effects reshape subsequent decisions. It composes these operators into increasingly demanding tasks while using corpus-level feedback to target missing outcome combinations. Crucially, Gradus jointly calibrates tasks and solutions against the environment: execution feedback identifies which side to refine while preserving the intended behavioral requirements. Verification thus becomes a recovery signal rather than a terminal filter, coupling coverage-guided exploration with the preservation of difficult examples. From off-the-shelf backbones, the resulting curriculum lifts a 30B model from 59% to 89% on -Bench and from 47% to 83% on CAR-Bench with an Opus 4.5 proposer, and to 76% and 62%, respectively, with an open-weight Qwen3-235B proposer. These results show the value of jointly controlling which behaviors are explored and which survive into verified training data.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Synthetic Task Generation, Curriculum Synthesis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_taChOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper addresses the problem of synthesizing training data for tool-use agents operating in policy-constrained environments (such as airline, retail, and telecom customer service in τ2\tau^2-Bench, as well as in-vehicle assistants in CAR-Bench). Specifically, the authors extract Guarded Operators from the target environments and maintain a global ledger to progressively evolve task blueprints along both structural dimensions (e.g., causal dependencies, parallelism, and repetition) and interaction dimensions (e.g., ambiguous references and user pressure). To ensure data quality, the framework incorporates an "initialization–simulation–verification–diagnostic routing" closed loop that performs bidirectional refinement—modifying either the task specification or the solution side—for candidate instances that fail verification. Empirically, the authors synthesize approximately 1,000 instances each on τ2\tau^2-Bench and CAR-Bench to fine-tune the Qwen3 model family, and further construct "Frontier Exams" from deeply evolved tasks to benchmark frontier models including Opus 4.5 and GPT-5.2.

Critical strengths
  1. Soundness of Causal Structural Evolution: The approach of leveraging Guarded Operators to formalize precondition activation and blocking relationships among tool calls (e.g., first invoking a rebooking API to upgrade the cabin class in order to unlock business-class refund eligibility, and then executing the refund) is well-motivated. Expanding single-step operations into multi-step causal chains at the action-topology level provides a sensible way to construct structurally complex tasks.
  2. Identifying the Limitations of Single-Pass Rejection Sampling and End-to-End Empirical Evaluation: The paper pinpoints a practical limitation of single-pass rejection sampling—namely, its tendency to discard long-tail compositional tasks—and conducts end-to-end training experiments combining SFT and reinforcement learning (GRPO) across multiple tool-intensive subdomains.
Critical weaknesses
  1. The teaser figure (Figure 1) is visually cluttered, aesthetically unpolished, and uninformative, requiring a complete redesign: Figure 1 is crammed with numerous cartoon character illustrations irrelevant to the academic content, haphazardly piling visual elements together with chaotic information flow and ill-defined module boundaries. Readers cannot discern the core data-structure transformations or the algorithmic pipeline of the proposed method from Figure 1 at all. This severely degrades the presentation quality of the manuscript, and the figure needs to be completely redrawn from scratch.
  2. Synthesizing Interaction Data Without Prior Field Research is Methodologically Unsound: While structural evolution (Sec. 2.3) is logical, jumping directly into synthesizing interaction difficulty (DECORATE(z,σ)\text{DECORATE}(z, \sigma); Alg. 1, Line 18) without first conducting field research on real user behavior is fundamentally unreasonable. Instead of grounding failure modes in empirical customer service studies, the authors rely on armchair heuristics (ambiguity, user pressure), such as mechanically scripting users to bundle invalid claims and "push back twice" (App. A.7). In reality, support friction stems from cognitive asymmetry between users and system policies (misunderstanding tiered rules), accidental misoperations needing rollback, or genuine intent shifts upon discovering hidden constraints, rather than deliberate ambiguity or scripted adversarial pestering. Generating data in a vacuum without field research inevitably distorts the task distribution, as confirmed by the authors' own human audit (App. A.9, Tables 11 and 12): 73.4% (Airline) and 53.9% (Retail) of evolved tasks are judged Unusual or Contrived (with only 26.6% and 46.2% rated Natural, and Airline Gwet's AC1=0.399AC_1 = 0.399).
  3. Flawed task-side recovery: unrealistic sandbox assumptions and unchecked task simplification. While using an LLM judge to trigger task re-initialization is acceptable, the mechanism has two flaws: (1) Recovering "contradictory states" is an artifact of an unguarded sandbox: In App. A.7 (L997, L1089), a "flown segment" task is wrongly grounded to a cancelled flight (HAT277) yet passes REPLAY(ref, s0) (Alg. 1, L25) only because "the API does not enforce the flown-portion... requirements" (L944). Real-world production software enforces strict state-machine invariants at the API boundary, forbidding such buggy states outright. (2) Unchecked difficulty dilution during rewriting: Alg. 1 updates coverage ledger LL statically in Stage 2 (L820, L831) and never verifies in Stage 3 (L841–850) whether the rewritten (task, ref, s0) still covers the target branches ϕ(z)\phi(z). In App. A.7, recovery explicitly allows altering ref to match alternative trajectories (L1098) and resolves disagreement (Table 7, L1105–1136) by appending exclusionary prompt constraints (banning basic-economy and connecting flights) until an open search collapses into a trivial rejection (agent_db_match=true, L1121).
Questions for authors
  1. Benchmark overlap & decontamination: What fraction of test tasks in τ2\tau^2-Bench and CAR-Bench share an identical state-changing tool sequence or (policy, outcome) branch set with the ~1,000 synthesized training tasks? How much performance remains after removing all training examples that share the same tool-call graph or rule path with any test instance?
  2. Preventing difficulty dilution in task recovery: Since Ledger LL is updated statically in Stage 2 (Alg. 1, L820, L831), why is there no check in Stage 3 (L841–850) verifying that a rewritten (task, ref, s0) still covers the target branches ϕ(z)\phi(z)? Given that App. A.7 permits altering ref to fit alternative trajectories (L1098) and resolves mismatches by appending exclusionary prompt constraints (Table 7, L1105–1136), how do you prevent systematic task simplification? Also, are the difficulty labels in Fig. 3(c) assigned statically from pre-rewrite blueprints or re-evaluated post-rewrite?
  3. Realism of interaction variation: Are the interaction kernels grounded in any real customer-service logs or field research? How do you explain that 73.4% (Airline) and 53.9% (Retail) of evolved tasks are rated Unusual or Contrived by human auditors (Table 11), and why were realistic user issues (e.g., policy misunderstanding, accidental action rollback) omitted in favor of scripted nagging ("push back twice")?
  4. Updated frontier baselines & Figure 1: Can you evaluate the Frontier Exams on newer frontier models (e.g., GPT-6 / GPT-5.4, Claude Opus 5.5, Kimi K3, Gemini 4) and redesign Figure 1 into a clean, readable technical diagram?
9 comments
AuthorsOpenReview ↗

Comment 1/6: On the unguarded APIs and the A.7 example (W3)

We sincerely thank the reviewer for the detailed and thoughtful review, and for recognizing the motivation for guarded operators and for preserving long-tail tasks through recovery. We have addressed the concerns with targeted experiments and a revised figure:

  • Recovery preservation and difficulty (Comment 2). Across the four recovery cohorts, 87.7–100% of accepted tasks preserve the initial state-changing tool-call sequence. Importantly, difficulty is measured on the final tasks after recovery, and the recovered cohorts remain substantially harder than the initially accepted cohort.
  • Interaction design and realism (Comments 3–4). We added the suggested rule misunderstanding and intent shift kernels, retrained with them, and conducted a new human audit of the training corpus, separately from the intentionally challenging Frontier Exams.
  • Train-test overlap/leakage (Comment 5). We measured exact tool-graph overlap and retrained all compared 8B students after removing the overlapping training tasks. GRADUS maintains its advantage.
  • Newer frontier models and Figure 1(Comment 6). We evaluated GPT-6 Astra, Claude Opus 5.5, and Gemini 4 on the frozen Frontier Exams and redesigned Figure 1 in the revised manuscript.

Why the tools are loosely guarded. We agree that production systems enforce invariants at the API boundary. Leaving policy enforcement to the agent is, however, a design choice of the benchmarks rather than of GRADUS. Both τ²-Bench and CAR-Bench deliberately evaluate whether an agent follows a written policy, so their tools accept calls the policy forbids, and GRADUS generates training data for this setting. Many guards would also remain the agent's responsibility even behind strict APIs. They depend on the conversation, such as disclosing options and risks, obtaining explicit confirmation, or resisting a request to bend the policy. A refusal, moreover, issues no call for an API to reject.

The A.7 example shows how a defect in a generated task is recovered. The defect was in the initial task design: it intended a reservation with an already-flown segment (whose un-flown segment therefore cannot be cancelled), but was grounded on flight HAT277, a past flight that had been cancelled by the airline. A past flight being airline-cancelled is not intrinsically an inconsistent backend state; the inconsistency was between the provisional reference's assumption and the actual environment state. Our workflow uncovered and repaired this step by step:

  • Solution-side reflection 1. The agent refused the cancellation without checking the flight status. Its inference was reasonable (the policy fixes today as 2024-05-15, so the 2024-05-10 segment should have been flown), but reflection asked it to verify the status first.
  • Solution-side reflection 2. The agent checked, found that the flight had been cancelled by the airline, and correctly cancelled the reservation.
  • Verification. Verification then exposed the mismatch with the provisional reference and routed the task for re-initialization, which selected a genuinely landed segment.

Only the repaired, verified realization entered the corpus. We will revise A.7 to state which check each step passed.

AuthorsOpenReview ↗

Comment 2/6: On whether recovery dilutes difficulty (W3, Q2)

We thank the reviewer for raising this important point. We answer it with three findings, each backed by data.

A blueprint fixes the intended lesson; recovery repairs its instantiation. Stage 2 specifies the actions, causal dependencies, policy/outcome targets, and interaction requirements; its ledger tracks those blueprint-level targets. In Stage 3, our re-initialization prompt explicitly instructs the LLM to retain these requirements, re-grounding entities and state values or repairing a provisional reference rather than selecting an easier behavioral target. If an intended target cannot be instantiated, initializer declares the task infeasible.

This explains the replacement-flight subtask in Appendix A.7. Its target was refusal of an unavailable booking, but the first request inadvertently allowed connecting and basic-economy alternatives. Recovery made the intended direct-economy constraint and the user's rejection of alternatives explicit. The task still required the upgrade–cancel chain, exception-set cancellation, refusal for a genuinely flown segment, compensation-policy resistance, and a baggage-fee boundary. The change corrects a mismatch between wording and blueprint, rather than abandoning the target for an easier one. Respectively, scenarios that intend for a "accept" case (open search for connecting flights) are initialized and verified in other blueprints, managed by global ledger.

Structural preservation. We compared the accepted recovered tasks with their first realizations using ordered state-changing tool-call sequences:

CohortIdentical write-call sequence before and after recovery
Airline, Round 293.8%
Airline, Round 388.0%
CAR, Round 287.7%
CAR, Round 3100.0%

Thus, 87.7–100% retain the original operational sequence. This is observable evidence of preservation, complemented by an independent measure of the final task difficulty.

Recovery retains the hardest tasks instead of discarding them (Q2). The difficulty labels in the figure the reviewer refers to (Figure 3(c) in the paper) are neither assigned from the blueprints nor used by the pipeline; they are a post-hoc measurement on the final verified tasks, taken after recovery. Each task is run 8 times by the off-the-shelf student (Qwen3-30B-A3B-Thinking, before training) and labelled by its number of successes: Easy (8/8), Medium (5 to 7), Medium-Hard (2 to 4) or Hard (0 to 1). This measurement aligns steadily with evolution depth, from 0.87 at depth 0 to 0.17 at depth 4 (Figure 3(a)), indicating tasks at deeper evolution get harder. Harder tasks are also more likely to fail verification on their first realization, and therefore to need one or more rounds of recovery. Consequently, the tasks accepted in later rounds are markedly harder than those accepted initially (mean zero-shot success 0.31 and 0.27 vs. 0.60; Table R-difficulty). Recovery does not make these tasks hard; it keeps hard tasks that a pipeline without recovery would have discarded, and their measured difficulty after recovery confirms that it does not simplify them.

Table R-difficulty. Difficulty of verified airline tasks, measured after recovery, by the round in which they were accepted.

Accepted inMean zero-shot successMedium-Hard + HardHard
Round 1 (initial)0.6041%28%
Round 2 (recovered)0.3175%49%
Round 3 (recovered)0.2783%52%
AuthorsOpenReview ↗

Comment 3/6: On field research, interaction patterns and the reviewer's suggested behaviours (W2, Q3)

We thank the reviewer and agree that field research on real user behaviour is valuable. GRADUS builds a capability-oriented curriculum over the behaviors permitted or prohibited by a fixed environment; it is not designed to reproduce the distribution of production customer-service requests.

Thus, our contribution is a plug-and-play framework. Any interaction pattern, whether it comes from field studies, domain data or expert knowledge, can be written as a kernel markdown file and composed with the evolution process, without changing the pipeline at all.

To demonstrate this, we introduced two behavior kernels specifically suggested by the reviewer, without changing the pipeline:

New kernelUser behaviourExpected agent behaviour
Rule misunderstandingSincerely misunderstands a tiered or conditional ruleExplain the rule as it applies to the user, then act on the revised request
Intent shift on a discovered constraintLearns of an unknown constraint and genuinely changes the goalConfirm the new goal before acting

These kernels produced tasks that passed the same verification as our original data. We replaced 18.2% of airline and 13.1% of CAR training tasks with them, keeping the training-set size and evolution-depth distribution unchanged. With the same Opus 4.5 proposer and Qwen3-8B SFT recipe:

Table R-new-kernels-results. Training with tasks from the new kernels (same training-set size).

SettingAirlineCAR base
GRADUS 8B (original)0.710.76
GRADUS 8B + new kernels0.700.74

The new kernels integrate without materially changing benchmark performance. To showcase the actual generated tasks, Appendix A.18 gives verified examples: misunderstanding the airline 24-hour cancellation rule and changing an in-car request after discovering that AC is on (please see latest PDF revision). These results demonstrate that empirical findings about user friction can be easily incorporated through writing additional kernels without any code change in GRADUS.

How our current kernels are designed. The original kernels were not mined from customer-service logs, but their role is to produce plausible, verifiable variations rather than claim that any one pattern is statistically typical. They cover two things: how users communicate their needs with varying levels of ambiguity, and what they do when their goal conflicts with the policy. The persistence behavior in Appendix A.7 is an instance of the latter: a user asks again for an exception after a refusal (see "Goal conflict with policy"). Our original taxonomy includes 13 sub-dimensions across indirect parameter values (indirect references, computed values, observed-state dependencies, policy-derived values), underspecified requests (infer from context, infer from stored preferences, need clarification, policy priority), and goal/policy conflicts (incorrect claims, appeals, inconsistent statements, mixed requests). Many address genuine informational asymmetry and ambiguity rather than only adversarial persistence.

In summary, the reviewer's suggestions show how the framework is meant to grow: findings about real user behavior can be readily added as kernels directly.

Reviewer_1upwOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript introduces Graded Rungs of Ascending Difficulty from an Underlying System (GRADUS), which constructs training curricula for agents operating in a fixed tool environment. It extracts descriptions of actions, their policy conditions, and their consequences; combines these into progressively more complex task blueprints and tracks coverage to guide generation. A joint refinement loop repairs tasks, reference solutions, and demonstrations using execution feedback. This manuscript evaluates the resulting data through supervised fine-tuning and reinforcement learning on τ2\tau^2-Bench and CAR-Bench. Its strongest 30B student improves from 59% to 89% average success on τ2\tau^2-Bench and from 47% to 83% on CAR-Bench. Component ablations, coverage analyses, recovery statistics, and human-audited challenge tasks provide additional validation.

Critical strengths
  1. Substantial improvements under meaningful experimental controls. With the same Opus proposer, 8B student, and reported task budget, GRADUS outperforms all three controlled synthesis baselines on every evaluated split. Improvements also occur with an open-weight proposer and across student architectures, which is persuasive evidence that the generated curriculum has practical training value.
  2. A useful integration of coverage and recovery, supported by component evidence. Separating intended behavioral requirements from task instantiation gives the method a principled way to revisit failed examples. The ablations support the importance of this organization. These results substantiate the contribution beyond the headline model scores.
Critical weaknesses
  1. The claimed verification of task and solution correctness is insufficiently established. Section 2.4 accepts examples when reference and demonstration outcomes agree and programmatic policy checks pass. Outcome agreement alone cannot establish that either solution correctly implements the user’s request. The additional policy checks are therefore essential, yet their implementation and coverage remain underspecified. Meanwhile, the human audit illustrates the remaining gap. Only 25/32 airline and 70/79 CAR exam candidates were jointly judged correct after automatic validation. These selected difficult tasks do not estimate the training corpus’s error rate, but they demonstrate that automatic acceptance is not sufficient evidence of semantic correctness.
  2. The comparisons do not adequately separate curriculum quality from additional computational effort and supervision. The paper matches task counts and maximum sequence lengths, but does not report matched synthesis tokens, model calls, or total training target tokens. GRADUS uses parallel attempts, reflection, and repeated task reconstruction, while the controlled baselines lack comparable recovery. The cumulative validity ablation has the same ambiguity: adding recovery also permits further generation and search. Thus, the experiments establish the effectiveness of the complete pipeline more convincingly than the advantage of recovery over spending the same resources on additional sampling or reflection.
Questions for authors
  1. What exactly does policycheck verify, and how are requirements involving confirmation, refusal, and communication evaluated? What prevents an incorrect reference and demonstration that reach the same state from being accepted?
  2. Can the authors provide an independent audit of randomly sampled accepted training examples, stratified by evolution depth and recovery round, measuring both semantic correctness and preservation of the original blueprint’s requirements?
  3. Does GRADUS retain its advantage when total supervised target tokens are matched, including a baseline that receives equivalent sampling and reflection opportunities without task revision?
5 comments
AuthorsOpenReview ↗

Comment 1/4: What verification checks (W1, Q1)

We sincerely thank the reviewer for the detailed and thoughtful review, and for recognizing GRADUS's gains under controlled comparisons and the contribution of coverage-guided generation and recovery. In response, we conducted a new training-corpus audit, acceptance condition ablations, a matched-recovery-budget control, and matched-supervision experiments (Comments 2-4).

On what verification verifies (W1, Q1). The validity loop has three distinct roles (Sec. 2.4; Algorithm 1):

  1. Policy-aware evaluator (reflection) during simulation. An evaluator reads the task, dialogue, tool evidence, and policies to assess action correctness, policy compliance, and communication obligations. It checks requirements such as explicit confirmation, relevant disclosure, and appropriate refusal, which cannot be inferred from final database state alone. Its feedback drives parallel selection and reflection-guided retries.
  2. State matching and program-check verification. The reference and demonstration are replayed from the same initial state, and their outcomes (states) are compared. Where available, environment-specific programmatic checks additionally test procedural or policy prerequisites (e.g., checking weather before opening the sunroof in CAR). Outcome matching is therefore one execution anchor rather than the only source of validation evidence.
  3. Diagnostic routing after disagreement. Given the reference and demonstration, an LLM reads the task, policy, replay evidence, and detected differences and routes the candidate to task/reference re-initialization or additional demonstration search.
Componentτ²-BenchCAR-Bench
State matching✓✓
Environment-specific programmatic procedural checks—✓
Policy-aware simulation evaluator✓✓
LLM diagnosis and recovery routing✓✓

Why a mutually wrong reference and demonstration are not simply assumed correct. The reference is generated during initialization, whereas the solving agent does not see that reference or the hidden task specification. The matching of the two provide strong signals for task correctness. Further, the policy-aware trajectory evaluation, environment-specific procedural checks and LLM recovery routing supply complementary signals.

Is the performance gain due to stricter acceptance then? (W1, Q1) No. We trained GRADUS under each acceptance criterion with the same number of training tasks (Table R-accept). Even with state matching alone, GRADUS exceeds the strongest baseline by 8 points on airline and 19 points on CAR. Each additional check then adds a further gain: LLM routing raises GRADUS to 0.71 and 0.76 (main paper reports), and the program checks raise CAR further to 0.80.

Table R-accept. Acceptance criterion (Opus 4.5 teacher, Qwen3-8B student, same number of training tasks).

MethodAcceptanceAirlineCAR base
Trajectory-firstState match0.590.56
GRADUSState match0.670.75
GRADUS+ LLM routing (main results)0.710.76
GRADUS+ LLM routing + program checkn/a0.80

In next Comment 2, we also report the human audit confirms that such shared errors are rare in practice.

AuthorsOpenReview ↗

Comment 2/4: Audit of the training corpus (W1, Q2)

A direct audit of accepted training tasks. The difficult, selected Frontier Exam candidates should not be used to estimate training-set correctness. We therefore had two annotators independently evaluate 100 accepted tasks each for airline and CAR, sampled across evolution depth and recovery round. Joint correctness requires both annotators to judge the reference correct; pool estimates re-weight strata to their representation in the accepted corpus.

Training-audit measureAirlineCAR
First-pass shallow (depths 0–2), joint correct100%93.3%
First-pass deep (depths 3–4), joint correct77.1%88.6%
Recovered, joint correct100%97.1%
All, weighted joint correctness97.5%93.3%
Gwet's AC1 for correctness0.930.96
Natural or plausible100%100%
Oracle not materially over-specified99%97.5%

For the metrics, we use the conservative joint criterion. Of 35 audited first-pass deep airline tasks, 27 were approved by both annotators, six had differing judgments, and two were rejected by both. Crucially, the recovered cohorts attain high joint correctness; the accepted training pool is substantially better grounded than one might infer from deliberately difficult Frontier Exams.

AuthorsOpenReview ↗

Comment 3/4: Blueprint preservation and recovery versus extra search (W2, Q2–Q3)

Recovery preserves the blueprint's behavioral target. The evolved blueprint fixes required actions, policy/outcome branches, causal structure, and interaction requirements. Our re-initialization prompt explicitly instructs the LLM to retain these targets while grounding them with consistent entities/state values or repairing the provisional reference. It does not replace a failed target with an easier one. In A.7, recovery clarified that a replacement flight had to be direct economy with alternatives declined; this was necessary to instantiate the original refused-booking target, not a newly selected outcome.

To measure this, first, we compared all accepted recovered tasks with their first realizations by checking the number of ordered state-changing tool calls change before and after recovery:

CohortSame write-call sequence
Airline Round 293.8%
Airline Round 388.0%
CAR Round 287.7%
CAR Round 3100.0%

The results show that 87.7–100% of recovered tasks share same number of write-calls as their original intended task. These preservation rates complement Figure 3(c), where difficult tasks are disproportionally represented in later recovery rounds, measured by zero-shot success rates. Thus, recovery retaining challenging lessons rather than routinely simplifying them.

Does recovery beat an equal budget of extra solution search? We ran the requested control experiment with the same blueprints and number of training tasks: remove task revision and spend the recovery budget on additional parallel sampling and reflection (Qwen3-235B-A22B proposer; Qwen3-30B-A3B student):

ConfigurationTelecomCAR disambiguation
Full: search + reflection + recovery0.760.61
No recovery0.680.51
No recovery; extra search with matched recovery budget0.690.54

Recovery outperforms the matched-search alternative by seven points in both domains. Additional samples may help solve an already valid task, but cannot correct an inconsistent task/reference specification. Re-initialization addresses that different failure mode. The control therefore isolates a benefit of task revision beyond additional sampling or reflection opportunities.

Reviewer_YC7cOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper treats a fixed, executable tool environment as a latent syllabus for agent training. An LLM compiles the environment's policies, tool semantics and state schema into guarded operators that record each action's condition branches and conditional consequences. Code composes these operators into blueprints through atomic generation, structural kernels and interaction kernels, steered by a corpus-level coverage ledger. Each blueprint is then realized through initialization, simulation and verification: an LLM grounds a concrete task and reference, parallel rollouts with evaluator-guided reflection produce demonstrations, and a tuple is accepted only when the replayed reference and demonstration outcomes agree and programmatic policy checks pass; disagreements are routed back to solution search or task re-initialization. Shallow and deep lessons feed SFT and GRPO-based RL respectively. The paper evaluates on τ²-Bench airline, retail and telecom and on CAR-Bench base, disambiguation and hallucination, and also evolves benchmark tasks into frontier exams for held-out frontier models. With an Opus 4.5 proposer, the final 30B model improves from 59% to 89% on τ²-Bench and from 47% to 83% on CAR-Bench; with Qwen3-235B it reaches 76% and 62%; the 8B student beats three controlled synthesis baselines on every split.

Critical strengths

The controlled 8B comparison is large, consistent and well designed. With the same Opus 4.5 proposer, same Qwen3-8B student and same task budget, GRADUS beats the three controlled synthesis baselines on every split, with gains of roughly 10 to 22 points; the curriculum transfers to the 30B mixture-of-experts student and executable RL improves all six splits, ending above every evaluated frontier model on CAR-Bench. This is a genuinely strong empirical result and the paper's most persuasive evidence.

The guarded-operator and recovery framing is a clean, reusable contribution, and the paper documents it concretely. Equation 2, Algorithm 1 and the complete Appendix A.7 trace show how "what a task should teach" can be separated from the concrete task, reference and demonstration, with an explicit division between LLM and code. Recovering failed lessons instead of discarding them is a sensible alternative to rejection sampling, and the human audit and coverage analysis are valuable additions that most synthesis papers omit.

Critical weaknesses
  1. The verified training data cannot be audited. Acceptance rests on programmatic policy checks whose only named component, POLICYCHECK, appears solely in Algorithm 1 and is never defined. Appendix A.7 itself states that the API does not enforce the flown-portion, cancellation-trigger, reason or confirmation guards, and the same trace contains a policy-wrong reference that passes replay and a demonstration that matches the intended outcome for a fabricated reason. Both annotators judged only 25 of 32 airline and 70 of 79 CAR verified exam candidates correct, and the training corpus is never audited.

  2. Recovery can rewrite the task and the oracle, and the headline comparison is unmatched. Adding task recovery over reflection gains only four to five points, about one standard deviation, and recovery is never removed in the leave-one-out ablations. Appendix A.7 shows recovery re-grounding the initial state, tightening user constraints and allowing the reference to absorb an available alternative until hashes agree; coverage is never rechecked after realization. Moreover, all controlled baselines are 8B-only, only GRADUS rows receive RL, and the 30B headline has no matched 30B baseline or compute-matched reflection-only control.

Questions for authors

Acceptance gate specification and coverage. What exactly does POLICYCHECK check, how is it implemented, and which obligations does it cover: explicit confirmation, refusal, required disclosure, state-dependent reasoning, and communication quality? If a reference and a demonstration are both semantically wrong but reach the same outcome, what prevents acceptance? What is the estimated false-accept rate on a random sample of accepted training tuples, stratified by evolution depth and recovery round?

Attribution of the headline improvement. Please report teacher tokens, environment calls and wall-clock per method per accepted example, give the values of the search hyperparameters, and match total supervised tokens and assistant-step rows. Add a 30B baseline, a baseline trained with the same RL stage, and a control that uses the same parallel sampling and reflection budget but no task revision. Do the reported gains survive these controls?

Coverage and lesson preservation through recovery. How does the ledger treat recovered or re-grounded tasks, and why is coverage not rechecked after Stage 3? What invariant ensures that rewriting the initial state, the task text or the reference preserves the intended policy-outcome lesson? How are the difficulty classes in Figure 3 defined, and are they measured before or after recovery? Please reconcile the depth indexing and the Table 6 caption with the depth rows and with Figure 3.

Training-data audit. The human audit covers only the frontier-exam candidates, not the accepted training corpus. Please report a stratified random audit of accepted training examples under the same annotation protocol, with joint-correct rates, and estimate how much of the downstream gain would survive if the exam-audit error rate also applies to training data.

Decontamination and evaluation validity. What fraction of synthesized training tasks share a tool-call sequence, initial state or policy-outcome branch set with any benchmark test task? How much performance remains after removing training examples that overlap with the test tasks? Does the frontier-exam comparison evaluate original and evolved success on the same task subset, rather than full-benchmark originals against the accepted exam subset?

Realism of interaction variation. Are the ambiguity, indirect-reference and user-pressure kernels grounded in real customer-service interactions, and is the naturalness of the training data validated anywhere outside the deliberately hard exam candidates? The exam audit already finds most airline and about half of retail evolved tasks unusual or contrived, and the scripted instruction to "push back twice" in Appendix A.7 suggests that at least some kernels are authored heuristics rather than observed user behaviour.

9 comments
AuthorsOpenReview ↗

Comment 1/8: Verification Policy Check Detail and Acceptance Gates

We sincerely thank the reviewer for the exceptionally thorough review and for recognizing the strength of our controlled 8B comparisons and the guarded-operator/recovery framework. We conducted targeted follow-up experiments addressing verification, training-data quality, recovery attribution, compute and supervision budgets, RL controls, decontamination, and frontier-exam validity. The following comments report the results.

What is checked during verification? (W1, Q1) The validity loop combines three complementary mechanisms (Sec. 2.4; Algorithm 1), which we will distinguish more explicitly in the manuscript:

  1. Policy-aware evaluator (reflection) during simulation. An evaluator reads the task, dialogue, tool evidence, and policies to assess action correctness, policy compliance, and communication obligations. It checks requirements such as explicit confirmation, relevant disclosure, and appropriate refusal, which cannot be inferred from final database state alone. Its feedback drives parallel selection and reflection-guided retries.
  2. State matching and program-check verification. The reference and demonstration are replayed from the same initial state, and their outcomes (states) are compared. Where available, environment-specific programmatic checks additionally test procedural or policy prerequisites (e.g., checking weather before opening the sunroof in CAR). Outcome matching is therefore one execution anchor rather than the only source of validation evidence.
  3. Diagnostic routing after disagreement. Given the reference and demonstration, an LLM reads the task, policy, replay evidence, and detected differences and routes the candidate to task/reference re-initialization or additional demonstration search.
Componentτ²-BenchCAR-Bench
State matching✓✓
Environment-specific programmatic procedural checks—✓
Policy-aware simulation evaluator✓✓
LLM diagnosis and recovery routing✓✓

Agreement alone is not our sole evidence of semantic correctness. The solving agent never sees the provisional reference or hidden task, and simulation evaluates policy compliance independently of state equality. In Appendix A.7, a cancellation refusal was challenged by reflection evaluator because the agent had not checked actual flight status; the corrected rollout revealed the reference's flawed grounding and triggered task repair/recovery.

Is the performance gain due to stricter acceptance then? (W1, Q1) Not. We trained GRADUS under each acceptance criterion with the same number of training tasks (Table R-accept). Even with state matching alone, GRADUS exceeds the strongest baseline by 8 points on airline and 19 points on CAR. Each additional check then adds a further gain: LLM routing raises GRADUS to 0.71 and 0.76, and the program checks raise CAR further to 0.80. Our main gains are therefore not contingent on a more selective verifier.

Table R-accept. Acceptance criterion (Opus 4.5 teacher, Qwen3-8B student, same number of training tasks).

MethodAcceptanceAirlineCAR base
Trajectory-firstState match0.590.56
GRADUSState match0.670.75
GRADUS+ LLM routing (main results)0.710.76
GRADUS+ LLM routing + program checkn/a0.80

In next Comment 2, we also conducted an independent stratified human audit of accepted training examples, directly estimating residual errors rather than assuming that automatic verification is infallible.

AuthorsOpenReview ↗

Comment 2/8 : Independent audit of accepted training tasks (W1, Q1, Q4, Q6)

The original human audit targeted deliberately difficult Frontier Exam candidates; it was not designed to estimate training-corpus quality. Following the reviewer's suggestion, two annotators independently audited 100 randomly sampled accepted training tasks each for airline and CAR, stratified by evolution depth and recovery round and evaluated under the same correctness/realism/strictness protocol. We reweighted stratum-level judgments to the accepted training pool.

MeasureAirlineCAR
First-pass shallow (depths 0–2): jointly correct100%93.3%
First-pass deep (depths 3–4): jointly correct77.1%88.6%
Recovered: jointly correct100%97.1%
All: weighted joint correctness97.5%93.3%
Gwet's AC1 (correctness)0.930.96
Natural or plausible100%100%
Oracle not materially over-specified99%97.5%

Our joint-correct criterion is deliberately conservative: both annotators must approve the oracle. Of 35 audited deep first-pass airline tasks, 27 were accepted by both, six drew differing judgments, and two were rejected by both. Across the accepted pools, the estimated non-joint-correct shares are 2.5% airline and 6.7% CAR.

The results directly address the possibility that the Frontier Exam audit's error rates also characterize training: the randomly sampled training tasks show substantially higher oracle correctness, and recovered tasks have particularly high joint correctness. No audited task was rated contrived, and almost all oracles were judged not materially over-specified. Thus, we can assess training quality directly rather than extrapolating from intentionally challenging exam candidates.

AuthorsOpenReview ↗

Comment 3/8: Recovery ablation and paired significance (W2, Q2)

The right-hand cumulative validity ablation in Table 2 already contains a no-recovery setting: “+ Solution: reflection” includes the full pipeline up to reflection, and the final row adds task recovery. We will relabel this contrast to make it explicit.

The Table 2 standard deviations reflect repeated evaluations, whereas the reviewer's question is about the difference between methods on the same tasks. We therefore re-evaluated the two checkpoints on matched tasks and conducted paired significance tests:

SplitFullWithout recoveryPaired gain95% CIPermutation p
Telecom0.76 ± 0.030.68 ± 0.04+0.08[0.03, 0.13]0.004
CAR disamb.0.61 ± 0.070.51 ± 0.07+0.10[0.03, 0.17]0.007

The recovery contribution is positive and statistically supported under both paired comparisons.

Is recovery merely more solution search? Parallel rollouts and reflection search focus on discovering a better demonstration of an existing task. Task recovery focus on repairing a different failure mode: the instantiated task, reference and state can be inconsistent, invalid or infeasible. Additional rollout attempts alone cannot make an inconsistent specification valid.

We tested this distinction directly. Using the same blueprints, training-task count, proposer (Qwen3-235B-A22B), and Qwen3-30B-A3B student, we removed task revision and redirected its search budget to extra parallel search and reflection:

SettingTelecomCAR disamb.
Full, with recovery0.760.61
No recovery0.680.51
No recovery + extra search at matched recovery budget0.690.54

Even after allocating the recovery budget to solution search, the full method remains 7 points better on both splits. This directly supports the benefit of repairing task realizations, above and beyond extra sampling/reflection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Evidence Addressing the Main Concerns

We sincerely thank all reviewers for their constructive feedback. We have uploaded a revised manuscript incorporating the additional experiments and clarifications (highlighted in green).

Key results at a glance

Reviewer concernNew evidenceKey finding
Training-data reliabilityStratified audit of 200 accepted tasks97.5% / 93.3% weighted joint correctness (airline / CAR)
Gains from stricter acceptance?GRADUS under state matching aloneStill ahead: 0.67 vs. 0.59, 0.75 vs. 0.56. Adding more acceptance criteria leads to further gains
Recovery vs. extra searchPaired tests; matched-budget controllead by +8–10 points (p ≤ 0.007); lead extra search by 7 points
Recovery preserves difficulty?Write-sequence preservation; post-recovery difficulty92.2% / 87.4% preserved (n = 270 / 87); still harder
Supervision, RL, scaleMatched tokens, identical GRPO, 30BGRADUS leads in every comparison
Train–test overlapRemove overlapping tool-call graphs, retrainLead retained by 0.12 / 0.21

1. Can the synthesized training data be trusted? Yes. (YC7c, 1upw) A policy-aware LLM evaluator checks action correctness, policy compliance and communication during simulation and reflection, and execution-grounded verification compares each outcome with an independently constructed reference; environment-defined programmatic checks can be added on top. The gains do not solely depend on these checks though: under state matching alone, the baselines' criterion, GRADUS still leads the strongest baseline by 8 points (airline) and 19 points (CAR). Further acceptance gating leads to more gains. A stratified human audit of the training corpus itself finds 97.5% (airline) and 93.3% (CAR) of accepted tasks correct, measuring quality directly rather than inferring it from the deliberately difficult exam candidates. (Sec. 2.4; App. A.12–A.13; Tables 23–24)

2. Does recovery add a distinct benefit without simplifying lessons? Yes. (YC7c, 1upw, taCh) Recovery addresses a different failure from solution search. Sampling and reflection look for a better solution, but when the task itself is flawed (e.g., its reference contradicts the initial state), no correct solution exists and more search cannot help; only repairing the task makes it usable. Recovery re-instantiates the entities and values in the blueprint's placeholders without changing its storyline, so 92.2% (airline) and 87.4% (CAR) of recovered tasks keep their exact tool-call sequences, and they remain harder after recovery. Recovery improves performance by 8 to 10 points (paired p ≤ 0.007), while spending its budget on more search instead falls 7 points short, as expected when the remaining failures lie in the tasks rather than the solutions. (Sec. 3.5; App. A.14; Tables 25–28)

3. Do the gains hold under matched budgets, identical RL and decontamination? Yes. (YC7c, 1upw, taCh) We added a control for each alternative explanation: matched supervised tokens, identical GRPO for GRADUS and Trajectory-first, 30B students, and retraining after removing every training task that shares a tool-call graph with a test task. GRADUS leads in all of them, including a margin of 0.12 (airline) and 0.21 (CAR) after decontamination, so additional supervision, RL asymmetry and train–test overlap do not explain the improvements. (Sec. 3.2; App. A.11, A.15–A.17; Tables 21, 30–31, 33)

4. Are the interaction patterns grounded and realistic? (taCh, YC7c) GRADUS is a plug-and-play framework; its kernels are modular transformation specifications, not a claimed taxonomy of user behavior. Following reviewer taCh, we added policy-misunderstanding and intent-shift kernels without any pipeline change, keeping training-set size and depth distribution fixed, and the pipeline produced verified tasks exhibiting exactly these behaviors (examples in App. A.18). The realism concern came from the Frontier Exam audit, whose tasks are deliberately long-tail; auditing the training data directly, we found no contrived tasks among 200 random samples. (App. A.8, A.13, A.18; Tables 24, 34–35)

5. Are the Frontier Exams valid? Yes. (YC7c, taCh) On exactly the original tasks from which the exams were evolved, the evolved tasks are harder for every model, and the newest frontier models remain far from saturating them. Figure 1 has also been redesigned. (Sec. 3.7; App. A.10; Tables 17–18)

We believe these results address the main soundness and evaluation concerns. We would appreciate it if the reviewers could let us know whether any concern remains, and would be grateful if they could reconsider their assessments where the new evidence resolves their concerns. Thank you again for your time and thoughtful feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.