acceptodds
Under review as a conference paper at ICLR 2027

Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

Abstract

Agent learning is shifting from learning from human data toward learning through interaction with the world. However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks. We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs). Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution. WildAWM is entirely powered by open-source models, demonstrating that high-quality computer-use environments can be generated without proprietary frontier models. Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness. The resulting pipeline synthesizes 341 long-horizon environment-task pairs that can require over 100 agent steps to solve and remain challenging even for frontier models, with GPT-6-Astra solving only 60% of them. We train two Qwen models of different sizes exclusively in these synthetic worlds. Both models achieve substantial gains across 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, demonstrating that learning in WildAWM's synthesized worlds generalizes broadly to unseen computer-use tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agent, computer-use, environment synthesis, task synthesis

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XKSbOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces, WildAWM, a scalable pipeline for synthesizing persona-driven and cross-application environments. Empirical results demonstrate the introduced pipeline could synthesize environments that (1) facilitate agent training and (2) are diverse and solvable.

Critical strengths

1. The manuscript has a timely good motivation. Realistic computer use span across softwares and interfaces; existing environment synthesizing methods do not cover this.

2. The synthesizing pipeline is comprehensive. The pipeline covers different aspects of environment synthetic: diversity (different persona), cross-agent interface, solvability, difficulty and verifier reliability.

Critical weaknesses

1. The main claim is not fully supported by the experiments. The Introduction motivates WildAWM by arguing that realistic computer use spans multiple softwares and interaction interfaces, whereas many existing environment-synthesis approaches focus on a single application or interface. However, the experiments do not directly establish the benefit of this design. Table 1 provides a comparison of method capabilities, while Table 2 demonstrates improvements over the underlying base models after training on WildAWM. Neither provides a direct comparison with agents trained on single environments or agents trained using Agent World Model.

2. The method is not fully ablated. The proposed synthesis pipeline has two major steps: 1) persona-driven diversification and 2) cross-interface task synthesis. However, the existing ablations primarily examine the quality gates used during task synthesis (Table 4) and components of the training procedure (Table 5). They do not isolate the contributions of persona introduced diversity and cross-interface diversity.

3. None of synthesized environment's property is validated by human. The proposed generation pipeline generate diverse and solvable tasks. Human judgement even on a small subset would provides more guarantees on the validity of the tasks and the properties above.

Questions for authors
  1. Would you please provide a direct comparison with agents trained on single environments or agents trained using Agent World Model?

  2. Would you please provide an ablation on persona-driven diversification's contribution to synthesized environments?

6 comments
AuthorsOpenReview ↗

Response to Reviewer XKSb (1/3)

Thank you for the constructive review and the clear suggestions for additional comparisons and ablations. We would like to respond with additional results and clarifications as follows. We also updated the PDF, which is marked in blue to highlight the changes.

Weakness 1 & Question 1

The main claim is not fully supported by the experiments. The Introduction motivates WildAWM by arguing that realistic computer use spans multiple softwares and interaction interfaces, whereas many existing environment-synthesis approaches focus on a single application or interface. However, the experiments do not directly establish the benefit of this design. Table 1 provides a comparison of method capabilities, while Table 2 demonstrates improvements over the underlying base models after training on WildAWM. Neither provides a direct comparison with agents trained on single environments or agents trained using Agent World Model.

Would you please provide a direct comparison with agents trained on single environments or agents trained using Agent World Model?

Thank you for this helpful suggestion. We would first like to provide some additional context from our existing analyses of the cross-interface design. The policy analysis (Figure 5) shows that training raises the interface-switch rate from 18.4% to 25.5%, close to GLM-5.2, whereas RL from the base model alone lowers it to 15.7%. WildAWM environments are also more diverse than CUA-Gym's (higher Vendi score at every sample size, Figure 3a).

To directly address your question, we conducted a controlled comparison with single-interface environments. We fine-tuned Qwen3.6-35B-A3B on trajectories from CUA-Gym (GUI), AWM (MCP), and WildAWM, using the same teacher, data budget, and training recipe. We then applied RL to each SFT model under matched settings:

ModelO2WWAQCBCLECUGOSVOS-2ALEAvg.WildAWM-val
Qwen3.6-35B-A3B69.6053.5457.8638.0053.7333.1613.2223.5742.845.68
CUA-Gym-SFT (GUI)71.5459.2958.3855.0070.4130.119.5227.4147.718.33
AWM-SFT (MCP)73.9956.1954.6353.0031.2838.8112.0418.7842.3414.39
WildAWM-SFT (XAI)79.8561.5062.7157.3355.3736.4211.2027.4448.9837.30
CUA-Gym-RL (GUI)73.4365.0459.4256.0071.0047.3516.3024.9951.696.06
AWM-RL (MCP)66.9163.2755.5053.3368.8936.0715.0123.1247.7612.12
WildAWM (XAI)80.7767.2661.4454.6765.9549.2221.7326.3153.4235.98

We find that:

  1. WildAWM achieves the highest average and leads on most benchmarks for both SFT and RL models. WildAWM-RL performs particularly well on the long-horizon benchmarks OSV, OS-2, and ALE, where single-interface training shows less consistent transfer.
  2. Single-interface environments do transfer, but unevenly: among the SFT models, CUA-Gym is the strongest on CUG and AWM on the OSWorld benchmarks, yet AWM is the weakest on CUG and ALE, and no single-interface source is best overall.
  3. Cross-interface tasks benefit from capabilities that are less effectively learned through single-interface training: on WildAWM-val, the AWM-RL and CUA-Gym-RL models achieve 12.12 and 6.06 pass@1, respectively, compared with 35.98 for WildAWM.

We have updated the paper accordingly, with changes shown in blue: Table 3 now includes CUA-Gym-SFT, AWM-SFT, WildAWM-SFT, CUA-Gym-RL, and AWM-RL, and point (3) of "Main Results" in Sec. 4.

AuthorsOpenReview ↗

Response to Reviewer XKSb (2/3)

Weakness 2 & Question 2

The method is not fully ablated. The proposed synthesis pipeline has two major steps: 1) persona-driven diversification and 2) cross-interface task synthesis. However, the existing ablations primarily examine the quality gates used during task synthesis (Table 4) and components of the training procedure (Table 5). They do not isolate the contributions of persona introduced diversity and cross-interface diversity.

Would you please provide an ablation on persona-driven diversification's contribution to synthesized environments?

Thank you for this valuable suggestion. Our existing analyses provide initial evidence of the diversity achieved by the synthesis pipeline: WildAWM states are larger (median 52.4K vs. 1.1K tokens) and more diverse than CUA-Gym's (higher Vendi score at every sample size, nearest-neighbor cosine distance 0.25 vs. 0.20, Figure 3a), and task coverage keeps growing as synthesis scales from 6 to 36 personas (134 domains and 91 task types without saturation, Figure 4c).

To directly evaluate its contribution to downstream training, we fine-tuned Qwen3.6-35B-A3B using the same SFT recipe and teacher (GLM-5.2), varying persona coverage from 4 to 16. To control the number of samples, each set uses only 32 tasks, matching the number of tasks available with 4 personas, rather than the full training data.

Training tasks fromO2WWAQCBCLECUGOSVOS-2ALEAvg.
4 personas63.6754.8753.6951.1747.5627.125.4218.2340.22
16 personas67.2861.5054.2350.6756.0639.206.7420.5244.53

We find that:

  1. With the same number of tasks, raising persona coverage from 4 to 16 improves the average from 40.22 to 44.53 (+4.31) and improves seven of the eight benchmarks.
  2. The largest gains are on the GUI benchmarks CUG (+8.50) and OSV (+12.08). These results suggest that broader persona coverage improves transfer even with the same number of training tasks.

For cross-interface synthesis, the matched comparison with GUI-only CUA-Gym and MCP-only AWM environments (Table 3) provides direct evidence of the benefits of our cross-interface approach: under the same teacher and data budget, WildAWM transfers best on average, and on cross-interface validation tasks, the single-interface models after RL achieve 12.12 and 6.06 pass@1, compared with 35.98 for WildAWM.

We have updated the paper accordingly, with changes shown in blue: Sec. 5.2 summarizes the persona ablation, and Appendix A.4.1 (Table 13) reports it in full.

AuthorsOpenReview ↗

Response to Reviewer XKSb (3/3)

Weakness 3

None of synthesized environment's property is validated by human. The proposed generation pipeline generate diverse and solvable tasks. Human judgement even on a small subset would provides more guarantees on the validity of the tasks and the properties above.

Thank you for your suggestion. We have conducted additional human evaluations of both task validity and verifier reliability.

For the tasks, we randomly sampled 100 tasks each from WildAWM, AWM, and CUA-Gym. Three annotators independently read each task description. They counted the applications and data sources involved in each task and independently assessed its validity and whether it requires multiple interaction interfaces, with majority voting used for categorical judgments:

WildAWMAWMCUA-Gym
Apps and data sources per task4.061.141.29
Valid tasks99%87%96%
Multi-interface tasks100%0%7%

The annotators agree closely on all three judgments (Krippendorff's α = 0.95 for the counts, Fleiss' κ = 0.97 for multiple interfaces, and Gwet's AC1 = 0.84 for validity).

For the verifier, we manually inspected 100 rollouts, reviewing the task, the trajectory, and compare the initial and final world states without seeing existing verdicts. The reliability study with the new human rows is:

ComparisonCons.PrecisionRecallF1κ
Verifier vs. GLM-5.20.950.980.930.950.90
Verifier vs. GPT-5.6-Sol0.920.850.980.910.83
Verifier vs. Claude-Opus-50.920.980.880.930.84
Verifier vs. Judge majority0.950.980.930.950.90
Verifier vs. Human (new)0.880.900.870.880.76
GLM-5.2 vs. Opus-50.971.000.950.970.94
GPT-5.6-Sol vs. Opus-50.891.000.800.890.79
Majority vs. Human (new)0.930.920.940.930.86

Against the human labels (52 successes, 48 failures), the errors of each judge are:

JudgeCons.κFalse acceptanceFalse rejection
Our verifier0.880.7610.4%13.5%
Judge majority0.930.868.3%5.8%

We find that:

  1. Task validity: 99% of sampled WildAWM tasks are judged valid, the highest among the three sources, despite involving more than three times as many applications and data sources. All sampled WildAWM tasks require multiple interfaces, supporting the intended cross-interface design.
  2. Verifier validity: the verifier agrees with humans on 88% of rollouts (κ = 0.76). In a blind corruption audit, it also rejected 39 of 41 final states that an independent agent had corrupted to violate the task.
  3. Observed solvability: although these tasks are designed to be hard and only about three rollouts per task were inspected, humans confirmed a correct completion by the agent for 24 of the 33 tasks.

We have updated the paper accordingly: Table 4 adds the rows "Human" and "Majority vs. Human", Sec. 5.1 adds sentences reporting the human task assessment, the human comparison, and the corruption audit, Appendix A.6.1 (Table 14) reports the human task assessment, Appendix A.6.2 describes the annotation protocol and reports the per-judge errors (Table 15), and Appendix A.6.3 reports the corruption audit (Tables 16 and 17). We also have case study examples in Appendix C.

Reviewer_26WTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces WildAWM, a pipeline that uses only open-source models to synthesize persona-driven, multi-day personal-computer environments with multi interfaces, along with cross-interface tasks and programmatic verifiers. Five execution-based quality gates check task validity. The result is 300+ long-horizon tasks, and GPT-6-Astra solves about 60% of them. Qwen3.5-4B and Qwen3.6-35B-A3B are trained on these tasks using an SFT warm-start followed by GRPO. The trained models improve on all 8 out-of-distribution web, claw, and computer-use benchmarks.

Critical strengths
  • Addresses an important practical problem: building scalable, resettable, verifiable environments for CUAs across GUI, MCP, and CLI jointly.
  • Persona-driven multi-day synthesis produces much richer states than task-injected baselines, with deterministic replay checks to ensure state consistency.
  • Well-designed 5-stage execution quality gates
  • Fully open-source synthesis pipeline with plans to release code, environments, tasks, and verifiers.
  • Strong empirical gains across 8 OOD benchmarks at two model scales.
Critical weaknesses
  • Environment benefit is confounded with GLM-5.2 distillation. Models are warm-started on 100 GLM-5.2 SFT trajectories, and dropping this warm start causes the largest ablation drop. Since RL is short and gains concentrate on tasks "within the teacher's reach", it is unclear how much WildAWM worlds contribute beyond distilling GLM-5.2. A control using the same number of GLM-5.2 trajectories from CUA-Gym/AWM is needed.
  • No controlled comparison with competing environment pipelines. CUA-Gym and OpenForgeRL numbers use different base models and harnesses.
  • Evaluation caveats. Single-run results without CIs; excluding infra/reCAPTCHA failures; NVFP4 quantization on open models; and giving the 4B text policy a 35B VLM captioner for screenshots.
  • Unsupported scaling and cost claims. Synthesis cost (GPU-hours/tokens) is missing from the main text, and overall scale is modest.
Questions for authors
  1. Can you provide a control baseline trained on the same number of GLM-5.2 SFT trajectories and RL budget collected in an existing environment?
  2. Have you evaluated programmatic verifier precision/recall against human annotations on a sampled subset?
  3. What is the end-to-end compute/token cost per accepted task, and how does acceptance rate change as personas scale?
  4. How does the 4B model perform on GUI tasks without the external 35B VLM captioner?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 26WT (1/3)

Thank you for the detailed review. We would like to clarify each point and provide additional results to address the concerns. We also updated the PDF, which is marked in blue to highlight the changes.

Weakness 1, Weakness 2 & Question 1

Environment benefit is confounded with GLM-5.2 distillation. Models are warm-started on 100 GLM-5.2 SFT trajectories, and dropping this warm start causes the largest ablation drop. Since RL is short and gains concentrate on tasks "within the teacher's reach", it is unclear how much WildAWM worlds contribute beyond distilling GLM-5.2. A control using the same number of GLM-5.2 trajectories from CUA-Gym/AWM is needed.

No controlled comparison with competing environment pipelines. CUA-Gym and OpenForgeRL numbers use different base models and harnesses.

Can you provide a control baseline trained on the same number of GLM-5.2 SFT trajectories and RL budget collected in an existing environment?

Thank you for raising this concern. We would like to provide some additional context and results:

  1. The warm-start ablation shows that SFT provides a better starting point for RL, but does not necessarily imply that the gains come solely from the teacher. We also provided results without any teacher trajectory: WildAWM+RL and WildAWM+RFT. RL in WildAWM still raises Qwen3.6-35B-A3B from 42.84 to 48.11 on average (Table 6), and RFT on the base model's own successful trajectories gains 73 net OOD tasks (Table 21).
  2. On top of the warm start, RL in WildAWM brings its largest gains on the computer-use benchmarks where SFT alone helps least (Table 3): CUG rises from 55.37 to 65.95 (+10.58), OSV from 36.42 to 49.22 (+12.80), and OS-2 from 11.20 to 21.73 (+10.53, nearly doubled), and WA also gains +5.76. Overall, RL raises the net OOD gain after SFT to +129 tasks (Table 21).
  3. Since our paper aims at synthesizing tasks that are challenging even for frontier models, the weaker models within our compute budget solve few of them at first (5.68% pass1 for Qwen3.6-35B-A3B), so most GRPO groups would receive no reward and no gradient. To avoid wasting compute, we adopt this warm start, which is a common practice in LLM/agent post-training. More capable models may be trained on our tasks with RL directly.

A teacher's trajectories are only as useful as the tasks and environments in which they are collected, and these are exactly what WildAWM synthesizes. To directly address your concern, we further conducted controlled experiments: under the same teacher (GLM-5.2), harness, training budget and recipe, we train Qwen3.6-35B-A3B with SFT and then RL in three synthetic environment methods: CUA-Gym (which synthesizes GUI only envs), AWM (which synthesizes MCP only envs), and WildAWM. The results are:

ModelO2WWAQCBCLECUGOSVOS-2ALEAvg.WildAWM-val
Qwen3.6-35B-A3B69.6053.5457.8638.0053.7333.1613.2223.5742.845.68
CUA-Gym-SFT (GUI)71.5459.2958.3855.0070.4130.119.5227.4147.718.33
AWM-SFT (MCP)73.9956.1954.6353.0031.2838.8112.0418.7842.3414.39
WildAWM-SFT (XAI)79.8561.5062.7157.3355.3736.4211.2027.4448.9837.30
CUA-Gym-RL (GUI)73.4365.0459.4256.0071.0047.3516.3024.9951.696.06
AWM-RL (MCP)66.9163.2755.5053.3368.8936.0715.0123.1247.7612.12
WildAWM (XAI)80.7767.2661.4454.6765.9549.2221.7326.3153.4235.98

We find that:

  1. With the same training, SFT on WildAWM outperforms SFT on CUA-Gym and AWM on average (48.98 vs. 47.71 and 42.34) and on five of eight benchmarks, including all web and claw benchmarks. This confirms the superiority of WildAWM as a synthetic method.
  2. With the same RL compute on top, WildAWM remains the best on average and leads on six of eight benchmarks, including the long-horizon OSV, OS-2, and ALE. Excluding CUG, which is in-domain for CUA-Gym, its lead grows to 2.70 and 6.89 points.
  3. The difference is largest in the setting WildAWM targets: on the cross-interface tasks of WildAWM-val, the AWM-RL and CUA-Gym-RL models reach only 12.12 and 6.06 pass1, versus 35.98.

We have updated the paper accordingly, with changes shown in blue: Table 3 now includes CUA-Gym-SFT, AWM-SFT, WildAWM-SFT, CUA-Gym-RL, and AWM-RL, and point (3) of "Main Results" in Sec. 4.

AuthorsOpenReview ↗

Response to Reviewer 26WT (2/3)

Weakness 3 & Question 4

Evaluation caveats. Single-run results without CIs; excluding infra/reCAPTCHA failures; NVFP4 quantization on open models; and giving the 4B text policy a 35B VLM captioner for screenshots.

How does the 4B model perform on GUI tasks without the external 35B VLM captioner?

We provide the following additional results and clarifications:

  1. Single runs. We provide 5 independent runs, WildAWM-35B-A3B averages 52.91 ± 0.83 over 1,432 OOD tasks on 8 benchmarks, and every run exceeds the base model by at least 9.20 points. A paired bootstrap over tasks gives an average gain of +10.04 with 95% CI [+8.19, +11.93].
  2. Infra and reCAPTCHA exclusions. These failures come from the evaluation infrastructure or from websites bot detection, which are excluded for every model for fair comparison. From the existing runs stats, the excluded share is small and does not impact on main conclusions.
ModelInfra failures (of non-O2W tasks)O2W blockedAvg. reportedAvg. on intersection subsetAvg. with blocked = failure
GLM-5.203063.2163.5262.13
Opus-503570.8370.7069.42
GPT-6-Astra03773.1173.0871.68
Qwen3.5-4B13121.8322.0721.49
WildAWM-4B32430.0430.6229.58
Qwen3.6-35B-A3B02742.8442.9642.06
WildAWM-35B-A3B04053.4253.4052.07

The intersection subset keeps only tasks that no model excluded. On this subset the averages move by at most 0.58 points and the ranking is unchanged. Even if every blocked O2W task counts as a failure, the gains remain significant: +8.09 (4B) and +10.01 (35B-A3B).

  1. NVFP4. All Qwen models, both base and trained, are evaluated in BF16, so their results are unaffected by quantization. NVFP4 is used only to serve the large open-source models (GLM-5.2, DeepSeek-V4-Flash, and MiniMax-M3), and we have enhanced the clarification in Appendix A.1 in the updated PDF.

  2. Vision captioner. To answer Question 4 directly, we re-evaluated both 4B models with Qwen3.5-4B itself as the VLM captioner tool. Both models in each block are evaluated under the same protocol.

Vision toolPolicyO2WWAQCBCLECUGOSVOS-2ALEAvg.
Qwen3.6-35B-A3BQwen3.5-4B26.3930.0944.0626.6721.3918.180.877.0021.83
Qwen3.6-35B-A3BWildAWM-4B45.6539.8248.3736.6737.3522.481.038.9630.04
Qwen3.5-4BQwen3.5-4B30.1128.3241.9830.6726.3621.211.5912.1524.05
Qwen3.5-4BWildAWM-4B37.7335.4046.1232.3340.3230.351.0711.8629.40

WildAWM-4B keeps nearly the same average (29.40 vs. 30.04) and still improves over the base model by 5.35 points on average and by 7.46 points on the five GUI benchmarks.

We have updated the paper accordingly, with changes shown in blue: "Implementation Details" in Sec. 4 and Appendix A.1 state the numerical precision and the exclusion rules. Appendix A.2 describes the vision tool, and Appendix A.3 summarizes the robustness checks and reports the 4B re-evaluation (A.3.1, Table 8), the exclusion statistics (A.3.2, Table 9), and the multiple runs (A.3.3, Table 10).

AuthorsOpenReview ↗

Response to Reviewer 26WT (3/3)

Weakness 4 & Question 3

Unsupported scaling and cost claims. Synthesis cost (GPU-hours/tokens) is missing from the main text, and overall scale is modest.

What is the end-to-end compute/token cost per accepted task, and how does acceptance rate change as personas scale?

Thank you for this helpful suggestion. We would like to clarify that Table 2 already reports the token cost per task. To make the cost explicit, we now report the cost estimation via OpenRouter and add the acceptance rate as the number of personas grows. We use the prices: GLM-5.2 $0.49/$1.50, DeepSeek-V4-Flash $0.04/$0.10, and MiniMax-M3 $0.23/$0.96 per million input/output tokens.

Task synthesis (642 attempts)Value
Tokens per accepted task (mean)70.9M
Cost per accepted task (mean / median / P90)$33.2 / $27.8 / $62.4
Cost share: task generation / counterfactual probe / judgments / rollouts / proposals84 / 7 / 5 / 2 / 1%
Personas available at the time of the attempt67–1213–2425–3334–36
Attempts1065524159157
Acceptance rate61.3%43.6%58.3%42.1%45.2%

We find that:

  1. An end-to-end accepted task costs $33.2 on average, covering its generation, all revision rounds, rollouts, judgments, and probes, with the cost of rejected candidates amortized over the accepted tasks.
  2. As synthesis scales to 25–36 personas, the acceptance rate stabilizes at 42–45%. Over the last 500 attempts it shows no significant trend. Thus the cost per accepted task stays roughly constant as synthesis scales.
  3. Task coverage keeps growing without saturation (134 domains and 91 task types, Figure 4c), so the corpus size reflects our compute budget rather than a limit of the pipeline.

We have updated the paper accordingly: Table 2, moved from the appendix to Sec. 3.2, now also reports the cost per accepted task, and Sec. 3.2 summarizes the cost and its breakdown and the "Synthesis Cost and Acceptance at Scale" paragraph in Appendix A.5 details the pricing. Sec. 5.1 adds the acceptance rate as the persona pool grows, shown in the new panel Figure 4d.

Question 2

Have you evaluated programmatic verifier precision/recall against human annotations on a sampled subset?

Beyond the three agent judges in Table 4 (GLM-5.2, GPT-5.6-Sol, and Claude-Opus-5, each run under two or three harnesses), we manually inspected all 100 rollouts, reading the task, the trajectory, the final answer, and the initial and final world states without seeing any existing automatic verdicts. The verifier reliability study with the new human rows is:

ComparisonCons.PrecisionRecallF1κ
Verifier vs. GLM-5.20.950.980.930.950.90
Verifier vs. GPT-5.6-Sol0.920.850.980.910.83
Verifier vs. Claude-Opus-50.920.980.880.930.84
Verifier vs. Judge majority0.950.980.930.950.90
Verifier vs. Human (new)0.880.900.870.880.76
GLM-5.2 vs. Opus-50.971.000.950.970.94
GPT-5.6-Sol vs. Opus-50.891.000.800.890.79
Majority vs. Human (new)0.930.920.940.930.86

Taking the human labels as ground truth (52 successes, 48 failures), each judge compares as follows:

JudgeCons.PrecisionRecallκFalse acceptanceFalse rejection
Our verifier0.880.900.870.7610.4%13.5%
Judge majority0.930.920.940.868.3%5.8%

We find that:

  1. The verifier has an 88% agreement with human labels, with a Cohen's kappa of 0.76, which indicates substantial reliability.
  2. Its errors are balanced, whereas the frontier agent judges trade one error type for the other (Table 15).
  3. In an additional blind corruption audit, an independent agent edited final states so that they remain plausible but violate the task. On 41 pairs of original and corrupted states, the verifier accepts only 2 corrupted states (4.9%), and the corruption flips 31 of its 33 accepted original states to rejection.

We have updated the paper accordingly, with changes shown in blue: Table 4 adds the rows "Human" and "Majority vs. Human", Sec. 5.1 adds two sentences reporting the human comparison and the corruption audit, Appendix A.6.2 describes the annotation protocol and reports the per-judge errors (Table 15), and Appendix A.6.3 reports the corruption audit (Tables 16 and 17).

Reviewer_dUCzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WildAWM synthesizes persistent, persona-based computer environments combining GUI, MCP, and CLI access, then generates tasks and programmatic verifiers using execution-based quality gates. The resulting 341 tasks support teacher-trajectory SFT and online RL of two Qwen models. The paper reports improvements across eight external benchmarks and analyzes synthesis quality, training components, and interface-use behavior.

Critical strengths
  • The environment-generation contribution is concrete and technically substantial: cumulative world states, deterministic replay checks, interface-grounded tasks, successful reference rollouts, and executable outcome checks are integrated into one pipeline. The detailed examples and evolving synthesis skills make the approach considerably more inspectable than a high-level data-generation recipe.

  • Evaluation covers two model scales and eight external benchmarks under a common harness. Average reported gains are 10.58 and 8.21 points. Gate ablations, RL-component ablations, and SFT/RFT behavior comparisons provide useful supporting evidence. Particularly informative is the acknowledgement that RL improves external transfer while not improving in-domain pass@1 over SFT, and that the hardest task quartile remains largely unsolved.

Critical weaknesses
  • Verifier validation remains primarily model-relative. The reported 95% agreement on 100 rollouts is encouraging, but GLM-5.2 also generates and judges synthesis artifacts, and other judges may share errors induced by the rubric or evidence representation. The counterfactual gate tests resistance to an incorrect asserted verdict; it does not directly measure false acceptance under deliberately corrupted final states. The appendix supplies a wrong-product example, but no systematic independently adjudicated corruption audit. This limits the strength of the reliable-reward claim.

  • The experiments establish gains over base models but do not isolate the value of cumulative multi-interface worlds against alternative training data under matched teacher, harness, trajectory budget, and RL compute. Existing-method numbers explicitly use different harnesses or unavailable checkpoints. The substantial SFT contribution makes this distinction consequential: the evidence supports effective synthetic supervision more directly than superiority of this particular environment construction. The gate ablations address individual filters, not this comparison.

Questions for authors
  1. Can you provide an independently adjudicated verifier audit with plausible wrong-recipient, wrong-quantity, partially completed, and stale-state outcomes, reporting false acceptance and rejection separately?
  2. Can you compare against a matched simpler or existing synthetic environment pipeline? If infeasible, which conclusions about persistent worlds and cross-interface synthesis can be narrowed to reflect the available controls?
3 comments
AuthorsOpenReview ↗

Response to Reviewer dUCz (1/2)

Thank you for your thorough review and constructive suggestions. We would like to address each concern by point-to-point response.

Weakness 1 & Question 1

Verifier validation remains primarily model-relative. The reported 95% agreement on 100 rollouts is encouraging, but GLM-5.2 also generates and judges synthesis artifacts, and other judges may share errors induced by the rubric or evidence representation. The counterfactual gate tests resistance to an incorrect asserted verdict; it does not directly measure false acceptance under deliberately corrupted final states. The appendix supplies a wrong-product example, but no systematic independently adjudicated corruption audit. This limits the strength of the reliable-reward claim.

Can you provide an independently adjudicated verifier audit with plausible wrong-recipient, wrong-quantity, partially completed, and stale-state outcomes, reporting false acceptance and rejection separately?

We would like to clarify that, to reduce reliance on the synthesis model, Table 4 already includes two proprietary frontier judges not used in synthesis, GPT-5.6-Sol and Claude-Opus-5, each run under multiple harnesses. The verifier agrees with each of them on 92% of rollouts (k = 0.83 and 0.84).

Following your suggestion, we further conducted a blind manual audit of all 100 rollouts, comparing the final world states against task requirements without seeing existing verdicts. The verifier reliability study with the new human rows is:

ComparisonCons.PrecisionRecallF1κ
Verifier vs. GLM-5.20.950.980.930.950.90
Verifier vs. GPT-5.6-Sol0.920.850.980.910.83
Verifier vs. Claude-Opus-50.920.980.880.930.84
Verifier vs. Judge majority0.950.980.930.950.90
Verifier vs. Human (new)0.880.900.870.880.76
GLM-5.2 vs. Opus-50.971.000.950.970.94
GPT-5.6-Sol vs. Opus-50.891.000.800.890.79
Majority vs. Human (new)0.930.920.940.930.86

Taking the human label as ground truth (52 successes, 48 failures), we also report false acceptance and false rejection separately:

JudgeCons.κFalse acceptanceFalse rejection
Our verifier0.880.7610.4%13.5%
Judge majority0.930.868.3%5.8%

We find that:

  1. The verifier, which only executes scripts at RL time, agrees with humans in 88% of cases.
  2. Its errors are balanced between false acceptance and false rejection, and 90% of the rollouts it rewards are human-confirmed successes.

Regarding deliberately corrupted final states, we agree that the counterfactual probe tests robustness to asserted verdicts rather than false acceptance of incorrect states. To address this concern, we conducted an additional blind corruption audit on these rollouts. For each rollout, we instructed an independent coding agent (Claude Opus 5.5), without access to the verifier code or any verdicts, to edit the final state so that it remained plausible but no longer satisfied the task requirements. We keep 41 pairs whose reference labels are successful for the original state and failed for the corrupted state. We run the verifier on both states of each pair only afterwards:

Verifier verdictCorrupted rejectedCorrupted acceptedTotal
Original accepted31233
Original rejected808
Total39241

By corruption type:

Corruption typeCasesFalse acceptance
Wrong recipient90/9
Wrong quantity81/8
Partially completed40/4
Stale state61/6
Wrong output content110/11
Other30/3
Total412/41

We have the following observations:

  1. The verifier accepts only 2 of the 41 corrupted states (false acceptance 4.9%), and none of the wrong-recipient, partially completed, or wrong-output-content corruptions, demonstrating its robustness against these plausible task violations.
  2. Among the 33 original states it accepts, the corruption flips its verdict to failure in 31 cases (93.9%), so the rejections are caused by the task violation rather than by a uniformly strict verifier.
  3. On the original states, it rejects 8 of 41 (false rejection 19.5%). Such errors withhold rewards from some correct trajectories rather than incorrectly rewarding failed ones, making them less susceptible to exploitation during RL.

We have updated the paper accordingly, with changes shown in blue: Table 4 and Sec. 5.1 add results of verifier agreement with human and the corruption audit. Appendix A.6.2 describes the annotation protocol and reports the per-judge errors (Table 15), and Appendix A.6.3 reports the corruption audit (Tables 16 and 17).

AuthorsOpenReview ↗

Response to Reviewer dUCz (2/2)

Weakness 2 & Question 2

The experiments establish gains over base models but do not isolate the value of cumulative multi-interface worlds against alternative training data under matched teacher, harness, trajectory budget, and RL compute. Existing-method numbers explicitly use different harnesses or unavailable checkpoints. The substantial SFT contribution makes this distinction consequential: the evidence supports effective synthetic supervision more directly than superiority of this particular environment construction. The gate ablations address individual filters, not this comparison.

Can you compare against a matched simpler or existing synthetic environment pipeline? If infeasible, which conclusions about persistent worlds and cross-interface synthesis can be narrowed to reflect the available controls?

Thank you for this valuable suggestion. We would first like to clarify that the paper also includes two results that do not rely on the teacher. RL in WildAWM without any SFT warm start still raises Qwen3.6-35B-A3B from 42.84 to 48.11 on average (Table 6), and RFT on the base model's own successful trajectories gains 73 net OOD tasks (Table 21 in Appendix). These results suggest that the environments can provide useful learning signals independently of teacher-generated supervision.

More importantly, we understand that the reviewer's main concern is whether our multi-interface worlds offer additional gains over alternative synthetic data under matched settings. We train Qwen3.6-35B-A3B with SFT and then RL in environments from three sources, holding fixed the teacher (GLM-5.2), the harness and tools (OpenCode with GUI and MCP tools), the training compute, and the SFT and RL recipes. We use the open-source released environments and verifiers of CUA-Gym (which synthesizes GUI interface only) and AWM (which synthesizes MCP interface only).

ModelO2WWAQCBCLECUGOSVOS-2ALEAvg.WildAWM-val
Qwen3.6-35B-A3B69.6053.5457.8638.0053.7333.1613.2223.5742.845.68
CUA-Gym-SFT (GUI)71.5459.2958.3855.0070.4130.119.5227.4147.718.33
AWM-SFT (MCP)73.9956.1954.6353.0031.2838.8112.0418.7842.3414.39
WildAWM-SFT (XAI)79.8561.5062.7157.3355.3736.4211.2027.4448.9837.30
CUA-Gym-RL (GUI)73.4365.0459.4256.0071.0047.3516.3024.9951.696.06
AWM-RL (MCP)66.9163.2755.5053.3368.8936.0715.0123.1247.7612.12
WildAWM (XAI)80.7767.2661.4454.6765.9549.2221.7326.3153.4235.98

We find that:

  1. With the same teacher and budget, WildAWM gives the best SFT average (48.98 vs. 47.71 for CUA-Gym and 42.34 for AWM) and is best on most benchmarks. This suggests that the gains cannot be attributed solely to the teacher.
  2. With the same RL compute, WildAWM remains the best on average and leads on six of eight benchmarks. Excluding CUG, which is in-domain for CUA-Gym, its lead grows to 2.70 and 6.89 points.
  3. No single-interface source is best everywhere: CUA-Gym is the strongest on CUG (which is exactly the in-domain eval) and AWM on the OSWorld (which has a lot of MCP tools), but neither consistently leads across benchmarks.
  4. On WildAWM-val, the AWM-RL and CUA-Gym-RL models achieve 12.12 and 6.06 pass@1, respectively, compared with 35.98 for WildAWM, suggesting that cross-interface training develops capabilities less effectively learned from single-interface environments.

Regarding the scope of our conclusions, under these controlled conditions, WildAWM's cross-interface worlds show stronger transfer than the GUI-only and MCP-only alternatives. We have updated the paper accordingly: Table 3 now includes CUA-Gym-SFT, AWM-SFT, WildAWM-SFT, CUA-Gym-RL, and AWM-RL.

AuthorsOpenReview ↗

We would like to hear back from Reviewer dUCz

Dear reviewer dUCz,

Thank you for your time and efforts in reviewing our paper and responses. As the discussion DDL is approaching, we would like to follow up to see if the response addresses your concerns. We would greatly appreciate the opportunity to discuss with you.

Thank you again for your time and valuable suggestions.

Sincerely, Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Paper Revision

We sincerely thank all reviewers for their insightful and constructive feedback. Following these comments, we have revised the paper (new PDF uploaded) and highlighted the main changes in blue. Below, we summarize all changes:

Additional Experimental Results:

  • Matched comparison with existing environments. We trained Qwen3.6-35B-A3B in the released environments of CUA-Gym (GUI only) and AWM (MCP only) under the same settings. Table 3 now includes CUA-Gym-SFT, AWM-SFT, WildAWM-SFT, CUA-Gym-RL, and AWM-RL. Sec. 4 ("Model & Baselines") describes the setup, and point (3) of "Main Results" reports the comparison. (Reviewer dUCz, Reviewer 26WT, Reviewer XKSb)
  • Human assessment of tasks. We add human assessments of tasks from WildAWM, AWM, and CUA-Gym for validity and multi-interface use, reported in Appendix A.6.1 (Table 14) and summarized in Sec. 5.1. (Reviewer XKSb)
  • Verifier agreement with human labels. We now add human verdicts for the verifier reliability study. Table 4 adds the rows "Human" and "Majority vs. Human", and Appendix A.6.2 (Table 15) reports false acceptance and false rejection for each judge. (Reviewer dUCz, Reviewer 26WT, Reviewer XKSb)
  • Corrupted-state audit. We added a blind audit on original and plausibly corrupted final states, with results by corruption type, in Appendix A.6.3 (Tables 16 and 17) and Sec. 5.1. (Reviewer dUCz, Reviewer 26WT, Reviewer XKSb)
  • Ablation on persona diversity. We added a fixed-budget ablation that varies the persona coverage of the training tasks in Appendix A.4.1 (Table 13) and summarized in Sec. 5.2. (Reviewer XKSb)
  • Synthesis cost and acceptance at scale. Table 2 now reports the cost per accepted task. Sec. 3.2 summarizes the cost breakdown, and Appendix A.5 details the pricing. The new panel Figure 4d and Sec. 5.1 show the acceptance rate as the persona pool grows. (Reviewer 26WT)
  • Evaluation robustness. Appendix A.3 adds three analyses: the 4B models re-evaluated with Qwen3.5-4B itself as the vision tool (A.3.1, Table 8), the statistics of the excluded tasks with the intersection-subset and blocked-as-failure averages (A.3.2, Table 9), and five independent runs with confidence interval (A.3.3, Table 10). (Reviewer 26WT)

Improved Description:

  • "Implementation Details" in Sec. 4 and Appendix A.1 now state that only the large open-source synthesis models are served with NVFP4, while all Qwen models are evaluated in BF16. (Reviewer 26WT)
  • Sec. 5.3 now states that RL in WildAWM without any teacher supervision still improves the base model from 42.84 to 48.11. (Reviewer dUCz, Reviewer 26WT)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.