acceptodds
Under review as a conference paper at ICLR 2027

CUWorldEval: Learning Worlds for Multi-Turn Computer-Use Agent Evaluation

Abstract

Evaluating computer-use agents requires knowing whether they can complete tasks through interaction, not just match recorded actions. Offline action matching is inexpensive but cannot capture the consequences of alternative actions, while evaluation in real applications is costly to repeat. We introduce CUWorldEval, a learned environment that predicts screen descriptions and available controls after each action, allowing agents to attempt multi-turn tasks without executing the underlying applications. The key challenge is that plausible individual predictions do not guarantee reliable task evaluation. Errors can accumulate, change the agent's subsequent actions, and lead to task outcomes that differ from real software. We propose Multi-Turn On-Policy Distillation (Multi-Turn OPD), which combines error-conditioned teacher training with distillation on student-generated interaction histories. We first train a teacher to predict reference next states from the student's imperfect histories, using full reference-trajectory summaries to guide correction. We then freeze the teacher and distill it on fresh student rollouts under recorded actions. Each predicted state becomes part of the next turn's history, training the student to continue despite its own errors rather than only from accurate reference states. We evaluate the same agents on matched Word, Excel, and PowerPoint tasks in CUWorldEval and real software. For the two agents with offline baselines, CUWorldEval yields higher balanced accuracy, and it reaches up to 76% agreement with real-software task outcomes. These results support learned environments for interactive agent evaluation and show why their reliability must be assessed through complete task outcomes, not only individual predictions. Code is available at https://anonymous.4open.science/r/CUWorldEval-05EB.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world model, computer using agent;opd

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_F9tJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The motivation is that evaluating computer agents on real desktop apps and live websites is not always feasible because of cost and irreversible consequences (e.g. sending a message). Hence there is a high value in constructing world models (WM) of apps and websites. This paper proposes, Multi-Turn On-Policy Distillation (Multi-Turn OPD), to train an open base model (e.g. Qwen3.5-9B) to predict a text representation of the computer state and availabe actions, given a current trajectory. Multi-Turn OPD has two phases, involving teacher and student models:

  1. starting with a base set of trajectories and some fixed agent, generate next state predictions using the student WM and train the teacher WM to predict the ground truth next states from agent trajectories generated using the student-WM. The teacher is given privileged data (summaries of the base trajectory data) in this phase.
  2. Distill the teacher WM into the student WM

Experiments were done on desktop Office apps (Word, Excel, Powerpoint). Evaluation of WM accuracy is done by a surrogate measure of agreement between task outcomes when a fixed agent executes on the WM and on the actual app, as judged by some other strong VLM.

Critical strengths

The topic is timely. That's all I can say about this work.

Critical weaknesses

Critical: This part of the method makes no sense: using the student WM's predicted state as part of the history, pairing it with the original trajectory's action, and training the teacher WM to predict the original trajectory's next state. Suppose your task is "open the file menu in Excel" and due to some error by the student WM, your history hth_t has a blank desktop page as the current predicted state sts_t, and your recorded action ata_t is click("file menu") (actually correct if excel were open), and suppose your dataset's next state st+1s_{t+1} is a correct Excel app with an open file menu. How does this training tuple make any sense? Answer me this: If I tell you the current state is a blank desktop and the action taken is click("file menu"), would you tell me the next state should be an open Excel window with the file menu open? Seriously? This makes no sense.

Other problems:

  1. Your chosen apps (excel, word, powerpoint) for evaluating the method are all desktop apps where every action is easily reversible. There is no practical motivation use a world model for these apps at all. I know your method can be applied to problems that have irreversible or hard-to-reverse steps (e.g. sending an email, making a payment), but seriously, choosing these desktop apps in a world modeling paper is very poor motivation.
  2. Unclear from the main paper how many steps are needed for the tasks, so it's unclear how effective the WM will be in practice for long task (e.g. OSWorld 2.0) that would take a human >1hr and hundreds to steps to solve.
  3. Unclear from the main paper the latency and cost of running the world model. If the world model takes longer to run and needs a GPU (costly these days), I'll just use the actual app for eval anyway and use some other heuristics to get around the irreversible points. A windows virtual machine is less than $100 per month, while a GPU node for running your world model easily costs 10x more.
  4. The WM not being able to work with pure screenshot-only agents is a critical limitation. Most SoTA agents (opus, gpt) are screenshot-only.
  5. If the app has some hidden state transitions (e.g. backend database or server state) that is not immediately shown in the UI, then how would the world model construction work?
  6. The offline action-matching baseline is a terrible baseline. Many computer tasks are pretty much all or nothing, where success requires all actions to be optimal. Saying success when half of predicted actions match a recorded trajectory is a terribly loose condition.
  7. Task-outcome protocol section: Why not also 5 rollouts per task in the world model? If your intent is to test whether the world model can serve as a surrogate of the VM, then you also need multiple rollouts of the world model to evaluate reliability in multiple runs.
  8. Your judging protocol assumes that the task completion is checkable from the final state alone. There are many tasks with intermediate criteria that require the full trajectory to be judged.

Minor:

  1. Section 3.1 writes initial screenshot and initial state separately. The screenshot is already part of the initial state, no point separating them.
  2. Section 3.1: the available controls at some time step is part of the action space, it should not be confused with the state.
  3. Section 3.2: You use the term "reference summaries" in the "From response-level OPD..." paragraph without explanation, then you describe it later in another paragraph "Error-conditioned teacher preparation". Improve your writing structure.
  4. Unclear what model is the teacher. Still Qwen3.5-9B? Not clear from the main paper.
Questions for authors

None that could affect my score.

12 comments
AuthorsOpenReview ↗

Reply to Reviewer F9tJ

Thank you for your detailed review and for the time you invested in examining our training and evaluation design. Below, we address your questions about the supervision target, evaluation cost, and the reliability of the outcome judgments.

Reply to W1 (Critical): Training the teacher on student-generated histories

We thank the reviewer for this concrete example, which helps clarify our training setup. Besides the state predicted by the student, the teacher model also receives two other kinds of inputs, and taking them into account makes the meaning of this training example clearer.

(1) The history contains not only the states predicted by the student. It also contains the task instruction, the real initial screenshot O_0 and initial state S_0, and all previously recorded actions together with the current action. These all come from the real trajectory rather than being generated by the model.

(2) The teacher model can also see the summary Z of the entire reference trajectory. The very purpose of training the teacher is to enable it, with the help of this privileged information, to predict the correct next state even when the student's history may contain errors (Section 3.2).

With these inputs in mind, consider the example again: Excel is already open at the start of the task, no subsequent action closes Excel, and the recorded action clicks a control that actually exists in Excel. Taking these inputs together, along with the fact that the action trajectory is fixed, "Excel open with the File menu expanded" is the correct inference, whereas "blank desktop" is the one that contradicts all other information and is clearly a prediction error of the student. Z describes the state change at each step of the reference trajectory, and the teacher can use it to recognize this error and give the correct next state.

A predicted state is only the model's estimate, not the environment itself. If the model predicts incorrectly, the state of the real application does not change accordingly, so the real next state remains the correct training target. Another possible design would be to let the model "keep predicting along the erroneous state". We did not adopt this design for three main reasons:

  1. No supervision signal. States hallucinated by the model never appear in real software, and there is no real data that could serve as the training target for "what happens when clicking the File menu on a blank desktop".
  2. Errors become entrenched. Once an error occurs, the model would keep following it, which is exactly the error-accumulation problem we aim to solve.
  3. It contradicts the purpose of an evaluator. The goal of the world model is to reproduce task outcomes in real software. A world model that is internally consistent but has already drifted from reality will inevitably give results inconsistent with the real environment.

Training on the model's own generated histories with the real trajectory as the target is a standard approach to error accumulation in sequence prediction and multi-step model learning, e.g., Scheduled Sampling [1], Data as Demonstrator [2], and DAgger [3]. On this basis, we additionally provide the teacher model with the privileged summary Z, further removing ambiguity in the target so that the teacher can still provide supervision pointing to the real trajectory when the student's history contains errors. This is the part of our method specifically designed to guide the student model. The experimental results also support this design. Compared with SFT, CUWorldEval's single-step description consistency is essentially unchanged (79.00 and 78.71), but in multi-turn replay, where the history is generated by the model itself, description consistency rises from 70.93 to 72.24, and the task-outcome BA improves on all three agents (Table 3). Meanwhile, continuing SFT with the same data and number of training epochs yields a BA 6.04–11.92 percentage points lower than that of CUWorldEval.

References:

[1] Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

[2] Improving Multi-Step Prediction of Learned Time Series Models

[3] A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

AuthorsOpenReview ↗

Reply to W2 (Other 1): Motivation for choosing Office applications

We thank the reviewer for the comment. We first clarify the motivation of this paper and then explain why we chose Office as the testbed.

(1) The starting point of this paper is to reduce evaluation cost and improve scalability, not to avoid irreversible operations. The motivation in the introduction is that evaluating in real environments requires setting up the environment, executing actions, and verifying results; repeating these steps across many tasks, agents, and runs is costly, so an evaluation approach is needed that is cheaper but still retains full interaction. Reversibility does not remove reset, execution, and verification overhead; these are the costs measured in our repeated-evaluation setting. In a virtual machine, a snapshot must be restored and a template reopened before every rollout, every step requires taking a screenshot and converting it into a textual description, and timeouts require reruns; these overheads have nothing to do with whether operations are reversible. In our experiments, the VM throughput is only 8.71–15.08 tasks/h, whereas the world model reaches 35.84–114.33 tasks/h, 2.91–7.77× faster; one evaluation of 100 tasks × 5 rollouts takes 33–57 hours on the VM but only 4–14 hours with the world model. Research often requires repeatedly comparing multiple checkpoints, configurations, and agents, which multiplies this gap. In addition, every VM instance used for evaluation requires a license for commercial software such as Office; the larger the degree of parallelism, the heavier the burden of subscription fees and account management. The world model requires real software only when collecting training data, and evaluation can be run in batches directly on GPUs.

(2) We chose Office because it can be used to validate the world model. To test whether a world model can replace the real environment, the same agent must be able to execute the same tasks repeatedly in both environments so that the results can be compared. Irreversible scenarios such as sending emails or making payments cannot be executed repeatedly in this way, so the reliability of the world model cannot be tested there. Only after demonstrating that the world model agrees with the real environment in a verifiable domain is there a basis for extending it to irreversible scenarios.

(3) Office is itself one of the most important and hardest-to-model scenarios for computer-use agents. For example, Excel has on average 374.5 controls per step before filtering, along with many menus, dialogs, and formatting states, which pose a stringent test of state prediction and long-horizon error accumulation.

AuthorsOpenReview ↗

Reply to W3 (Other 2): Number of task steps and long-horizon tasks

We thank the reviewer for the comment. The reviewer asks how many steps the test tasks require and whether the world model is effective on longer tasks. Statistics on task steps are given in the test-set statistics in the appendix; we explain in two points below.

(1) Task length. The reference trajectories of the 100 test tasks have 2–12 steps, 5.24 steps on average; among them, 28 tasks have 2–3 steps, 31 have 4–5, 27 have 6–8, and 14 have 9–12. PowerPoint tasks are the longest, with 20 of 33 tasks requiring at least 6 steps. For comparison, OSWorld-Human annotates the minimum number of human steps required to complete OSWorld tasks (counting each action as one step): on average 7.5, 7.8, and 13.2 steps for LibreOffice Writer, Impress, and Calc, respectively, showing that our task lengths are of the same order of magnitude as the office tasks in OSWorld.

(2) Reliability on long tasks. The longer the task, the more pronounced the accumulation of the world model's own prediction errors, which is exactly the problem Multi-Turn OPD aims to solve. Table 2 shows that from single-turn replay to multi-turn replay, all state metrics of general-purpose models drop markedly. Taking GPT-5.5 as an example, description consistency, control hit rate, Text+Type, Macro IoU, and Micro IoU drop by 17.59, 10.50, 12.87, 18.26, and 16.15 percentage points, respectively, whereas CUWorldEval's metrics drop by only 4.31–6.47 percentage points. In multi-turn replay, CUWorldEval's description consistency (72.24), Text+Type (54.65), Macro IoU (66.55), and Micro IoU (59.76) are all the highest among all models.

Reviewer_pS6bOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

CUWorldEval trains a learned environment to predict the next screen description and available controls in Office software. Agents complete multi-turn tasks in this environment, and their outcomes are compared with runs in real software. Multi-Turn OPD trains the environment to recover from its own accumulated prediction errors. On 100 Word, Excel, and PowerPoint tasks, balanced accuracy rises from 63.41% to 74.40% for one agent and from 61.95% to 74.82% for another, relative to offline action matching (Table 1)

Critical strengths
  • It evaluates agreement on task completion, which is more relevant than matching individual actions or screens.

  • The learned and real environments use a shared agent interface. The single-step, multi-turn, and outcome experiments make the effects of accumulated errors visible.

Critical weaknesses
  • There is only one learned-environment rollout per task. The reported standard deviation captures judge variation, not variation between simulated trajectories.

  • The offline baseline treats a task as successful when more than half its actions exactly match a recorded path. This may undervalue other valid paths. The continued-SFT and OPD training budgets are also unmatched.

Questions for authors
  • Could blinded human review of both environments’ final states, or checks of actual document state, measure how often a simulated description falsely implies success?

  • How do balanced accuracy, error types, and confidence intervals change with multiple learned-environment rollouts per task?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer pS6b

Thank you for your constructive suggestions regarding rollout variability, the training-budget comparison, and blinded validation. Below, we explain the repeated-rollout results, the scope of the continued-SFT control, and what the human assessment establishes.

Reply to W1: Only one world-model rollout per task

Table R3. Task-outcome agreement with 5 world-model rollouts × 5 GPT-5.2 judge passes per task. Each cell reports the majority-vote result (each rollout is labeled by majority over its 5 judge passes, and a task counts as successful if at least 3 of its 5 rollouts are successful) / mean ± std over the 5 rollouts.

AgentWorld modelCons.BAP/N
GPT-5.5Offline Data Evaluation60 / 59.00±2.3562.64 / 62.66±1.7559.60 / 58.66±2.26
Code2World45 / 44.20±1.9255.71 / 53.78±1.7142.94 / 42.75±2.23
Qwen3.5-9B55 / 53.60±2.3064.73 / 61.67±1.9854.23 / 52.96±2.47
Qwen3.5-27B64 / 67.00±2.9261.10 / 64.59±2.9760.94 / 64.32±2.99
Qwen3-VL-32B67 / 64.80±5.3662.75 / 61.98±4.7662.97 / 61.72±4.95
Grok-4.567 / 68.40±2.0759.45 / 60.92±1.2459.51 / 61.11±1.28
Gemini-3.5 Flash71 / 69.40±3.7868.46 / 66.70±3.8368.34 / 66.40±3.60
GPT-5.572 / 69.80±3.1161.98 / 60.55±4.0261.62 / 60.14±5.04
CUWorldEval72 / 69.60±2.4177.14 / 73.98±2.9271.90 / 69.41±2.44
Qwen3.5-27BOffline Data Evaluation64 / 64.40±1.1463.17 / 63.43±1.0963.05 / 63.37±1.11
Code2World60 / 60.00±1.8754.57 / 54.49±1.8252.38 / 52.18±2.05
Qwen3.5-9B61 / 61.40±0.5556.16 / 56.20±0.5654.81 / 54.44±0.96
Qwen3.5-27B62 / 63.40±1.5261.10 / 62.66±1.6361.00 / 62.50±1.57
Qwen3-VL-32B65 / 65.20±2.5965.87 / 65.67±2.2564.83 / 64.86±2.46
Grok-4.567 / 68.00±1.8767.94 / 69.24±2.2566.84 / 67.77±1.86
Gemini-3.5 Flash67 / 66.80±1.6466.83 / 66.95±1.7766.43 / 66.35±1.64
GPT-5.569 / 68.60±0.8970.75 / 70.41±0.8868.97 / 68.56±0.88
CUWorldEval75 / 75.60±0.8972.12 / 73.22±1.4672.70 / 73.71±1.37

We thank the reviewer for pointing this out. We add 5 independent rollouts for all world models in Table 1 and also rerun the offline baseline 5 times; each result is judged 5 times by the GPT-5.2 judge (see Table R3). We report two kinds of results: first, majority-vote metrics (each rollout is first labeled by majority over its 5 judgments, and a task is counted as successful if it succeeds in at least 3 rollouts); second, the mean and standard deviation across the 5 rollouts, the latter reflecting exactly the variation between simulated trajectories pointed out by the reviewer.

The results show that on both agents, GPT-5.5 and Qwen3.5-27B, CUWorldEval has the highest mean BA: on the GPT-5.5 agent it is 73.98±2.92, versus 66.70±3.83 for the second-best Gemini-3.5 Flash; on the Qwen3.5-27B agent it is 73.22±1.46, versus 70.41±0.88 for the second-best GPT-5.5. To further test whether these leads are significant, we perform Welch's t-tests using the BA of each rollout as samples: on the GPT-5.5 agent, CUWorldEval leads Gemini-3.5 Flash by 7.28 percentage points (p≈0.011); on the Qwen3.5-27B agent, CUWorldEval leads GPT-5.5 by 2.81 percentage points (p≈0.009). We will replace Table 1 with the 5-rollout results.

AuthorsOpenReview ↗

Reply to W2: Threshold of the offline baseline and training budget

We thank the reviewer for the comments. The reviewer has two concerns: the offline baseline uses "more than half of the actions match the recorded path" as the success criterion, which may undervalue other equally valid completion paths; and the training budgets of continued SFT and CUWorldEval are unmatched. We respond in two parts.

(1) The success criterion of the offline baseline. We agree that the criterion "more than half of the actions exactly match the recorded path" undervalues other equally valid completion paths. This is precisely an inherent limitation of offline evaluation, and also the starting point for proposing interactive world-model evaluation. To test whether the conclusion depends on the choice of threshold, we sweep the threshold of the offline results from 0 to 1. Even if the threshold most favorable to the offline baseline is selected directly on the test set, its best BA is only 63.41 (GPT-5.5 agent, threshold 0.50) and 64.96 (Qwen3.5-27B agent, threshold 0.65), still clearly lower than CUWorldEval (74.40 and 74.82 with a single rollout; means of 73.98 and 73.22 over 5 rollouts). At low thresholds, the offline baseline judges 82%–85% of the tasks as successful, with many more false positives; at high thresholds, it misses most of the true successes. Thus, the problem lies not in how the threshold is chosen but in the path-matching judging approach itself.

(2) Consistency of the training budget. Continued SFT and CUWorldEval are trained under the same conditions: both start from the same SFT checkpoint, use the same tasks, and train for the same number of epochs. The teacher-preparation stage of CUWorldEval also uses the SFT-stage tasks and introduces no additional tasks or trajectories. In this setting, the BA of continued SFT on GPT-5.5, Qwen3.5-27B, and Gemini-3.5 Flash is 64.29, 62.90, and 59.46, which is 10.11, 11.92, and 6.04 percentage points lower than that of CUWorldEval (74.40, 74.82, and 65.50). Since the two use the same amount of data and number of training epochs, this gap cannot simply be attributed to more data or longer training.

AuthorsOpenReview ↗

Reply to Q1: How to measure how often simulated descriptions falsely imply task success

We thank the reviewer for suggesting a blinded assessment. We use two complementary checks and distinguish their scope.

(1) Under five-rollout majority voting, CUWorldEval's judge-based false-positive rate relative to the VM reference is 5.7% for the GPT-5.5 agent and 11.9% for the Qwen3.5-27B agent. These quantify erroneous success outcomes relative to the VM reference, which is the task-outcome fidelity target of this evaluation. The agents interact independently in the two environments, so these rates are measured at the task-outcome level.

(2) Six annotators independently assess 100 CUWorldEval terminal states from 50 sampled tasks, without seeing the judge's decisions. Agreement with the judge is 82%–88%, with Cohen's κ of 0.64–0.76. This provides human validation of the completion decisions underlying the evaluation. State-prediction fidelity under recorded actions is measured separately by the replay evaluations. These checks are distinct from a direct audit of fabricated descriptions under an identical action sequence.

Reviewer_Xt1MOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors use a Qwen3.5-9B world model to lets agents attempt Word, Excel, and powerpoint tasks without the applications, predicting a screen description and control list after each action. It is trained with Multi-Turn On-Policy Distillation (OPD): a teacher that sees a full-trajectory summary learns from the student's own predicted histories, then supervises the student on fresh rollouts. Three agents run on 100 held-out tasks in the world model and a VM with a GPT-5.2 judge comparing outcomes; for two of them, its balanced accuracy on VM outcomes exceeds offline action matching by about more then 10 points.

Critical strengths
  • The VM and the world model run the same agent, prompt, and task, and only outcomes are compared (p.3).
  • Mixed results are reported: OPD lowers one-step description consistency (79.00 to 78.71) but raises multi-turn consistency and balanced accuracy for all three agents (Table 3, p.8).
Critical weaknesses
  • Among the training variants, the full method has the best balanced accuracy for all three agents, but no run cleanly isolates the distinguishing design of Multi-Turn OPD. The response-level OPD contrast of Section 3.2 has no matching run, and the one teacher ablation, OPD (SFT-student), reaches only 52.50 balanced accuracy with the Qwen3.5-27B agent, below SFT (64.96). The text never discusses that score, and run-to-run variance is not reported? (Table 4).

  • Evidence for a useful evaluator is thin: 100 tasks, one world-model rollout each, and error bars that cover judge passes only (p.6), which cannot bound the 3.4 point balanced-accuracy lead over Grok-4.5 with the Qwen3.5-27B agent.

  • The real-software outcome is an LLM-judged text proxy, with the judge reading final-state text and action history against a GPT-5.5-generated reference. Humans agreed with the adopted setup on 71 of 77 VM pairs from 29 tasks, but it was the best of four compared on those pairs, and validation on world-model terminal states is left to future work (Table 12).

  • Ths might be minor, while GPT 5.2 appears a bit outdated given its release time. i consider this is only due to the current fast model update time?

Questions for authors
  1. What are the VM success rates and confusion counts behind Table 1 (i.e., TP, FP, FN, TN)? Does CUWorldEval over- or under-report success?
  2. Can you add task-level bootstrap intervals for balanced accuracy in Tables 1 and 4?
  3. In OPD (SFT-student), does the teacher see Z, and what explains the 52.50?
  4. Was the corrected completion reference (Table 12) built with the 77 validation pairs in view?
9 comments
AuthorsOpenReview ↗

Reply to Reviewer Xt1M

Thank you for your detailed questions about the baselines, statistical uncertainty, and judging protocol. Below, we respond with the relevant comparisons, task-level uncertainty estimates, and details of reference construction and human validation.

Reply to W1: Ablation of the key design, and why OPD (SFT-student) performs poorly on Qwen3.5-27B

We thank the reviewer for the detailed comments. The reviewer's concerns are mainly threefold: whether the existing ablations examine the key design of Multi-Turn OPD, why OPD (SFT-student) reaches only 52.50 on Qwen3.5-27B, and whether the results in Table 4 are affected by rollout variance. We respond in three points.

(1) Examining the key design. The key design of Multi-Turn OPD consists of teacher preparation and recursive student rollouts, and the existing experiments examine them from three aspects.

  • Necessity of teacher preparation. The teacher in OPD (SFT-student) can also see the privileged information Z; its only difference from CUWorldEval is that it skips teacher preparation and directly uses the frozen SFT student as the teacher. It drops markedly on Qwen3.5-27B, whereas CUWorldEval outperforms OPD (SFT-student) on all three agents.
  • Role of recursive student histories. Compared with SFT, CUWorldEval's single-step description consistency is essentially unchanged (79.00 and 78.71), but its multi-turn description consistency rises from 70.93 to 72.24, and its BA improves on all three agents (Table 3). The gains are concentrated in the setting where the history is generated by the model itself, which is exactly the design goal of recursive student rollouts.
  • The gains do not come from extra training. With the same data and number of training epochs, the BA of continued SFT (64.29, 62.90, 59.46) is 6.04–11.92 percentage points lower than that of CUWorldEval (74.40, 74.82, 65.50).

(2) Why OPD (SFT-student) drops markedly on Qwen3.5-27B. Although the teacher of this variant can see Z, it has never learned during training how to use Z, nor has it learned to provide corrective supervision on histories containing errors. Therefore, when the student's history deviates from the reference trajectory, it cannot effectively bring the student back to the correct state; the resulting student tends to stay in the same state after an error and repeatedly gives the same prediction, so that the agent also falls into repeated actions: 75% of the episodes run until the step limit, and many tasks that succeed in the VM are judged as failures. This defect is most visible on Qwen3.5-27B because it retries the same action more often than the other agents (in the VM, 35% of its steps are identical to the previous step, compared with only 5% for Gemini-3.5 Flash). CUWorldEval, with teacher preparation, can correct such deviations: the fraction of episodes reaching the step limit drops to 39.4%, and false negatives drop from 32 to 13.

(3) Variance. For CUWorldEval, we add results with 5 rollouts (see W2): the BA on GPT-5.5 and Qwen3.5-27B is 73.98±2.92 and 73.22±1.46, close to the single-rollout results (74.40 and 74.82) with small standard deviations, which shows that CUWorldEval's results in Table 4 do not come from a single lucky rollout. In addition, we perform a task-clustered bootstrap on Table 4: the macro-average BA gain of CUWorldEval over SFT is +8.47 with a 95% interval of [+2.51, +14.43], which excludes 0, showing that the gain still holds after accounting for the uncertainty of task sampling.

AuthorsOpenReview ↗

Reply to W2: Only one world-model rollout per task

Table R3. Task-outcome agreement with 5 world-model rollouts × 5 GPT-5.2 judge passes per task. Each cell reports the majority-vote result (each rollout is labeled by majority over its 5 judge passes, and a task counts as successful if at least 3 of its 5 rollouts are successful) / mean ± std over the 5 rollouts.

AgentWorld modelCons.BAP/N
GPT-5.5Offline Data Evaluation60 / 59.00±2.3562.64 / 62.66±1.7559.60 / 58.66±2.26
Code2World45 / 44.20±1.9255.71 / 53.78±1.7142.94 / 42.75±2.23
Qwen3.5-9B55 / 53.60±2.3064.73 / 61.67±1.9854.23 / 52.96±2.47
Qwen3.5-27B64 / 67.00±2.9261.10 / 64.59±2.9760.94 / 64.32±2.99
Qwen3-VL-32B67 / 64.80±5.3662.75 / 61.98±4.7662.97 / 61.72±4.95
Grok-4.567 / 68.40±2.0759.45 / 60.92±1.2459.51 / 61.11±1.28
Gemini-3.5 Flash71 / 69.40±3.7868.46 / 66.70±3.8368.34 / 66.40±3.60
GPT-5.572 / 69.80±3.1161.98 / 60.55±4.0261.62 / 60.14±5.04
CUWorldEval72 / 69.60±2.4177.14 / 73.98±2.9271.90 / 69.41±2.44
Qwen3.5-27BOffline Data Evaluation64 / 64.40±1.1463.17 / 63.43±1.0963.05 / 63.37±1.11
Code2World60 / 60.00±1.8754.57 / 54.49±1.8252.38 / 52.18±2.05
Qwen3.5-9B61 / 61.40±0.5556.16 / 56.20±0.5654.81 / 54.44±0.96
Qwen3.5-27B62 / 63.40±1.5261.10 / 62.66±1.6361.00 / 62.50±1.57
Qwen3-VL-32B65 / 65.20±2.5965.87 / 65.67±2.2564.83 / 64.86±2.46
Grok-4.567 / 68.00±1.8767.94 / 69.24±2.2566.84 / 67.77±1.86
Gemini-3.5 Flash67 / 66.80±1.6466.83 / 66.95±1.7766.43 / 66.35±1.64
GPT-5.569 / 68.60±0.8970.75 / 70.41±0.8868.97 / 68.56±0.88
CUWorldEval75 / 75.60±0.8972.12 / 73.22±1.4672.70 / 73.71±1.37

We thank the reviewer for pointing this out. We add 5 independent rollouts for all world models in Table 1 and also rerun the offline baseline 5 times; each result is judged 5 times by the GPT-5.2 judge (see Table R3). Each cell of the table reports two kinds of results: first, majority-vote metrics (each rollout is first labeled by majority over its 5 judgments, and a task is counted as successful if it succeeds in at least 3 rollouts); second, the mean and standard deviation across the 5 rollouts, the latter reflecting exactly the variation between rollouts.

The results show that on both agents, GPT-5.5 and Qwen3.5-27B, CUWorldEval has the highest mean BA: on the GPT-5.5 agent it is 73.98±2.92, versus 66.70±3.83 for the second-best Gemini-3.5 Flash; on the Qwen3.5-27B agent it is 73.22±1.46, versus 70.41±0.88 for the second-best GPT-5.5. To further test whether these leads are significant, we perform Welch's t-tests using the BA of each rollout as samples: on the GPT-5.5 agent, CUWorldEval leads Gemini-3.5 Flash by 7.28 percentage points (p≈0.011); on the Qwen3.5-27B agent, CUWorldEval leads GPT-5.5 and Grok-4.5 by 2.81 and 3.98 percentage points, respectively (p≈0.009 and p≈0.013).

Multiple rollouts show that the variation of single results cannot be ignored: the BA of some baselines differs from the single-rollout results by 3–7 percentage points; for example, the standard deviation of Qwen3-VL-32B on the GPT-5.5 agent reaches 4.76. Nevertheless, CUWorldEval still has the highest mean BA on both agents. We will therefore replace Table 1 with the 5-rollout results.

AuthorsOpenReview ↗

Reply to W3: Reliability of the task-outcome judging protocol

We thank the reviewer for the question. We clarify that the judging protocol is supported by empirical human validation in addition to a shared judge and reference. Below, we address the consistency of judging inputs, human validation of completion judgments on world-model outputs, and reference development and validation.

(1) Shared judging inputs. The VM terminal screenshot is converted into a textual state, and VM and world-model trajectories are evaluated with the same completion reference and GPT-5.2 judge. This controls the input format and judging procedure in the WM–VM comparison, and human validation provides additional evidence about the resulting completion judgments.

(2) Human validation of world-model outcome judgments. We randomly sample 50 tasks, covering 18 Excel, 16 PowerPoint, and 16 Word tasks, and collect 100 terminal states from the GPT-5.5 and Qwen3.5-27B agents in CUWorldEval. Six annotators independently assess completion without seeing the judge's decisions. Their agreement with the judge is 82%–88%, with Cohen's κ of 0.64–0.76. This directly assesses whether the judge's completion decisions on world-model states agree with human judgments, extending validation beyond the original VM samples.

(3) Reference development and validation. The final reference was revised after examining discrepancies on the original 77 validation pairs, so its agreement on those same pairs measures agreement on the development samples. The additional human assessment uses newly sampled terminal states, and none of its 50 tasks belongs to the 29 tasks used for reference development. Annotators saw only an anonymized sample ID, the application category, the task instruction, and the world model's textual description of the final state; they did not see the agent identity, task ID, completion reference, VM outcome, judge outputs, action trajectory, intermediate states, or screenshots. Because annotators did not see the completion reference, this agreement also indicates that reference-based judgments are consistent with independent human judgments of task completion. We therefore treat the 92.2% on the 77 pairs as agreement on the development samples, and the 82%–88% as validation on tasks not involved in the revision.

The human assessment validates completion judgments given the supplied states. State-prediction fidelity is assessed separately through the single-turn and multi-turn replay evaluations and the WM–VM task-outcome comparisons. These are distinct validation targets, and some uncertainty from VM text conversion and reference construction remains.

Reviewer_A1rXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary
  • The authors propose CUWorldEval, a world model for evaluating computer-use agents without executing real applications in virtual machines (VMs). The authors claim that this world model provides a more scalable and cost-efficient alternative to VMs.
  • They propose Multi-Turn On-Policy Distillation (OPD), which trains the world model on its own generated histories to reduce accumulated prediction errors. They evaluate the method by comparing task outcomes between the world model and real VMs on Word, Excel, and PowerPoint tasks.
Critical strengths
  • The authors evaluate the world model through task-outcome agreement with real environments, rather than relying only on next-state prediction accuracy.
  • The authors provide ablation studies comparing different algorithms, such as SFT, SFT + continued SFT, and OPD (SFT-student).
Critical weaknesses
  • The main motivation of the paper is to replace expensive VM-based evaluation with a more efficient learned environment. However, the authors do not provide a comparison of actual evaluation costs, such as wall-clock time, computational cost, or throughput. I believe demonstrating the efficiency improvement is important to justify the practical benefits of the proposed world model.
  • One important application of the learned world model is to serve as a cost-efficient policy evaluator, as motivated by the authors. However, the authors do not sufficiently demonstrate its reliability as a policy evaluator. For example, does the world model accurately estimate the aggregate success rates of different computer-use agents, or at least preserve their relative rankings? I believe evaluating score prediction accuracy and rank correlation across multiple agents is important, as one of the primary purposes of agent evaluation is to compare different agents and identify better-performing ones.
  • Another important application of learned world models is to provide scalable environments for RL fine-tuning. However, the authors do not evaluate whether agents trained in CUWorldEval achieve better real-environment performance compared with agents trained in actual VMs under matched training costs. Moreover, the reported task-outcome agreement of only 65–76% raises concerns about whether the world model can provide sufficiently reliable training signals, potentially leading to incorrect rewards or exploitation of simulation errors. I am not sure whether the proposed learned world model offers sufficient practical benefits over existing VM-based environments, especially given its relatively limited task-outcome agreement.
  • In the authors' VM setting, subsequent screenshots are provided to GPT-5.2 to generate textual descriptions, rather than directly shown to the evaluated agents (which are already VLMs, including Qwen3.5-27B). I believe the authors should additionally consider a multimodal VM setting without intermediate textual descriptions and compare its policy evaluation results, the real-environment performance of RL-fine-tuned agents, and evaluation efficiency (e.g., wall-clock time and throughput) against the proposed learned world model, as I pointed out in previous bullets.
  • In Section 3.2, the authors mention using an LLM to generate summaries of complete ground-truth trajectories, which provide privileged information to the teacher during OPD training. However, they do not specify which LLM was used to generate these summaries or provide sufficient details about the summarization procedure. Since the quality of these summaries may affect the teacher's supervision, I believe these details are necessary for reproducibility.
Questions for authors

See the weakness section, please.

10 comments
AuthorsOpenReview ↗

Reply to Reviewer A1rX

Thank you for your review and for raising questions about evaluation efficiency, outcome fidelity, and the intended scope of CUWorldEval. Below, we address the distinction between evaluation and policy training, the interpretation of the agent-comparison results, and the role of the observation interface. We also clarify the scope of the reported efficiency evidence.

Reply to W1: Lack of a comparison of actual evaluation cost

Table R1. Evaluation throughput of WM and VM with different agents.

AgentWMVMTime for 100 tasks: WM / VMSpeedup
GPT-5.543.84 tasks/h15.08 tasks/h2.28 / 6.63 h2.91×
Qwen3.5-27B114.33 tasks/h14.72 tasks/h0.87 / 6.79 h7.77×
Gemini-3.5 Flash35.84 tasks/h8.71 tasks/h2.79 / 11.48 h4.11×

We thank the reviewer for pointing out that the paper lacks a comparison of actual evaluation costs. We add analyses from three aspects: evaluation throughput, environment preparation cost, and software licensing.

(1) Evaluation throughput. We compare the throughput of WM and VM with three agents (Table R1). WM reaches 35.84–114.33 tasks/h, whereas VM reaches only 8.71–15.08 tasks/h, a 2.91–7.77× improvement. The speedup is largest for locally deployed agents (e.g., Qwen3.5-27B), reaching 7.77×, and most of the models that need to be evaluated repeatedly in research and training are exactly such locally deployed models.

(2) Environment preparation cost. VM-based evaluation requires first setting up the software environment and creating snapshots, which is very time-consuming; for example, OSWorld reports that its task construction took about 1,800 person-hours. Once trained, CUWorldEval can be reused for evaluation repeatedly.

(3) Software licensing. Every VM instance used for evaluation requires a license for commercial software such as Office, with ongoing subscription fees; the larger the degree of parallelism, the heavier the burden of subscription fees and account management. WM requires real software only when collecting training data, and no Office instance needs to run during evaluation.

Therefore, WM is better suited to agent evaluation that must be performed at scale and repeatedly.

AuthorsOpenReview ↗

Reply to W2: Lack of agent ranking evaluation

Table R2. Agent ranking evaluation. 2 Agents: GPT-5.5 and Qwen3.5-27B; 3 Agents: adding Gemini-3.5 Flash; 4 Agents: further adding SFT-631.

MetricAgent SetCUWorldEvalGPT-5.5Gemini-3.5 FlashGrok-4.5
Tie-aware PRA ↑ (%)2 Agents66.0064.0064.0063.00
3 Agents62.0058.6762.0062.00
4 Agents60.1757.5058.3355.83
Success-Rate MAE ↓ (pp)2 Agents10.5018.0012.5021.50
3 Agents10.0018.6713.0022.67
4 Agents7.7517.0014.5020.50

We thank the reviewer for the suggestion. We add an agent ranking evaluation that examines whether the world model preserves the relative ranking among agents and whether it accurately estimates their overall success rates; the results are shown in Table R2. Here, SFT-631 is a computer-use agent trained on GUI-360 data; including it lets the evaluated agents cover a wider range of capabilities.

(1) Ranking agreement. When agents are ranked by overall success rate, all world models agree with the VM (GPT-5.5 > Gemini-3.5 Flash > Qwen3.5-27B > SFT-631), so the overall ranking can hardly distinguish between world models. We therefore further use the stricter tie-aware Pairwise Ranking Agreement (PRA), which counts, within each task, whether the outcome relation of each pair of agents is the same in the VM and in the world model. The PRA of CUWorldEval is 66.00%, 62.00%, and 60.17% in the two-, three-, and four-agent settings; it is higher than all closed-source world models in the two- and four-agent settings and ties for the highest with Gemini-3.5 Flash and Grok-4.5 in the three-agent setting.

(2) Overall success-rate estimation. We compute the mean absolute error (MAE) between the estimated and VM overall success rates of the agents. The MAE of CUWorldEval in the three settings is 10.50, 10.00, and 7.75 percentage points, lower than all closed-source world models (whose lowest values are 12.50, 13.00, and 14.50, respectively), and the advantage grows as the number of agents increases, mainly because closed-source world models tend to overestimate the agents' success rates.

In summary, as a 9B model, CUWorldEval has the smallest overall success-rate estimation error among all world models, and its per-task ranking agreement is comparable to or higher than that of the best closed-source world model.

AuthorsOpenReview ↗

Reply to W3: RL training in the world model

We thank the reviewer for pointing out this application. We first clarify the scope of the paper: CUWorldEval is an evaluation environment, not an environment for training agents. As stated in Section 1, our study evaluates the fidelity of a learned environment for agent assessment, in which the world model "acts as the environment, not as a planner or a training mechanism for the evaluated agent"; RL training tests a separate application of that environment. The related-work section also distinguishes our work from SWE-World, which is used for agent training, and states that our focus is fidelity as an evaluator. The title, abstract, and all experiments of the paper are centered on evaluation, and the paper makes no claim about reinforcement-learning training.

Therefore, "whether agents trained in CUWorldEval perform better in the real environment under the same training cost" is a different research question. We agree that this is an important application of learned environments. However, answering it would require separately designing rewards, handling policies that exploit simulation errors, and conducting cost-matched training comparisons, which is beyond the scope of this paper. We will state this clearly in the Limitations and list it as future work.

The "exploitation of simulation errors" that the reviewer is concerned about mainly arises when a policy is directly optimized to obtain high rewards; in our evaluation setting, the evaluated agents have never been trained in CUWorldEval, so there is no optimization pressure targeting world-model errors. As for whether a task-outcome agreement of 65–76% is sufficient, this number should be compared with the repeatability of the real VM itself rather than with 100%, because the results of the same agent in repeated VM runs are not fully stable. We add a leave-one-out analysis: each time we hold out one VM rollout and compare its results with the majority results of the remaining 4 rollouts; the BA averaged over the 5 folds is 82.71 (GPT-5.5) and 80.05 (Qwen3.5-27B). In other words, even running the real VM once more yields a BA of only about 80–83 (which includes judge variation), and CUWorldEval's 74.40 and 74.82 are about 8 and 5 points below this reference, respectively.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.