- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CUA-Sandbox, an environment backend for computer-use RL. Instead of running a full Docker runtime for each rollout, it shares one initialized application runtime. Each environment gets a private "state capsule": a database branch, a file overlay, and its own session and profile namespaces. Actions are routed to the right capsule by environment ID. Reset, clone and fork swap capsules by pausing the environment, finishing in-flight requests, and switching to the new capsule in one atomic step. The paper evaluates on WebArena, VisualWebArena and OSWorld. It reports task success comparable to or higher than Docker for four models, matched SFT and RL gains for a 4B student, and, at 8 concurrent environments, up to 6.20× higher throughput, 9.2× less memory and 504× less storage per environment.
Critical strengths
- It targets a real bottleneck with a sensible idea. RL on real software needs many independent, resettable environments, and each Docker copy re-initializes the same application. Sharing the initialized application while keeping each environment's state private is intuitive. It keeps the original interfaces and evaluators, and it avoids the sim-to-real gap of learned world models.
- Parts of the evaluation show where the savings come from. App. F separates the two mechanisms. Copy-on-write cuts writable storage for 8 environments from 53 to 25 GiB, and runtime sharing then cuts runtime memory 4.3× (13,882 to 3,261 MiB). A stress test shows why atomic switching matters: without it, actions saw a mix of old and new state in 100/100 trials, and with it in 0/100. RL gains also match across backends (+3.88 vs. +3.87 points on OSWorld).
Critical weaknesses
The paper doesn't show that the shared runtime behaves like the original environment.
Success shifts in one direction. A faithful backend should leave task success unchanged, up to noise. Instead, CUA-Sandbox scores higher in all 12 model–benchmark pairs (sign test ), by up to 10 points. For example, Muse-Glimmer-30B goes from 24.4% to 34.4% on OSWorld. So the two backends aren't equivalent, and the paper doesn't say why. The cause isn't reported. The gap could come from Docker errors and timeouts, which the paper records (App. C) but never reports. It could also come from state leaking between concurrent environments, which would inflate success. Isolation is never tested directly. The non-DB ablation uses a read-only task, and the case studies are illustrations, not runs. In RL, 8 rollouts per task run in parallel, so a leak would silently corrupt rewards. The efficiency gains are smaller and less certain than the headline numbers.
The numbers don't agree. All three headline numbers come from the VisualWebArena column of Fig. 3. Table 6 says it lists the exact measurements behind Fig. 3, but its VisualWebArena values differ for every metric. With Table 6, memory drops 1.75×, not 9.2×, and throughput rises 4.25×, not 6.20×. The main baseline is the weakest one. The main comparisons are against plain Docker. WebServ, the closest prior system, appears only in App. A.2. Against it, launch time is about the same (1.73 vs. 1.78 s), storage is 2.2× lower, and memory isn't measured the same way for both systems. Total memory savings are about 2×. Browser and agent processes don't shrink (about 7,000 MiB for 8 environments, App. F). So total memory drops about 2.1×, while the runtime alone drops 4.8×. The RL speedup isn't measured. Efficiency is measured at 4–16 environments, but RL uses 128 per update, and the paper reports no training time or cost for either backend. In short, the idea is useful and targets a real bottleneck. But the paper doesn't yet show that the shared runtime is faithful, or how much it saves end to end. So I lean toward rejection, though clear answers to the questions below could change that.
Questions for authors
Clarifications
Why does CUA-Sandbox score higher in every pair? Please report errors and timeouts per backend, the number of runs, and how often the two backends agree on individual tasks. How do you verify isolation when concurrent rollouts of the same task make conflicting writes? Which VisualWebArena numbers are correct, Fig. 3 or Table 6? And what is the end-to-end RL training time under each backend? Suggestions
Report the engineering cost of supporting a new application, and which resources in each benchmark fell back to private backends. Show a use of fork, such as branching rollouts for tree-search RL, where cheap forks matter most.
3 comments
Thanks for the detailed response. The corrected table and the new isolation tests address Q2 and Q4. I'm raising my score to 3, assuming the changes below go into the paper.
Success gains. The timeout breakdown explains the Muse-Glimmer-30B gap on OSWorld, but that's 1 of 12 pairs. Please report timeout counts for the other 11 pairs. Please also describe the gain as fewer timeouts rather than a better agent. Training speedup. End-to-end RL is 1.30× faster (26 to 20 hours), far below the 6.20× rollout number in the abstract. Please put 1.30× in the abstract. The 1.07-point gain at 20 hours is within noise (about 4 of 361 tasks), so I'd call it equal success in less time. Baselines. Against WebServ, memory is about 9× lower, but launch time is the same and storage is only 2.2× lower. So the 504× storage number mostly comes from copy-on-write compared with plain Docker. Please move this comparison into the main text. In short, the cost savings hold up, but the paper should claim lower cost at equal success, not higher success.
Thank you for your thoughtful feedback and for raising your score. We appreciate your careful evaluation and constructive suggestions, and will revise the paper accordingly to ensure that our claims accurately reflect the results.
We thank the reviewer for the detailed assessment.
A1: Both backends use identical timeout settings in the main evaluation. For Muse-Glimmer-30B on OSWorld, the mutually exclusive outcomes are:
Extending only Docker's timeout increases its macro-averaged success from 24.40% to 33.70%, approaching CUA-Sandbox's 34.39%. This supports timeout sensitivity as a major contributor to the gap in this setting: lower overhead allows more tasks to finish within the budget. These results are now in Appendix B.4, Tables 11–12.
A2: We apologize for this reporting error. Figure 3 is correct; we have corrected the original Table 6 (now Table 9, Appendix B.2) to match it and checked the rollout discussion. Corrected Docker / CUA-Sandbox values are: preparation 10.404 / 0.504 s; batch time 44.762 / 7.215 s; throughput 0.179 / 1.109 env/s; memory 1745.2 / 190.0 MiB; incremental storage 6503.70 / 12.90 MiB. The gains remain 6.20× throughput, 9.2× memory reduction, and 504× storage reduction.
A3: For Qwen3.5-4B on OSWorld, we have added the following comparison to Section 4.2, Table 3, alongside Table 2:
Under the original matched data and rollout budget, CUA-Sandbox finishes in 20 versus 26 hours, reducing time by 23.1% (1.30× speedup). At the same 20-hour budget, observed success is 1.07 points higher. Shared training settings appear in Appendix C.2, Table 13.
A4: We have supplemented the read-only ablation with direct tests using two concurrent OSWorld environments, covering time zone, do-not-disturb mode, screen dimming, large text, and automatic screen locking, with five repetitions each.
We replay task setup and mutations deterministically, apply the original evaluators, and additionally modify identical paths under
/etcand/opt, including permissions. We alternate which environment is reset. Across 25 trials, all 669 checks pass without cross-environment contamination; host/shared-base probes remain unchanged. We also check baseline restoration and a subsequent application-configuration task for residual state.Five additional trials retain delayed writers that ignore SIGTERM and hold pre-reset file handles. No writes contaminate the successor generation; writers in the non-reset environment execute successfully as positive controls. Procedures and outcomes are now in Appendix A.3; shared/private boundaries are in Appendix D.2.
A5: WebServ targets web-agent workloads through improved browser observation/action interfaces and Incus-based provisioning with block-level CoW. It does not directly cover OSWorld-style desktop environments. We therefore use Docker as a common reference across benchmarks and WebServ as an optimized baseline for overlapping web workloads.
The original memory entries used different accounting methods. We now compare marginal memory growth per environment for both systems, reporting mean ± standard deviation over three runs:
CUA-Sandbox uses approximately one-ninth of WebServ's marginal per-environment memory. Existing measurements show comparable launch latency (1.73 vs. 1.78 s) and approximately 2.2× lower storage (12.50 vs. 28.01 MiB). Appendix A.2 now includes the scope and accounting explanations and common-metric results (Table 6); launch/storage results remain in Table 5.