acceptodds
Under review as a conference paper at ICLR 2027

VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards

Abstract

Reinforcement learning for computer-use agents increasingly relies on proactive reward agents that interact with the post-execution GUI environment to verify task completion beyond what recorded screenshots reveal. The reliability of such verification hinges on two coupled decisions, namely what to check and when to stop. Existing methods leave these decisions to prompted foundation models, yet neither model scale nor few-shot examples help, as prompted Judges ranging from 4B models to GPT-5.4 all reach between 68% and 70% macro-average agreement. These decisions are hard to prompt because what a check yields and when the evidence suffices depend on the environment rather than the task, and become apparent only once verification is executed. Learning them from the environment is nevertheless difficult, since verification starts from the Actor's terminal state, which cannot be faithfully replicated during training, so each state admits a single attempt. We therefore introduce VeriCUA, which makes each single attempt informative and compares attempts across terminal states. A trainable Judge decides what to check and when to stop, while a frozen Probe executes the checks in the live environment. The Judge is trained with online reinforcement learning on one verification trajectory per terminal state, with a reward reflecting both verdict correctness and Probe usage and returns normalized across terminal states. Trained only on OSWorld and evaluated without further training on MobileWorld and WebArena-Verified, VeriCUA attains the highest reward agreement on all three benchmarks and lies on the agreement–cost Pareto frontier among proactive verifiers, outperforming few-shot GPT-5.4 by 5.36 macro-average points with 47.6%–59.8% fewer Probe tokens. As the reward for policy training, it raises task success by 3.21 points overall and 5.40 points on held-out multi-application tasks.

Primary area:
reinforcement learning
Keywords:
computer-use agents, GUI agents, proactive verification, reward modeling, reinforcement learning with verifiable rewards

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PKDeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes VeriCUA, a framework that learns to verify GUI task completion through interaction with the environment. A trainable Judge decides what to check and whether to request more evidence, while a frozen Probe executes the checks. The Judge is initialized with supervised fine-tuning and then trained using REINFORCE++ with rewards based on verdict correctness and Probe use. Experiments on desktop, mobile, and web benchmarks show improved reward agreement and reduced Probe token consumption, with additional gains in downstream actor training.

Critical strengths
  1. The paper addresses an important limitation of GUI reward models: recorded screenshots may not contain enough evidence to determine task completion. Learning how to acquire the missing evidence is a useful direction.

  2. The evaluation covers three platforms, including mobile and web environments unseen during Judge training. Comparisons using the same frozen Probe help assess the benefit of training the Judge.

  3. The paper evaluates the verifier as a reward source for downstream actor training, providing evidence of practical value beyond agreement with benchmark labels.

Critical weaknesses
  1. The methodological novelty needs to be clearer. Active environment inspection and reasoner–actor collaboration have already been explored in prior work. The main change here is to train the Judge rather than rely on prompting, using the existing REINFORCE++ algorithm. I appreciate the practical value of learning a verifier, but the paper should explain more clearly what new verification-learning mechanism it introduces beyond applying an established training method to this setting. The motivation for learning what to check and when to stop is stronger than the case for algorithmic novelty.

  2. The contribution of online RL is not fully separated from the gains brought by supervised fine-tuning. In Figure 5(a), SFT raises OSWorld agreement from 73.73% to 79.41%, while online RL further improves it to 82.00%. Most of the overall gain therefore comes from SFT. The paper does provide this stage-wise comparison, but SFT-only results are missing from the cross-platform evaluation and downstream actor-training experiments. These comparisons would help establish how much of the reported transfer and policy improvement actually requires online RL, rather than being achievable through SFT alone.

  3. The reward design provides limited support for the claim of learning when to stop. It distinguishes whether the Probe was used at all, but does not distinguish one invocation from several or account for interaction steps and token consumption. Once probing has begun, further checks carry no explicit additional cost. This more directly teaches the Judge whether to start probing than when to stop within a multi-round verification process. The observed token savings are useful, but the paper should better explain how this reward leads to timely stopping, rather than mainly reducing the frequency of Probe use.

Questions for authors

Refer to the weaknesses.

5 comments

Dear Reviewer PKDe,

We sincerely appreciate the time and effort you have dedicated to reviewing our work. We are grateful for your insights and would like to address each of your comments in detail to help alleviate any concerns you may have.

W1. Novelty of the Verification-Learning Mechanism

We thank the reviewer for recognizing the importance of learning evidence acquisition for GUI verification, as well as the practical value demonstrated across three platforms and downstream Actor RL. We respectfully disagree that VeriCUA is merely an application of REINFORCE++ to an existing Judge–Probe framework. Prior proactive GUI verifiers relied on prompting because Actor terminal states cannot be reliably reproduced for standard multi-rollout RL. Our methodological contribution lies not in proposing a new general-purpose RL algorithm, but in formulating this unsolved problem so that proactive verification can be directly optimized with single-attempt online RL, achieving strong gains and zero-shot generalization across environments.

Prior works (e.g., ProRe and VAGEN) primarily rely on training-free verification and do not address this optimization challenge. VeriCUA makes online learning feasible through three complementary designs: (1) a frozen Probe that separates transferable verification learning from platform-specific GUI execution; (2) single-rollout REINFORCE++ with cross-environment return normalization, avoiding same-state rollout requirements; and (3) a verification-specific reward that jointly supervises verdict correctness and stop-or-probe decisions.

Our additional ablations further validate the importance of this formulation. Compared with correctness-only RL, VeriCUA improves reward agreement by 4.58, 9.50, and 5.25 pp on OSWorld, MobileWorld, and WebArena-Verified, respectively (see W2, Table 1). These results demonstrate that simply applying correctness-based RL does not reproduce our gains, highlighting the importance of verification-specific reward design.

We will revise the Introduction and Sections 2.3–2.4 to distinguish our verification-learning contribution more explicitly from the underlying RL optimizer.

W2 "The contribution of online RL is not fully separated from the gains brought by supervised fine-tuning."

We thank the reviewer for this valuable comment. We agree that the original stage-wise comparison alone does not fully establish the contribution of online RL to cross-domain generalization and downstream Actor learning. We therefore conduct additional experiments comparing SFT-only, correctness-only RL, and our full method under matched settings.

Table 1. Cross-domain Reward Agreement (%)

MethodOSWorldMobileWorldWebArena-Verified
Prompted VeriCUA73.7373.5061.50
SFT-only79.4168.7559.75
SFT + Binary RL77.4268.0057.50
SFT + Our RL82.0077.5062.75

Our online RL substantially improves cross-domain generalization beyond SFT. Compared with SFT-only, VeriCUA improves agreement by 2.59, 8.75, and 3.00 pp across the three benchmarks. Notably, Binary RL underperforms SFT-only on all three benchmarks, suggesting that correctness-only feedback is insufficient to guide effective verification learning. Our verification-specific reward instead achieves improvements of 4.58, 9.50, and 5.25 pp over Binary RL, respectively.

To further assess downstream utility, we introduce a complementary cross-environment Actor RL setting. Using the corresponding Judge checkpoints (SFT-only, SFT + Binary RL, and SFT + Our RL), we train downstream Actors on 768 randomly sampled tasks from CUAGym and SCALECUA, without using OSWorld tasks for Actor training. We then evaluate the trained Actors on the complete standard 361-task OSWorld evaluation set, reporting Avg@8 and Pass@8. This setting assesses whether the learned verification policies provide effective rewards for Actor training on different task sources.

Table 2. Downstream Actor RL Performance on OSWorld (%)

PolicyReward SourceOSWorld Avg@8 (%)OSWorld Pass@8 (%)
Initial Policy—31.40†N/A†
Policy + RewardGT reward33.1050.42
Policy + RewardSFT-only31.0747.92
Policy + RewardSFT + Binary RL32.8649.03
Policy + RewardSFT + Our RL34.6350.42

† The initial-policy score (31.40%) is taken from the Qwen3-VL technical report.

The improvement also translates into downstream Actor learning. Our reward improves Avg@8 by 3.56 pp over SFT-only and 1.77 pp over Binary RL, while achieving the highest Pass@8 (tied with GT rewards).

We will incorporate these additional ablations and clarify the cross-environment Actor training protocol in the revised manuscript.

W3 "limited support for the claim of learning when to stop"

We thank the reviewer for recognizing VeriCUA's efficiency improvements. While our reward does not explicitly penalize additional Probe rounds or internal execution costs, this reflects a deliberate design choice to prioritize reliable verification decisions over fine-grained execution-cost optimization. We clarify three points below.

Q1. The efficiency gains mainly arise from reducing Probe activation.

A: Our additional OSWorld analysis challenges this interpretation. On the fixed verifier evaluation set, Probe activation increases from 8.7% (prompted) to 11.1% (VeriCUA), while GUI steps per activated case decrease from 11.56 to 9.65. In contrast, Section 3.4 reports lower Probe activation during downstream Actor RL (5.70% to 4.28%), where the Actor trajectory distribution evolves during training. These results indicate that VeriCUA's efficiency improvements are not simply achieved by suppressing Probe usage. Appendix B (Table 3) further reports +14.3 pp in direct-evidence probing and −10.7 pp in aimless scrolling and waiting compared with prompted GPT-5.4, demonstrating more targeted evidence acquisition.

Q2. The reward does not explicitly encourage stopping within multi-round verification.

A: Our primary objective is to learn reliable verification decisions, rather than explicitly minimize Probe rounds. The terminal reward encourages correct verdicts after evidence acquisition (+0.7) and penalizes unresolved attempts (−1.0). While we do not distinguish one-round from two-round successful verification, only 5/414 OSWorld trajectories (1.2%) invoke a second Probe round under our current two-round budget. Thus, explicit per-round penalties would target a relatively uncommon behavior in our setting, rather than the primary source of verification cost.

Q3. The reward does not account for Probe-internal execution costs.

A: The Probe is frozen, and the Judge does not directly control its internal GUI actions or execution length. Therefore, penalizing these low-level costs would not provide effective supervision for the Judge's stop-or-continue decisions, nor optimize the frozen Probe's execution policy. Our formulation instead focuses on learning what evidence to acquire and when to issue a verdict.

We will clarify these design choices in Section 2.3 and incorporate the additional OSWorld Probe-activation, round-count, and conditional GUI-step statistics into the appendix.

AuthorsOpenReview ↗

Follow-Up on Submission 29024 – Request for Feedback

Dear Reviewer PKDe,

We sincerely appreciate your thoughtful feedback and have carefully addressed all points raised in your review with additional cross-domain SFT vs. RL ablations, downstream Actor RL experiments, and Probe stopping analyses.

We would be grateful if you could let us know if there are any remaining concerns or if further clarification is needed. If our responses have sufficiently addressed your concerns, we would deeply appreciate it if you could consider updating your rating. We are eager to engage in further discussion and clarify any unresolved points to ensure all matters are fully addressed.

Thank you again for your time and valuable input.

Best regards, Authors of Submission 29024

Reviewer_GpZdOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VERICUA, a proactive reward model for GUI agents that learns verification strategies through online reinforcement learning. The framework consists of a trainable Judge that formulates verification criteria and decides when sufficient evidence has been collected, and a frozen Probe that interacts with the GUI environment to gather additional evidence. To address the difficulty of reproducing Actor terminal states, the authors adopt REINFORCE++ with one verification rollout per terminal state, using executable outcomes and probing usage to construct the training reward. The Judge is trained on OSWorld and evaluated on OSWorld, MobileWorld, and WebArena-Verified. Experimental results show improved reward agreement over prompted verifiers, including larger proprietary models, while reducing probing costs. The authors also demonstrate that using VERICUA as a reward model improves downstream Actor performance on OSWorld.

Critical strengths

S1. Important problem with promising empirical results. Reliable reward modeling is an important challenge for GUI agent RL. VERICUA achieves higher reward agreement than strong prompted verifiers, including GPT-5.4 and Kimi-K2.6, while substantially reducing probing costs. These results demonstrate the potential of learning verification strategies rather than relying solely on model scaling.

S2. Encouraging evidence of cross-domain generalization. Despite being trained only on OSWorld, VERICUA achieves strong reward agreement on MobileWorld and WebArena-Verified without additional training. The downstream Actor RL experiments on OSWorld also show improvements over existing model-based reward methods, providing initial evidence of its practical utility.

Critical weaknesses

W1. Insufficient validation of cross-domain reward utility. The paper emphasizes the generalizability of learned rewards, but evaluates cross-domain transfer only through reward agreement on MobileWorld and WebArena-Verified, while all downstream Actor RL experiments remain on OSWorld. High reward agreement does not necessarily translate into effective supervision for online policy optimization. The absence of cross-domain downstream RL experiments leaves a key aspect of the paper's generalization claim insufficiently validated.

W2. Limited methodological novelty and justification. The proposed training framework largely builds on existing components, including a Judge–Probe architecture and REINFORCE++ optimization. While the empirical results are encouraging, the methodological contribution appears relatively incremental. In particular, the choice of single-rollout optimization is motivated by the difficulty of reproducing terminal states, but alternatives such as VM snapshotting or state restoration are not sufficiently discussed.

Questions for authors

Q1. Please address the concerns raised in the weaknesses.

Q2. Have the authors evaluated the OSWorld-trained Judge as a reward model for downstream Actor RL on MobileWorld? If so, how does its performance compare with training using executable ground-truth rewards?

Q3. Have the authors considered using VM snapshots or environment state restoration to enable multiple verification rollouts from the same Actor terminal state? If so, what are the main technical limitations or computational costs?

3 comments

Dear Reviewer GpZd,

We sincerely appreciate your detailed review and recognition of our problem, empirical gains over frontier verifiers, and cross-domain transfer. Below, we address each comment in detail.

W1 & Q2: Cross-Domain Downstream Actor RL on MobileWorld

Following your suggestion, we evaluated the OSWorld-trained VeriCUA-4B Judge as the online reward model for downstream Actor RL on MobileWorld against executable ground-truth (GT) rewards:

PolicyReward SourceMobileWorld Avg@8 (%)MobileWorld Pass@8 (%)
Initial PolicyNone5.6613.7
Policy + RewardVeriCUA (OSWorld-trained, 0-shot)6.7314.5
Policy + RewardExecutable GT Reward8.8715.4

Transferred zero-shot from desktop to Android without any MobileWorld training data for the Judge or rule engineering, VeriCUA lifts MobileWorld Avg@8 from 5.66% to 6.73% (+18.9% relative gain) and achieves 14.5% Pass@8, compared with 15.4% under executable GT rewards. Moreover, in our cross-distribution DAPO Actor RL study on 768 external CUAGym and SCALECUA tasks with zero OSWorld overlap (full table in our reply to Reviewer EFfc, W2 & Q3(a)), the VeriCUA Judge matches GT rewards in Pass@8 at 50.42% and surpasses GT at 33.10%, SFT-only at 31.07%, and Binary RL at 32.86% in Avg@8 by reaching 34.63%. These results confirm that VeriCUA's cross-domain reward agreement translates directly into effective online policy optimization.

W2: Methodological Justification

Our methodological choices are motivated by two fundamental challenges in learning transferable and efficient proactive verification policies. Prior Judge–Probe verifiers (e.g., ProRe) rely on training-free prompting, and directly combining them with REINFORCE++ fails on two fronts: (1) end-to-end training ties verification logic to OS-specific GUI actions, so we freeze the Probe and mask its tokens in Eq. 4 to learn transferable what-to-check and when-to-stop decisions; and (2) with one attempt per terminal state in Sec. 2.3, standard 0/1 correctness reward cannot calibrate when to probe or stop.

Testing this from the same SFT checkpoint across all benchmarks:

4B Judge (same SFT ckpt)OSWorld (ID)MobileWorld (OOD)WebArena-V (OOD)Macro Avg.
SFT-only (before RL)79.4168.7559.7569.30
Judge–Probe + REINFORCE++ (0/1 reward)77.4268.0057.5067.64
VeriCUA (ours)82.0077.5062.7574.08

Naively combining Judge–Probe with REINFORCE++ (0/1 reward) drops macro agreement below SFT initialization (67.64% vs. 69.30%). In contrast, our verification-specific reward lifts macro agreement to 74.08% (+4.78 pp over SFT and +6.44 pp over 0/1 RL) while using fewer Probe tokens on OSWorld and MobileWorld.

Q3 & W2: Why Not VM Snapshotting or State Restoration?

Thanks for raising a fair point. The key challenge is restoring an exact Actor terminal state, rather than resetting a task to its initial state. Terminal states hold unsaved edits, in-memory app states, browser sessions, and external connections. Yes, we carefully considered both options when designing our pipeline; to show why neither replaces single-rollout training, we outline their limits verified on live OSWorld terminal states:

  • VM Snapshotting: Multi-rollout RL needs a separate checkpoint per Actor terminal state across all 64 trajectories per batch and KK independent copies so Probe actions do not alter the GUI for later rollouts. In containerized setups like OSWorld, pause/resume only continues one instance rather than cloning KK copies, while disk snapshots reboot the OS and wipe open windows, tabs, and unsaved edits. Full VM cloning across 64 states per batch adds heavy storage/I/O overhead [1] and multiplies Probe costs by K×K\times. Snapshots also miss external states: in OSWorld, a 60s pause caused a 10–16s clock jump, dropped active HTTPS connections from 10 to 3, and broke expired web sessions upon resume.
  • Environment State Restoration: Restoring an arbitrary terminal state requires rolling back running processes, app memory, browser sessions, temporary files, and external services, which lack unified reversible interfaces across GUI platforms. In OSWorld, restoring files without process memory loses open windows and unsaved app states, while tools like CRIU [2] fail when external files or network connections change and cannot undo irreversible side effects such as submitted forms.

Revision plan: We will add the MobileWorld Actor RL results to the Appendix and clarify these design choices and the limitations of VM snapshotting and state restoration in Sec. 2.4.

We hope this addresses your concerns and welcome further discussion.

[1] VMware, vSphere Snapshots: Performance and Best Practices.

[2] CRIU Project, External Resources.

Dear Reviewer GpZd,

As the discussion period enters its final 48 hours, we would like to respectfully follow up on our response and uploaded revised manuscript (highlighted in blue). Following your suggestions, we have evaluated the OSWorld-trained VeriCUA Judge on cross-domain MobileWorld Actor RL and expanded our methodological justification alongside the limitations of VM snapshotting and environment state restoration (Secs. 2.3–2.4, 3.3–3.4; Apps. H–K).

We would truly appreciate hearing whether these new experiments and explanations have resolved your main concerns, or if any further clarification would be helpful. Should you find our updates satisfactory, we would be very grateful if you could consider revisiting your evaluation.

Thank you again for your time and constructive feedback.

Best regards, Authors of Submission 29024

Thank you for the detailed responses and additional experiments. However, I remain unconvinced by the evidence for the generalizability of the RL results and the effectiveness of the proposed method. I will therefore maintain my original rating.

Reviewer_EFfcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VeriCUA learns a proactive verifier for GUI-agent task completion. A trainable Judge formulates success checks and decides whether further inspection is needed, while a frozen Probe gathers evidence from the actor’s terminal environment. The Judge is initialized through supervised learning on successful teacher verification traces and then trained with online reinforcement learning using verdict correctness and probing-dependent rewards. Training uses one verification attempt per terminal state. The paper evaluates reward agreement and probing cost on desktop, mobile, and web environments, and subsequently uses the trained verifier as a reward source for actor training.

Critical strengths
  1. The comparison against the same prompted architecture is useful. Holding the Judge backbone, Probe, prompts, and interaction budget fixed makes the effect of training reasonably clear. For the 4B model, agreement improves by 8.27 points on OSWorld, 4.00 on MobileWorld, and 1.25 on WebArena. Freezing the Judge after desktop training also provides meaningful evidence of transfer.

  2. The evaluation connects verification to interaction and downstream learning. The paper examines probing budgets, different Probe sizes, evidence-acquisition behavior, and actor training. The increase from 74.09% agreement without probing to 82.00% with two available rounds supports the value of inspecting otherwise unavailable evidence.

Critical weaknesses
  1. On OSWorld, agreement rises from 73.73% to 79.41% after supervised fine-tuning, then to 82.00% after RL. This is a useful RL improvement, but most of the total gain comes from supervised initialization. SFT-only results are missing from the transfer and downstream actor comparisons, and there is no correctness-only reward ablation to establish the contribution of the probing-dependent reward. The claim that few-shot examples cannot teach the strategy is also stronger than the baseline supports: Appendix H.3 supplies criteria-format examples without verdicts, returned evidence, or probing trajectories. Those examples do not demonstrate the check-and-stop behavior being learned.

  2. Against prompted VeriCUA rewards, the downstream improvement is 47.14% versus 45.71% overall and 22.97% versus 20.27% on MultiApp. These correspond to four additional solved tasks overall and two on MultiApp. Four evaluation rollouts for Pass@4 do not establish variation across independent actor-training runs. In addition, aggregate verifier agreement does not reveal the balance between false positive and false negative rewards. This is important for assessing whether the verifier provides a reliably better learning signal.

Questions for authors
  1. How do SFT-only and correctness-only RL variants compare with full VeriCUA on cross-environment agreement, probing cost, and downstream actor learning?

  2. Does the conclusion about few-shot prompting hold when examples include complete checks, returned evidence, and stopping decisions?

  3. Can you provide paired uncertainty or repeated actor-training results for Table 2, together with success/failure confusion matrices for the held-out verifier evaluation?

2 comments

Dear Reviewer EFfc,

We sincerely appreciate your detailed review and the recognition of the important research problem we address, along with our proposed method. Below, we address each of your comments in detail.

W1 & Q1 How Do SFT-Only and Correctness-Only RL Compare on Transfer, Probing Cost, and Actor Learning?

We compare Prompted, SFT-only, Binary RL with a 0/1 correctness reward, and VeriCUA across all three benchmarks, reporting Avg@4 agreement and Probe tokens per judgment in thousands, as in Fig. 4:

4B JudgeOSWorld (ID)MobileWorld (OOD)WebArena-V (OOD)Macro Avg.
Prompted VeriCUA73.73 (3.10k)73.50 (5.80k)61.50 (11.60k)69.58 (6.83k)
SFT-only79.41 (3.97k)68.75 (25.17k)59.75 (8.59k)69.30 (12.58k)
SFT + Binary RL77.42 (6.69k)68.00 (25.87k)57.50 (6.70k)67.64 (13.09k)
VeriCUA (ours)82.00 (4.40k)77.50 (12.50k)62.75 (9.70k)74.08 (8.87k)
  • Transfer & cost: SFT improves in-domain OSWorld but drops below prompting on unseen MobileWorld and WebArena-V, whereas VeriCUA improves macro-average agreement by 4.78 points over SFT-only and 6.44 points over Binary RL. Without our probing-dependent reward and unresolved penalty, Binary RL underperforms SFT-only on all three benchmarks and trails VeriCUA by 4.58 points on OSWorld (77.42% vs. 82.00%), while requiring 1.52× and 2.07× the Probe tokens of VeriCUA on OSWorld and MobileWorld, respectively.
  • Downstream Actor RL: On the 361-task OSWorld evaluation in W2 & Q3(a), VeriCUA outperforms both SFT-only and Binary RL in Avg@8 and Pass@8.

W1 & Q2 Complete-Trajectory Few-Shot Prompting

Thank you for this meaningful suggestion. Beyond criteria-only few-shot, we prompted the 4B Judge with 3 complete trajectories containing Actor history, checks, Probe actions, evidence, status updates, and stopping verdicts under the same frozen Probes:

4B JudgeOSWorldMobileWorldWebArena-VMacro Avg.
Prompted VeriCUA, 0-shot73.7373.5061.5069.58
+ Criteria-Only Few-Shot75.0666.5061.7567.77
+ Complete-Trajectory Few-Shot73.9772.0056.2567.41
Trained VeriCUA-4B (ours)82.0077.5062.7574.08

Upgrading from criteria-only to full-trajectory examples yields no macro gain over zero-shot prompting, leaving VeriCUA-4B ahead by 6.67 macro points. Few-shot examples show the multi-turn format, but cannot teach which hidden state to inspect on a new task or when returned evidence is enough to stop, which requires online RL.

W2 & Q3 Downstream Uncertainty and Confusion Matrices

(a) Paired Actor uncertainty: To evaluate downstream rewards with paired uncertainty, we trained 4B Actors with DAPO on 768 CUAGym/SCALECUA tasks with no OSWorld overlap and ran 8 rollouts per task across the full 361-task OSWorld suite, totaling 2,888 rollouts per model and 20,000 task-paired bootstrap resamples:

Reward SourceAvg@8 (%)Pass@8 (%)VeriCUA Gain in Avg@8 [95% CI]
GT Reward33.1050.42 (182/361)+1.53 [+0.03, +3.06]
SFT-only Judge31.0747.92 (173/361)+3.55 [+1.94, +5.25]
SFT + Binary RL32.8649.03 (177/361)+1.77 [+0.17, +3.42]
SFT + VeriCUA (ours)34.6350.42 (182/361)Reference

VeriCUA improves Avg@8 by 3.55 points over SFT-only, 1.77 points over Binary RL, and 1.53 points over GT reward, with all paired 95% CIs above zero, while matching the GT-reward Actor in Pass@8.

(b) Verifier confusion matrices: We compare Prompted and trained VeriCUA 4B Judges across all three benchmarks under the main-text protocol:

Env. & VerifierTPTNFP†FN†FPR† ↓\downarrowFNR† ↓\downarrowAgree. (%)
OSWorld / Prompted65956227116432.53%19.93%73.73
OSWorld / VeriCUA (ours)7356232079124.94%11.02%82.00
MobileWorld / Prompted96198584822.66%33.33%73.50
MobileWorld / VeriCUA (ours)110200563421.88%23.61%77.50
WebArena-V / Prompted931531035140.23%35.42%61.50
WebArena-V / VeriCUA (ours)991521044540.62%31.25%62.75

Covers all trials on OSWorld N=1656N=1656, MobileWorld N=400N=400, WebArena-V N=400N=400. †Unresolved/failed outputs count as disagreements: FN for positive references, FP for negative.

VeriCUA reduces FNR across all three benchmarks while lowering FPR on OSWorld and MobileWorld. In group-relative Actor RL, missing a true success discards scarce positive rollouts on hard tasks, so lower FNR preserves stronger learning signals.

Revision plan: In the revised PDF, we will narrow the few-shot claim in Sec. 1, include the 95% paired bootstrap CIs in Sec. 3.4, and add the Probe token breakdown, full-trajectory few-shot prompts/results, and confusion matrices to the Appendix.

We hope this addresses your concerns and welcome further discussion.

AuthorsOpenReview ↗

Follow-Up on Submission 29024 – Request for Feedback

Dear Reviewer EFfc,

We sincerely appreciate your constructive review and positive assessment of our work. We have carefully addressed all points raised in your review with new experiments—including the cross-domain SFT-only and Binary RL ablations, complete-trajectory few-shot prompting baselines, task-paired bootstrap 95% CIs on downstream Actor RL, and full verifier confusion matrices.

As the discussion period progresses, we would be grateful if you could let us know whether our responses and additional experimental results have resolved your concerns, or if any further clarification is needed. If all your questions have been satisfactorily addressed, we would deeply appreciate it if you could consider updating your score accordingly.

Thank you again for your time and valuable input.

Best regards, Authors of Submission 29024

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank the Area Chairs and all three reviewers (PKDe, EFfc, GpZd) for their valuable time and constructive feedback. We are encouraged that reviewers recognized VeriCUA's clear motivation for proactive GUI verification, higher reward agreement at lower probing cost than prompted frontier models, zero-shot cross-domain transfer, and practical value for downstream policy learning.

In response to the reviewers' suggestions, we have incorporated extensive new experiments and discussions in the revised version (all updates highlighted in blue). Below is a summary of the key changes made:

  1. External Downstream Actor RL & Paired Uncertainty (PKDe, EFfc, GpZd): Training 4B Actors with DAPO on 768 external CUAGym and SCALECUA tasks without OSWorld overlap and evaluating across all 361 OSWorld tasks, frozen VeriCUA matches executable GT task coverage at 50.42% Pass@8 and surpasses GT in Avg@8 (34.63% vs. 33.10%), as well as SFT-only (31.07%) and Binary RL (32.86%) with all paired 95% CIs above zero, while solving 8 and 12 more tasks than ZeroGUI and prompted VeriCUA (Section 3.1 & 3.4, Table 2; Appendix F & H, Tables 8 & 11).
  2. Zero-Shot Cross-Domain Actor RL on MobileWorld (GpZd): Applying the frozen OSWorld-trained VeriCUA-4B Judge zero-shot to MobileWorld Actor RL with a frozen GUI-Owl-1.5-32B-Think Probe and no mobile Judge data or rules raises Avg@8 from 5.66% to 6.73% (+18.9% relative) and Pass@8 from 13.7% to 14.5% against 15.4% under executable GT rewards (Section 3.4; Appendix I, Table 12).
  3. Training-Stage, Reward-Design & Probing-Cost Ablations (PKDe, EFfc, GpZd): Across OSWorld, MobileWorld, and WebArena-Verified on the shared 4B backbone, naive 0/1 Binary RL drops macro agreement below SFT initialization while consuming 1.52×–2.07× the Probe tokens on OSWorld and MobileWorld, whereas VeriCUA reaches 74.08% macro agreement, gaining +4.78 points over SFT-only and +6.44 over Binary RL (Section 1; Section 3.3, Figure 5; Appendix K, Table 13).
  4. Complete-Trajectory Few-Shot & Verifier Confusion Matrices (EFfc): Prompting the 4B Judge with three complete multi-turn verification trajectories yields no macro gain over zero-shot prompting and trails VeriCUA-4B by 6.67 points (Section 3.1; Appendix L, Table 14; Appendix N.4), while held-out confusion matrices show VeriCUA lowers false-negative reward rates across all three benchmarks, including 19.93% to 11.02% on OSWorld and 33.33% to 23.61% on MobileWorld (Section 3.4; Appendix M, Table 15).
  5. Methodological Clarifications & State-Restoration Analysis (PKDe, GpZd): We clarify our single-attempt verification formulation beyond REINFORCE++, binary Probe-invocation feedback, and single-round stopping (Sections 1 & 2.3–2.4), and verify on live OSWorld terminal states why VM snapshotting and state restoration cannot fork independent rollouts without GUI mutation, 10–16s pause clock jumps, and expired external sessions (Section 2.4; Appendix J).

Please see our individual responses and the revised PDF for full details. Thanks again to all reviewers and Area Chairs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.