- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes VeriCUA, a framework that learns to verify GUI task completion through interaction with the environment. A trainable Judge decides what to check and whether to request more evidence, while a frozen Probe executes the checks. The Judge is initialized with supervised fine-tuning and then trained using REINFORCE++ with rewards based on verdict correctness and Probe use. Experiments on desktop, mobile, and web benchmarks show improved reward agreement and reduced Probe token consumption, with additional gains in downstream actor training.
Critical strengths
-
The paper addresses an important limitation of GUI reward models: recorded screenshots may not contain enough evidence to determine task completion. Learning how to acquire the missing evidence is a useful direction.
-
The evaluation covers three platforms, including mobile and web environments unseen during Judge training. Comparisons using the same frozen Probe help assess the benefit of training the Judge.
-
The paper evaluates the verifier as a reward source for downstream actor training, providing evidence of practical value beyond agreement with benchmark labels.
Critical weaknesses
-
The methodological novelty needs to be clearer. Active environment inspection and reasoner–actor collaboration have already been explored in prior work. The main change here is to train the Judge rather than rely on prompting, using the existing REINFORCE++ algorithm. I appreciate the practical value of learning a verifier, but the paper should explain more clearly what new verification-learning mechanism it introduces beyond applying an established training method to this setting. The motivation for learning what to check and when to stop is stronger than the case for algorithmic novelty.
-
The contribution of online RL is not fully separated from the gains brought by supervised fine-tuning. In Figure 5(a), SFT raises OSWorld agreement from 73.73% to 79.41%, while online RL further improves it to 82.00%. Most of the overall gain therefore comes from SFT. The paper does provide this stage-wise comparison, but SFT-only results are missing from the cross-platform evaluation and downstream actor-training experiments. These comparisons would help establish how much of the reported transfer and policy improvement actually requires online RL, rather than being achievable through SFT alone.
-
The reward design provides limited support for the claim of learning when to stop. It distinguishes whether the Probe was used at all, but does not distinguish one invocation from several or account for interaction steps and token consumption. Once probing has begun, further checks carry no explicit additional cost. This more directly teaches the Judge whether to start probing than when to stop within a multi-round verification process. The observed token savings are useful, but the paper should better explain how this reward leads to timely stopping, rather than mainly reducing the frequency of Probe use.
Questions for authors
Refer to the weaknesses.
5 comments
W3 "limited support for the claim of learning when to stop"
We thank the reviewer for recognizing VeriCUA's efficiency improvements. While our reward does not explicitly penalize additional Probe rounds or internal execution costs, this reflects a deliberate design choice to prioritize reliable verification decisions over fine-grained execution-cost optimization. We clarify three points below.
Q1. The efficiency gains mainly arise from reducing Probe activation.
A: Our additional OSWorld analysis challenges this interpretation. On the fixed verifier evaluation set, Probe activation increases from 8.7% (prompted) to 11.1% (VeriCUA), while GUI steps per activated case decrease from 11.56 to 9.65. In contrast, Section 3.4 reports lower Probe activation during downstream Actor RL (5.70% to 4.28%), where the Actor trajectory distribution evolves during training. These results indicate that VeriCUA's efficiency improvements are not simply achieved by suppressing Probe usage. Appendix B (Table 3) further reports +14.3 pp in direct-evidence probing and −10.7 pp in aimless scrolling and waiting compared with prompted GPT-5.4, demonstrating more targeted evidence acquisition.
Q2. The reward does not explicitly encourage stopping within multi-round verification.
A: Our primary objective is to learn reliable verification decisions, rather than explicitly minimize Probe rounds. The terminal reward encourages correct verdicts after evidence acquisition (+0.7) and penalizes unresolved attempts (−1.0). While we do not distinguish one-round from two-round successful verification, only 5/414 OSWorld trajectories (1.2%) invoke a second Probe round under our current two-round budget. Thus, explicit per-round penalties would target a relatively uncommon behavior in our setting, rather than the primary source of verification cost.
Q3. The reward does not account for Probe-internal execution costs.
A: The Probe is frozen, and the Judge does not directly control its internal GUI actions or execution length. Therefore, penalizing these low-level costs would not provide effective supervision for the Judge's stop-or-continue decisions, nor optimize the frozen Probe's execution policy. Our formulation instead focuses on learning what evidence to acquire and when to issue a verdict.
We will clarify these design choices in Section 2.3 and incorporate the additional OSWorld Probe-activation, round-count, and conditional GUI-step statistics into the appendix.
Follow-Up on Submission 29024 – Request for Feedback
Dear Reviewer PKDe,
We sincerely appreciate your thoughtful feedback and have carefully addressed all points raised in your review with additional cross-domain SFT vs. RL ablations, downstream Actor RL experiments, and Probe stopping analyses.
We would be grateful if you could let us know if there are any remaining concerns or if further clarification is needed. If our responses have sufficiently addressed your concerns, we would deeply appreciate it if you could consider updating your rating. We are eager to engage in further discussion and clarify any unresolved points to ensure all matters are fully addressed.
Thank you again for your time and valuable input.
Best regards, Authors of Submission 29024
Dear Reviewer PKDe,
We sincerely appreciate the time and effort you have dedicated to reviewing our work. We are grateful for your insights and would like to address each of your comments in detail to help alleviate any concerns you may have.
We thank the reviewer for recognizing the importance of learning evidence acquisition for GUI verification, as well as the practical value demonstrated across three platforms and downstream Actor RL. We respectfully disagree that VeriCUA is merely an application of REINFORCE++ to an existing Judge–Probe framework. Prior proactive GUI verifiers relied on prompting because Actor terminal states cannot be reliably reproduced for standard multi-rollout RL. Our methodological contribution lies not in proposing a new general-purpose RL algorithm, but in formulating this unsolved problem so that proactive verification can be directly optimized with single-attempt online RL, achieving strong gains and zero-shot generalization across environments.
Prior works (e.g., ProRe and VAGEN) primarily rely on training-free verification and do not address this optimization challenge. VeriCUA makes online learning feasible through three complementary designs: (1) a frozen Probe that separates transferable verification learning from platform-specific GUI execution; (2) single-rollout REINFORCE++ with cross-environment return normalization, avoiding same-state rollout requirements; and (3) a verification-specific reward that jointly supervises verdict correctness and stop-or-probe decisions.
Our additional ablations further validate the importance of this formulation. Compared with correctness-only RL, VeriCUA improves reward agreement by 4.58, 9.50, and 5.25 pp on OSWorld, MobileWorld, and WebArena-Verified, respectively (see W2, Table 1). These results demonstrate that simply applying correctness-based RL does not reproduce our gains, highlighting the importance of verification-specific reward design.
We will revise the Introduction and Sections 2.3–2.4 to distinguish our verification-learning contribution more explicitly from the underlying RL optimizer.
We thank the reviewer for this valuable comment. We agree that the original stage-wise comparison alone does not fully establish the contribution of online RL to cross-domain generalization and downstream Actor learning. We therefore conduct additional experiments comparing SFT-only, correctness-only RL, and our full method under matched settings.
Table 1. Cross-domain Reward Agreement (%)
Our online RL substantially improves cross-domain generalization beyond SFT. Compared with SFT-only, VeriCUA improves agreement by 2.59, 8.75, and 3.00 pp across the three benchmarks. Notably, Binary RL underperforms SFT-only on all three benchmarks, suggesting that correctness-only feedback is insufficient to guide effective verification learning. Our verification-specific reward instead achieves improvements of 4.58, 9.50, and 5.25 pp over Binary RL, respectively.
To further assess downstream utility, we introduce a complementary cross-environment Actor RL setting. Using the corresponding Judge checkpoints (SFT-only, SFT + Binary RL, and SFT + Our RL), we train downstream Actors on 768 randomly sampled tasks from CUAGym and SCALECUA, without using OSWorld tasks for Actor training. We then evaluate the trained Actors on the complete standard 361-task OSWorld evaluation set, reporting Avg@8 and Pass@8. This setting assesses whether the learned verification policies provide effective rewards for Actor training on different task sources.
Table 2. Downstream Actor RL Performance on OSWorld (%)
† The initial-policy score (31.40%) is taken from the Qwen3-VL technical report.
The improvement also translates into downstream Actor learning. Our reward improves Avg@8 by 3.56 pp over SFT-only and 1.77 pp over Binary RL, while achieving the highest Pass@8 (tied with GT rewards).
We will incorporate these additional ablations and clarify the cross-environment Actor training protocol in the revised manuscript.