- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
VISTA studies GUI grounding from a screenshot and a language instruction, with a click coordinate as output. It addresses the loss of useful relative advantages when fixed-view reinforcement-learning rollouts are uniformly successful or unsuccessful. During training, target-preserving crops create different views of the same annotated instance, and the target box is remapped into each view. Model-generated coordinate responses across these views form a shared comparison group with point-in-box rewards. The method adds a ground-truth center-coordinate anchor whose weight is positive only when the group contains both successful and unsuccessful model responses. The anchor is excluded from model-only group statistics and is optimized through an advantage-weighted supervised term alongside the policy objective. At inference, the trained model predicts a click directly from the screenshot and instruction; an optional inference-time view-aggregation procedure can be used separately.
Critical strengths
-
The introduction identifies a concrete optimization bottleneck, and the method separately specifies cross-view group construction and selective anchoring without changing the model-only baseline.
-
Experiments cover established GUI benchmarks, multiple model scales, and meaningful crop-augmentation and component controls. These comparisons support a useful training contribution.
Critical weaknesses
-
The technical increment over augmentation and existing oracle-guided policy optimization is narrower than the broad framework presentation suggests. LUFFY and BREAD already combine policy learning with external guidance, although their mechanisms differ from VISTA. The grounding-specific contribution is the cross-view grouping and gated anchor combination. Grouping alone adds only 0.3 points over the crop-augmentation control, while the full method incurs additional training time. Missing repeated-run uncertainty and matched-compute comparisons make the incremental value difficult to quantify. Failure of an ungated anchor with an extreme normalization weight is also not a sufficient test of the gating principle against a numerically stable alternative.
-
The construction guarantees target visibility and coordinate consistency, but not preservation of instruction meaning: a crop can remove a relational anchor or alter a relative-order description. This assumption is central to treating the views as equivalent learning instances and is not directly audited. The robustness diagnostic uses training examples and the training crop distribution, as acknowledged in the paper, so it cannot establish general robustness to unseen view changes.
Questions for authors
-
Please compare the full method and augmentation control at matched compute, with repeated-run uncertainty and a stably weighted ungated anchor.
-
How often do crops invalidate referring instructions? Please provide a focused validity audit and a held-out view diagnostic.
-
How were the data mixture and checkpoint selected, and why were the chosen backbone families used? Please fill available open-source baseline results where reproducible.