- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies GUI grounding that adapts after deployment from screenshots and language instructions without target-domain correctness labels. Its motivation is that scalar feedback provides little guidance about failed predictions and can leave relative-reward learning uninformative. A Reflector is first trained offline with labeled grounding attempts and then frozen. During adaptation, the grounding policy predicts a location, and the Reflector supplies an estimated success judgment and textual explanation. The grounding model conditions a self-teacher on this feedback to obtain token-level supervision along its own generated coordinate sequence. For failed attempts, contrastive calibration compares the reflection-conditioned distribution with an inverse prompt that treats the failed prediction as successful. Directional clamping and group-relative advantages control the resulting updates. The updated grounding model subsequently predicts target locations from screenshots and instructions, while the Reflector supplies feedback during adaptation.
Critical strengths
-
The introduction clearly motivates learning from failed grounding attempts, and contrastive calibration addresses a concrete failure mode of coordinate self-distillation.
-
The evaluation covers established GUI benchmarks and includes scalar-feedback, prompted-teacher, and Reflector-reranking controls, rather than comparing only with an unadapted model.
Critical weaknesses
-
The practical case for deployment-time parameter updates is not fully established. The paper identifies a genuine limitation of scalar feedback, but the broader framework largely combines existing reflection, privileged-context self-distillation, and policy optimization; the specific methodological advance is concentrated in the calibration design. Although reranking and scalar-feedback controls are included, the complete method is not compared under a matched total feedback/compute budget. Its full configuration takes 778 minutes versus 449 for GUI-RCPO, and the cheaper variant omits part of the main configuration. The value of the additional adaptation machinery therefore remains uncertain.
-
The benchmark coverage is relevant, but the claimed deployment setting is insufficiently specified. Adaptation repeatedly accesses unlabeled target inputs before evaluation, while prediction/update ordering, resets, and checkpoint selection remain unclear. This makes batch transductive adaptation difficult to distinguish from improvement on newly arriving interfaces. Offline labeled Reflector training also limits the scope of the annotation-free claim. The inconsistent Reflector entries in Tables 5 and 8 and missing uncertainty for the main gains further weaken the interpretation of feedback reliability.
Questions for authors
-
Please specify target-input exposure, prediction/update ordering, resets, and independent checkpoint selection. Which results represent transductive adaptation, and which test previously unseen inputs?
-
Under matched total feedback or wall-clock budgets, how does the full method compare with Reflector reranking and scalar-feedback learning? Please include offline Reflector preparation.
-
Please reconcile Tables 5 and 8, report uncertainty, and explain the choice of Qwen-family backbones. Reproducing available open-source baseline entries would also improve the comparison.
I have read your response to my review and to the other viewers. My comment below is based on this situation.
My concern about novelty remains. The reflection ablation varies the feedback and keeps the learning rule fixed, so it leaves open what R-OPSD adds to existing privileged-context self-distillation. The methodological case still rests mainly on contrastive calibration.
I share Reviewer 52b7’s concern about what calibration learns from failures. The reply reports probability changes under different prompts without making clear what those probabilities measure or how many rollouts they cover. Even if the prompt suppresses a missed coordinate, the probability can move to another wrong location. The reply leaves open whether the weight update improves target hit rates when every sampled prediction misses.
The baseline results also support narrower conclusions than the paper draws. GUI-SD’s collapse was measured after replacing annotated targets with average predicted boxes, so it is hard to separate the effect of the distillation method from the quality of its input. For GUI-RCPO, the claim that all-miss groups provide no learning signal is unsupported. Its region-consistency rewards can still differ within those groups.
The practical case for deployment-time weight updates remains incomplete. The main protocol shows gains after batch adaptation, with limited evidence for continued improvement on newly arriving interfaces. The lightweight variant and the inference rollout counts still leave my original budget concern open. The full method has yet to be compared with reranking or scalar-feedback learning at a matched total cost that includes Reflector preparation, adaptation, and inference.