acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation

Abstract

GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ability to adapt to unseen interfaces. Although recent methods attempt to adapt models via test-time reinforcement learning, they cannot reflect upon failed exploration. To overcome this, we propose a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth. It constructs a closed-loop of Exploration, Evaluation, Reflection, and Internalization. Specifically, the agent first explores unseen interfaces by predicting grounding coordinates for given instructions. To evaluate these explorations, we introduce an MLLM-based Reflector to assess the generated results and provide the corresponding reasoning reflections. To internalize reflection knowledge into the model weights, we propose Reflection-Guided On-Policy Self-Distillation, which translates high-level reasoning into dense token-level supervision via a conditioned self-teacher. Furthermore, we design a Contrastive Calibration method to prevent incorrect auto-regressive prefixes from corrupting the supervisory signals during failed explorations. Extensive experiments across six benchmarks demonstrate our framework's effectiveness, achieving an average accuracy improvement of 7.4% over the base model. To the best of our knowledge, this is the first work to successfully exploit on-policy self-distillation for test-time adaptation in GUI visual grounding. By filling the gap in post-deployment adaptation, our framework completes the self-evolving capability of GUI agents. The code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
GUI Visual Grounding, On-policy Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_Gc9XOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies GUI grounding that adapts after deployment from screenshots and language instructions without target-domain correctness labels. Its motivation is that scalar feedback provides little guidance about failed predictions and can leave relative-reward learning uninformative. A Reflector is first trained offline with labeled grounding attempts and then frozen. During adaptation, the grounding policy predicts a location, and the Reflector supplies an estimated success judgment and textual explanation. The grounding model conditions a self-teacher on this feedback to obtain token-level supervision along its own generated coordinate sequence. For failed attempts, contrastive calibration compares the reflection-conditioned distribution with an inverse prompt that treats the failed prediction as successful. Directional clamping and group-relative advantages control the resulting updates. The updated grounding model subsequently predicts target locations from screenshots and instructions, while the Reflector supplies feedback during adaptation.

Critical strengths
  1. The introduction clearly motivates learning from failed grounding attempts, and contrastive calibration addresses a concrete failure mode of coordinate self-distillation.

  2. The evaluation covers established GUI benchmarks and includes scalar-feedback, prompted-teacher, and Reflector-reranking controls, rather than comparing only with an unadapted model.

Critical weaknesses
  1. The practical case for deployment-time parameter updates is not fully established. The paper identifies a genuine limitation of scalar feedback, but the broader framework largely combines existing reflection, privileged-context self-distillation, and policy optimization; the specific methodological advance is concentrated in the calibration design. Although reranking and scalar-feedback controls are included, the complete method is not compared under a matched total feedback/compute budget. Its full configuration takes 778 minutes versus 449 for GUI-RCPO, and the cheaper variant omits part of the main configuration. The value of the additional adaptation machinery therefore remains uncertain.

  2. The benchmark coverage is relevant, but the claimed deployment setting is insufficiently specified. Adaptation repeatedly accesses unlabeled target inputs before evaluation, while prediction/update ordering, resets, and checkpoint selection remain unclear. This makes batch transductive adaptation difficult to distinguish from improvement on newly arriving interfaces. Offline labeled Reflector training also limits the scope of the annotation-free claim. The inconsistent Reflector entries in Tables 5 and 8 and missing uncertainty for the main gains further weaken the interpretation of feedback reliability.

Questions for authors
  1. Please specify target-input exposure, prediction/update ordering, resets, and independent checkpoint selection. Which results represent transductive adaptation, and which test previously unseen inputs?

  2. Under matched total feedback or wall-clock budgets, how does the full method compare with Reflector reranking and scalar-feedback learning? Please include offline Reflector preparation.

  3. Please reconcile Tables 5 and 8, report uncertainty, and explain the choice of Qwen-family backbones. Reproducing available open-source baseline entries would also improve the comparison.

1 comment

I have read your response to my review and to the other viewers. My comment below is based on this situation.

My concern about novelty remains. The reflection ablation varies the feedback and keeps the learning rule fixed, so it leaves open what R-OPSD adds to existing privileged-context self-distillation. The methodological case still rests mainly on contrastive calibration.

I share Reviewer 52b7’s concern about what calibration learns from failures. The reply reports probability changes under different prompts without making clear what those probabilities measure or how many rollouts they cover. Even if the prompt suppresses a missed coordinate, the probability can move to another wrong location. The reply leaves open whether the weight update improves target hit rates when every sampled prediction misses.

The baseline results also support narrower conclusions than the paper draws. GUI-SD’s collapse was measured after replacing annotated targets with average predicted boxes, so it is hard to separate the effect of the distillation method from the quality of its input. For GUI-RCPO, the claim that all-miss groups provide no learning signal is unsupported. Its region-consistency rewards can still differ within those groups.

The practical case for deployment-time weight updates remains incomplete. The main protocol shows gains after batch adaptation, with limited evidence for continued improvement on newly arriving interfaces. The lightweight variant and the inference rollout counts still leave my original budget concern open. The full method has yet to be compared with reranking or scalar-feedback learning at a matched total cost that includes Reflector preparation, adaptation, and inference.

Reviewer_AxLjOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a test-time self-evolving framework for GUI visual grounding that improves grounding performance without requiring ground-truth annotations during adaptation. The framework consists of four stages: Exploration, Evaluation, Reflection, and Internalization. It introduces an MLLM-based Reflector to assess grounding predictions and generate textual feedback, which is then incorporated into Reflection-Guided On-Policy Self-Distillation (R-OPSD) to provide token-level supervision. Additionally, Contrastive Calibration is proposed to mitigate unreliable supervision caused by incorrect prefixes in autoregressive coordinate generation.

Critical strengths

(1)R-OPSD provides an approach to converting natural-language reflections into token-level supervision, allowing the model to learn from richer feedback than scalar rewards.

(2)Contrastive Calibration addresses unreliable supervision caused by incorrect prefixes in autoregressive coordinate generation, providing a well-motivated approach to improving optimization stability.

Critical weaknesses

(1)The Reflector is trained mainly with binary correctness and formatting rewards, but R-OPSD relies on its generated textual reflections for supervision. High classification accuracy does not necessarily guarantee the accuracy or usefulness of these reflections, which is not directly evaluated.

(2)According to Equation (7), Direction-Based Advantage Clamping restricts token-level advantages to non-positive values for failed predictions. While this suppresses incorrect tokens, it does not necessarily encourage the model to predict the correct coordinates. The effectiveness of this correction mechanism remains insufficiently analyzed.

Questions for authors

(1)How much of the improvement comes from the semantic content of the reflections? Would R-OPSD still perform well with generic, shuffled, or incorrect reflections?

(2)How does negative-only token-level supervision help the model identify the correct coordinates? Could the authors analyze whether suppressing incorrect tokens actually increases the probability of correct predictions?

Reviewer_1UPaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts GUI grounding models using unlabeled target screenshots. A Reflector judges predicted click locations and explains errors. The model then learns from its own reflection-conditioned predictions through on-policy self-distillation. Table 1 reports 4–7 point average gains on two small Qwen models over GUI-RCPO.

Critical strengths

The written feedback seems more useful than a simple correct/incorrect reward. It improves results by 35 points across 3 benchmarks (Table 2b). Removing the failure correction breaks training (Table 2a), and the reflection-guided teacher beats the student by 58 points (Table 3).

Critical weaknesses

The method is not really label-free. The Reflector was trained on 10k examples with ground-truth boxes, while GUI-RCPO did not use those labels. Even a binary Reflector reward raises MMBench-GUI from 58.2 to 62.8. The paper should compare against directly finetuning on the same labeled data. Table 10 also shows generic GUI-R1 adaptation within about 1 point of target-specific adaptation.

Questions for authors
  1. Does the grounding model emit a point or a box? The Reflector prompt takes a point while the teacher prompts ask for a box.
  2. Might be better to finetune the grounding adapter on the 10k labeled GroundCUA pairs and report Table 1; also run the loop with the zero-shot Reflector from Table 5
Reviewer_52b7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies adaptation of GUI grounding models using unlabeled screenshots and instructions from the target setting. A separately trained, frozen Reflector judges sampled coordinates and explains failures; the grounding model uses this feedback as privileged context for a self-teacher. R-OPSD converts the teacher's predictions into token-level updates, with contrastive calibration and direction-based clamping to stabilize learning from incorrect coordinate sequences, optionally combined with GRPO advantages. Experiments adapt on ScreenSpot-v2 or MMBench-GUI and evaluate six benchmarks, reporting average gains of up to 7.4 percentage points for Qwen2.5-VL-3B. The appendix adds larger backbones, Reflector diagnostics, reranking comparisons, and training-cost measurements.

Critical strengths
  1. The paper shows that the reflection contains useful learning signal beyond a better binary judge. Table 2(b) includes GRPO using the same Reflector's binary scores, and Table 2(a) compares otherwise matched variants with and without reflection. For example, with SSv2 adaptation, adding reflection to CC and clamping raises SSP accuracy from 24.6% to 28.5%. Table 3 also checks the conditional teacher directly. These controls support the core idea of internalizing diagnostic feedback, rather than attributing everything to a stronger reward model.
  2. The gains are supported across models and deployment choices. Table 1 includes improvements on benchmarks other than the adaptation dataset, and Table 7 extends the result to 7B/8B models. Table 4 compares against Reflector-based best-of-8, while Table 6 explicitly reports the full method's 1.73× training cost and the cheaper variant without query-level advantages. This makes the practical trade-off much easier to assess than an accuracy-only comparison.
Critical weaknesses
  1. The link between CC's token-level signals and geometric correction is not established. Section 3.3, lines 308–314, describes CC as suppressing the first incorrect token and discounting a corrupted suffix. However, coordinate strings encode numerical positions: neighboring pixels can have different token sequences, while many different coordinates can hit the same valid element. Figure 3(a) does not explain how the first incorrect token is identified or show that the resulting updates shift probability toward valid target locations. An early coordinate error also need not make the remaining coordinates uninformative. Table 2 supports CC as a useful stabilization technique, but the claimed error-localization mechanism needs evidence connecting token-level updates to geometric improvement.
  2. The evidence for self-evolution is limited to short adaptation runs with a separately trained evaluator. The Reflector is trained using GroundCUA labels and then frozen (Section 3.2; Appendix A.1). Table 2(b) usefully controls for its binary rewards, but Table 5 only measures the zero-shot base model's judging accuracy; it does not test whether that model can replace the trained Reflector in the full adaptation loop. Moreover, the adapted backbones in Tables 1 and 7 are general-purpose Qwen models; the GUI-specific models are only static comparisons. Section 4.1 fixes adaptation at two epochs, with no accuracy-versus-step/epoch curves or longer-run results; Figure 3(b) varies the integration weight instead. The results establish a short-run adaptation benefit, but leave open its dependence on evaluator training, its value for already-specialized GUI models, and its stability under continued updates. These gaps make the broader self-evolution framing premature.
Questions for authors
  1. How is the first incorrect coordinate token defined in Figure 3(a), given that multiple coordinate strings can identify the same element? Can you relate CC's updates to changes in target-hit probability or distance to the valid target region, rather than only the advantage on sampled tokens?
  2. How much of the final grounding improvement remains when the unmodified, frozen base model generates both S and R, with the rest of R-OPSD unchanged? This end-to-end ablation would clarify how much the method depends on the Reflector's additional supervised training.
  3. Does R-OPSD still improve an already GUI-specialized policy? Please include test-time adaptation of a model such as UI-TARS-1.5-7B, comparing its unadapted performance, binary-reward GRPO, GUI-RCPO, and R-OPSD under the same Reflector and adaptation budget.
  4. How does grounding accuracy evolve over adaptation steps, including beyond the current two-epoch budget? Curves on both the adaptation benchmark and other benchmarks would show whether gains persist, plateau, or reverse under continued feedback-driven updates.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.