- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ZoomSD, an efficient GUI grounding method that selects zoom-in regions from native attention during prefill, avoiding the coarse coordinate decoding required by conventional two-stage methods. A lightweight ILR adapter refines crop selection through containment-ranking supervision, with gradients isolated from OPSD backbone training. Experiments on Qwen3-VL-4B/8B show improvements over GUI-SD across three benchmarks. At the same 4B checkpoint, ZoomSD matches or exceeds two-stage zoom-in accuracy while reducing inference latency by 12.4–32.5%.
Critical strengths
S1: Attention-based cropping provides a useful accuracy–latency trade-off. ZoomSD selects zoom regions without first generating coordinates. In the reported comparison, it matches or exceeds conventional two-stage accuracy with lower latency. Grounding improvements are also observed across multiple benchmarks and two model sizes.
S2: The learned selector is lightweight and modular. It improves crop containment through a small adapter whose updates are separate from answer learning. It can also be trained on a frozen grounding checkpoint, making it a practical addition to an already trained model.
Critical weaknesses
W1: The incremental value of the proposed learning design is not fully established. Native attention cropping already accounts for most accuracy gains (Figure 3), while ILR provides modest and variable improvements. Joint training also shows no consistent advantage over post-hoc selector training (Table 5). Without comparisons to simpler trainable selectors, it remains unclear how much benefit the ILR and containment-ranking design provides beyond the native attention prior.
W2: The two-stage baseline decodes a full formatted response to obtain the coarse coordinate, whereas ZoomSD uses a fixed prefix ending before coordinate generation. A more informative baseline would use the same prefix and decode only the necessary coordinate tokens. Without this control, the reported speedup is credible for the evaluated setup, but its advantage over an optimized coordinate-based selector remains uncertain.
Questions for authors
Q1: What is the incremental benefit of the proposed ILR and ranking design over simpler learned crop selectors? Figure 3 shows that native attention already accounts for most accuracy gains, while ILR provides smaller, variable improvements. Could the authors compare against a simpler trainable selector using the same features and crop budget, and report uncertainty for ILR's incremental gains?
Q2: How much of the latency advantage remains against an optimized coordinate-decoding baseline? The two-stage baseline generates a full formatted response, whereas ZoomSD uses a fixed prefix before coordinate generation. Could the authors compare against a baseline that prefills the same prefix and decodes only the necessary coordinates, using the same checkpoint and evaluation subsets? This would better clarify the speedup gains attributable to eliminating coordinate decoding.
See questions in weaknesses also.
3 comments
Dear Reviewer JvCo,
We hope you have had a chance to review our response. With the discussion period ending soon, we would like to respectfully follow up on your two main questions.
For Q1, we added controlled comparisons with global head weighting, a per-head MLP, and ILR trained with multi-positive classification. These results show that several lightweight learned selectors can refine native attention, and we have accordingly tempered our claim that the particular ILR-ranking design is uniformly superior. For Q2, we added the stronger prefix-based coordinate baseline you suggested, which decodes only the coordinate suffix. ZoomSD remains faster on all three benchmarks, with a 9.5% reduction in population-weighted mean latency.
We would greatly appreciate knowing whether these additional controls address your concerns about the incremental value of ILR and the latency advantage over an optimized coordinate-decoding baseline. We would also be happy to clarify any remaining questions before the discussion period closes.
Thank you again for your careful review and for suggesting these informative controls.
Best regards,
The Authors
Dear Reviewer JvCo,
We would like to follow up once more, as less than 24 hours remain in the discussion period. We would greatly appreciate any brief feedback on our response.
Specifically, we would like to know whether the additional comparisons with simpler learned selectors and the stronger prefix-based coordinate-decoding baseline address your two
main concerns. If any reservations remain, we would welcome a brief indication of the unresolved issues so that we can respond before the deadline.
Thank you very much for your time and consideration.
Best regards,
The Authors
We thank the reviewers for their detailed and constructive comments. Below we address each point in turn.
Weakness 1 / Question 1: Incremental value of ILR and containment ranking
We thank the reviewer for suggesting comparisons with simpler learned selectors. On Qwen3-VL-4B, we implemented three alternatives: learned global head weights, a per-head MLP, and the same ILR architecture trained with a multi-positive classification objective instead of containment ranking. The results, together with native attention and our original ILR-ranking selector, are:
As in Figure 3, ILR improves full-target containment in all six model–benchmark settings by 1.4–8.3 percentage points, with click-accuracy gains of 0.1–3.1 percentage points. Its main effect is therefore improved containment rather than the overall zooming gain.
The controls show that global head weighting alone is insufficient, while the MLP and ILR-classification variants achieve accuracy comparable to ILR-ranking, with no consistent winner across benchmarks. Several lightweight learned selectors can thus refine the native localization prior; the benefit is not unique to the particular ILR-ranking objective. We will include these alternative-selector comparisons in the revision.
Weakness 2 / Question 2: Latency against a stronger prefix-based coordinate baseline
We added a prefix-based coordinate-zoom baseline using the same fixed prefix as ZoomSD and the same checkpoint, crop geometry, resize operation, and second-stage inference. This baseline prefills the fixed coordinate prefix in stage 1, decodes only the coordinate suffix, and stops immediately after the coordinates are emitted. All methods were evaluated on the complete 7,570-example manifests on four NVIDIA A40 GPUs. Each cell reports the mean latency per example:
Even against this stronger baseline, ZoomSD still reduces latency: by 4.0%, 12.3%, and 14.1% on ScreenSpot-Pro, UI-Vision, and OSWorld-G, respectively, corresponding to a 9.5% reduction in the population-weighted mean latency. The margin narrows because the prefix-based baseline already eliminates most of the stage-1 decoding—the very source of ZoomSD's latency advantage.
This advantage is bounded by the nature of the stage-1 output. The prefix trick applies when stage 1 emits a short, fixed-format output; it cannot remove the decoding cost of long, query-dependent content before the coordinates, such as the extended natural-language deliberation produced by some reasoning models before a tool call.