acceptodds
Under review as a conference paper at ICLR 2027

GroundPO: Policy Optimization via Internalization and Distillation for Visual Grounding

Abstract

Visual grounding becomes substantially more challenging when a query identifies its target through implicit comparison, spatial relations, and other forms of reasoning. Recent advances in on-policy distillation (OPD) provide a promising paradigm for such reasoning-intensive grounding. However, we find that distilling from a teacher with privileged information can cause training collapse, manifested as Privilege-Induced Behavioral Shift (PrivShift): privilege-specific artifacts and excessively long reasoning trajectories are inherited by the student. To address this issue, we propose GroundPO, a two-stage framework following the principle of internalizing the knowledge before teaching the policy. We first perform Policy Internalization to absorb privileged information into the teacher policy, encouraging reasoning that is free from privilege leakage and maintains an appropriate response length. Policy Distillation then uses the resulting internalized policy as the teacher to transfer its grounding capability through OPD to a student without privileged input. We further introduce DeepGround, comprising a high-quality training set and a challenging benchmark spanning five tracks of reasoning-intensive grounding. Experiments show that GroundPO substantially mitigates PrivShift, while GroundPO-27B achieves state-of-the-art performance, surpassing Gemini-3.1-Pro. Further analysis shows that GroundPO improves fine-grained perception and relative-position reasoning, demonstrating benefits beyond visual grounding on V* Bench.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
MLLM, visual grounding, reinforcement learning, on-policy distillation, referring expression comprehension

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wFAYOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces GroundPO, a two-stage framework for reasoning-intensive visual grounding. It first uses GRPO to stabilize a teacher with privileged textual information, then applies on-policy distillation to train a student without privileged inputs. The authors also construct DeepGround, covering five reasoning tracks. Experiments show improved grounding performance and substantially reduced privilege leakage and excessive reasoning.

Critical strengths
  1. The paper is well presented, with a clear motivation.
  2. The proposed dataset DeepGround provides both training data and a challenging benchmark covering five complementary reasoning tracks. Results show strong improvements in reasoning-intensive and fine-grained grounding.
  3. No additional inference inputs or architectural changes. The proposed method is easy to follow.
Critical weaknesses
  1. The method is validated only on Qwen3.5 models. Testing multiple model sizes does not establish generality across model families.
  2. Although DeepGround includes screenshots and web images, GUI grounding performance is not separately evaluated, leaving its effectiveness in this setting unclear.
  3. Gains on conventional grounding are limited, and overall performance on V* Bench remains unchanged.
Questions for authors
  1. See weakness above.
  2. Could you provide the training-set distribution across reasoning tracks and more details on filtering and human verification?
  3. During Stage II, the internalized teacher still receives privileged textual information. Is this input necessary after policy internalization? Could you compare distillation with and without privileged teacher inputs to clarify its contribution to grounding performance and behavioral stability?
5 comments
AuthorsOpenReview ↗

W1: Generalization Across Model Families

Thank you for suggesting validation across model families. In addition to the original Qwen3.5-4B/9B/27B experiments, we have applied GroundPO to Qwen3-VL-4B-Thinking, Qwen3-VL-8B-Thinking, Qwen3.6-27B, and Qwen3.8-27B, with results shown in Tab. R3-1. F1@mIoU improves for all four models, by 12.16%, 10.01%, 19.07%, and 7.32%, respectively. These results extend the scope of validation from different parameter scales of Qwen3.5 to Qwen3-VL and other Qwen model families, supporting the applicability of GroundPO to different base models.

Tab. R3-1. DeepGround-Test performance of GroundPO across model families

ModelP@mIoUR@mIoUF1@mIoU
Qwen3-VL-4B-Thinking25.6420.6322.87
GroundPO (Qwen3-VL-4B)34.98 (+9.34)35.08 (+14.45)35.03 (+12.16)
Qwen3-VL-8B-Thinking27.4123.2125.13
GroundPO (Qwen3-VL-8B)35.14 (+7.73)35.14 (+11.93)35.14 (+10.01)
Qwen3.6-27B38.4236.0237.18
GroundPO (Qwen3.6-27B)57.00 (+18.58)55.53 (+19.51)56.25 (+19.07)
Qwen3.8-27B45.8248.2447.00
GroundPO (Qwen3.8-27B)54.53 (+8.71)54.11 (+5.87)54.32 (+7.32)
AuthorsOpenReview ↗

W2: Generalization to GUI Grounding

Thanks for suggesting a separate evaluation of GUI grounding. We have added four GUI grounding benchmarks, ScreenSpot-Pro, ScreenSpot-V2, UI-Vision, and MMBenchGUI (Tab. R3-2, also included in Tab. 5 of the revised manuscript). GroundPO improves the average GUI accuracy on Qwen3.5-4B, 9B, and 27B by 1.53%, 1.55%, and 1.05%, respectively. GUI grounding also requires identifying a specific visual element from a language description and localizing it precisely. Although GroundPO is trained only on DeepGround-Train, its gains carry over to GUI grounding.

Tab. R3-2. Accuracy comparison of GroundPO on V* Bench and four GUI grounding benchmarks (%).

ModelV* BenchScreenSpot-ProScreenSpot-V2UI-VisionMMBenchGUIGUI Avg.
Qwen3.5-4B81.6854.2791.8230.0480.8864.25
GroundPO-4B83.25 (+1.57)57.18 (+2.91)91.98 (+0.16)32.98 (+2.94)80.96 (+0.08)65.78 (+1.53)
Qwen3.5-9B85.8660.4793.0838.4683.6968.92
GroundPO-9B86.39 (+0.53)62.05 (+1.58)93.87 (+0.79)40.74 (+2.28)85.22 (+1.53)70.47 (+1.55)
Qwen3.5-27B86.9166.7994.3451.9886.8474.99
GroundPO-27B88.48 (+1.57)68.12 (+1.33)94.34 (+0.00)53.91 (+1.93)87.79 (+0.95)76.04 (+1.05)
AuthorsOpenReview ↗

W3: Conventional Grounding and General Visual Reasoning

Thank you for the attention to conventional grounding and general visual understanding capabilities.

V* Bench. We rechecked the evaluation protocol for V* Bench and evaluated it using the same GPT-OSS-120B judge as Vision-OPD. With this protocol, GroundPO-4B, 9B, and 27B improve V* Bench overall accuracy by 1.57%, 0.53%, and 1.57%, respectively (Tab. R3-2). Tab. 5 of the revised manuscript further reports the Direct Attributes and Relative Position dimensions: all three models improve on Relative Position, while Direct Attributes is maintained or improved. GroundPO thus improves grounding without degrading general visual reasoning.

Conventional grounding. This paper focuses on reasoning-intensive visual grounding. GroundPO improves over the corresponding base models on GroundingME, PR-Bench, and Ref-Adv at all scales, while on RefCOCO and RefCOCOg its performance stays close to the base models. Some RefCOCO/RefCOCOg annotations are coarser than the referring expression or do not match its target granularity. GroundPO tends to produce finer boxes, so its predictions can match the expression better yet receive lower IoU against the original annotation (App. C.5). Consistent with this, GroundPO achieves clear gains on the fine-grained benchmark RefCOCOm at all scales (Tab. 2).

Reviewer_SEoqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents GROUNDPO, a two-stage post-training framework for reasoning-intensive visual grounding. To mitigate "Privilege-Induced Behavioral Shift" (privilege leakage and verbosity drift) observed when distilling from a teacher with privileged information, the authors propose first training the teacher via GRPO on privileged entity descriptions (Policy Internalization), followed by on-policy distillation to an unprivileged student (Policy Distillation). The paper also introduces DEEPGROUND, an automated benchmark and training set.

Critical strengths
  1. The paper provides a clear and systematic characterization of the distribution mismatch caused by privileged information in visual grounding, comprehensively analyzing the behavioral impact of different privilege modalities (visual annotations, transition instructions, and textual descriptions).

  2. The identification and empirical formalization of PrivShift—specifically decomposing it into privilege leakage and verbosity drift under token-level distillation—is an interesting and insightful observation for multimodal distillation.

  3. The authors construct DEEPGROUND, a high-quality benchmark and training corpus with a rigorous automated-to-manual construction and verification pipeline. The benchmark is comprehensive and effectively covers multiple challenging reasoning patterns (e.g., contrastive, spatial, conceptual).

  4. The proposed two-stage framework is conceptually straightforward and empirically demonstrated to stabilize generation length and suppress leakage, offering practical engineering value for post-training pipelines.

Critical weaknesses

Although this paper provides a thorough empirical study on privileged distillation and introduces a solid benchmark (DEEPGROUND), I have several critical concerns.

  1. Behavioral shifts and privilege leakage caused by conditioning on privileged information have been extensively discussed in recent literature on on-policy self-distillation (OPSD). While this paper observes this phenomenon within visual grounding, it does not sufficiently elaborate on what makes this failure fundamentally unique to the visual grounding setting. It is recommend to further highlight the specificality of this privilege leakage failure in visual grounding scenario.

  2. The proposed "Policy Internalization" is essentially an offline GRPO stage prior to distillation rather than an online or distillation-time adaptation mechanism. In practice, the framework operates as a two-stage pipeline with a simple combination of RL and OPSD, causing the methodological advance appears somewhat incremental.

  3. A key limitation of the main results in Table 1 is that GROUNDPO is the only model trained on DEEPGROUND-TRAIN, while all external baselines are evaluated in a zero-shot manner. To decouple the benefits of the proposed framework from the quality of the new dataset, the authors should include direct post-training baselines (e.g., standard GRPO, vanilla OPSD, and recent OPSD variants) trained under the identical data setting.

  4. The two-stage framework introduces considerable training complexity and computational overhead. However, as shown in Table 3, vanilla unprivileged GRPO already accounts for the vast majority of the gain (+11.08% F1 over the base model), while the full privileged distillation pipeline adds only a modest +2.08% marginal improvement, raising questions regarding cost-effectiveness.

Questions for authors

See weakness (1-4) above.

3 comments
AuthorsOpenReview ↗

W1: Grounding-Specific Privilege-Induced Behavioral Shift

We agree that prior OPSD/OPD work has discussed behavioral shifts induced by privileged conditioning. What is specific to visual grounding is the following.

(i) Privilege for grounding is the answer location itself. In reasoning tasks, a reference solution can be given as text that the student could plausibly produce. In grounding, the most direct privilege is the ground-truth region, which must be drawn on the image or given as coordinates. The teacher then reasons by referring to this visual mark (“the target is in the red bounding box”), and the student imitates references to a mark that does not exist in its input. Adding a transition instruction (PV,T\mathcal{P}^{V,T}), as in prior OPSD work, does not remove this behavior (89.50% leakage in Tab. R2-2).

(ii) Existing visual-privilege designs do not transfer. Vision-OPD avoids explicit annotations by giving the teacher a crop of the relevant region. For grounding, however, the crop removes exactly the global context needed to localize the target in the full image (especially important in reasoning-intensive grounding), and the resulting collapsed student obtains an F1@mIoU of 0 (Tab. R2-2).

Textual privilege, built from our entity tree, largely removes the leakage in (i), but still suffers from verbosity drift. This motivates Policy Internalization. Tabs. 3 and 4 of the revised manuscript quantify these observations through grounding performance and PrivShift metrics, respectively. To our knowledge, this is the first study of privileged OPD for visual grounding, and we have revised the introduction to state these grounding-specific aspects explicitly.

AuthorsOpenReview ↗

W2 & W4: Two-Stage Framework Design and Training Efficiency

Thank you for their attention to methodological design and training overhead. GroundPO is a two-stage post-training framework, rather than a distillation-time adaptation mechanism. It differs from a simple combination of GRPO and OPD in two key aspects:

(i) Privileged Information Design and Internalization. GroundPO adopts textual privilege (PT\mathcal{P}^{T}) to mitigate privilege leakage, achieving 36.88% F1@mIoU, compared with 33.11% using visual privilege (PV\mathcal{P}^{V}). Unlike vanilla GRPO, Stage-I incorporates PT\mathcal{P}^{T} to internalize privileged grounding knowledge into the teacher policy, which is subsequently distilled into the student in Stage-II.

(ii) Thinking-Mode Consistency. Unlike the original OPSD setting [1], which distills a thinking-on teacher into a thinking-off student, GroundPO maintains thinking mode for both teacher and student throughout training. As discussed by Adhinarta et al. [2], OPSD's gains may largely reflect thinking-mode transfer. In contrast, GroundPO focuses on transferring internalized privileged grounding knowledge between thinking-enabled policies.

We acknowledge the additional training overhead. However, GroundPO improves Stage-I F1@mIoU by 0.97%/1.44% on 4B/9B (three-seed averages) and 3.59% on 27B, achieving an overall 18.89% gain over the Qwen3.5-27B base model. Together with improved generation stability, these results support the effectiveness of GroundPO beyond a simple combination of RL and OPSD. Detailed multi-seed results are provided in App. B.3 of the revised manuscript.

Tab. R2-1. Performance of GroundPO-4B/9B across multiple random seeds and results of GroundPO-27B on DeepGround-Test.

ModelP@mIoUR@mIoUF1@mIoU
Qwen3.5-4B24.2423.9224.08
Stage-I-4B35.60 ± 1.1135.62 ± 1.0935.61 ± 1.10
GroundPO-4B36.60 ± 0.6036.56 ± 0.5436.58 ± 0.57
Qwen3.5-9B29.3128.7529.02
Stage-I-9B39.45 ± 0.5039.39 ± 0.3239.42 ± 0.41
GroundPO-9B40.92 ± 0.7440.79 ± 0.7940.86 ± 0.76
Qwen3.5-27B38.0637.3037.68
Stage-I-27B53.0052.9552.98
GroundPO-27B56.6056.5456.57

Reference

[1] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models (Zhao et al., 2026)

[2] Does self-distillation improve reasoning? (Adhinarta et al., 2026)

AuthorsOpenReview ↗

W3: Controlled Comparisons Under Identical Training Data

Thanks for suggesting controlled comparisons. Tab. 3 of the original manuscript already included vanilla GRPO trained on the same data. We have now added SFT, vanilla OPD under three privilege configurations, Vision-OPD, continued GRPO, and joint GRPO+OPD, all trained on DeepGround-Train with Qwen3.5-4B. The SFT trajectories are generated by the Stage-I model. Grounding performance and generation-behavior statistics are shown in Tab. R2-2, and the comparisons have been added to Sec. 5.3 of the revised manuscript.

SFT (33.15) and vanilla GRPO (35.16) both outperform the base model (24.08) but fall below Stage-I with PT\mathcal{P}^{T} (36.88) and the complete GroundPO (37.24). All three vanilla OPD configurations exhibit PrivShift; in particular, vanilla OPD with PT\mathcal{P}^{T} has the highest precision but loses most of its recall to truncation and repetition. Stage-I removes these behaviors, and Stage-II further improves localization. Since all methods in Tab. R2-2 use the same training data, the gap between GroundPO and these baselines reflects the training framework rather than the dataset.

Tab. R2-2. Comparison of training methods and PrivShift analysis on Qwen3.5-4B

MethodP@mIoU (%)R@mIoU (%)F1@mIoU (%)Leakage (%)Trunc. (%)Rep. (%)Avg. length (tokens)
Qwen3.5-4B24.2423.9224.080.101.601.80249.3
Vision-OPD (crop-based)0.000.000.000.000.000.0055.7
vanilla OPD w. PV\mathcal{P}^{V}31.8723.8427.2857.3012.8011.101239.8
vanilla OPD w. PV,T\mathcal{P}^{V,T}32.8824.4128.0289.509.208.201048.0
vanilla OPD w. PT\mathcal{P}^{T}38.918.9114.504.2059.5641.443090.0
SFT33.2333.0733.150.000.670.67151.4
vanilla GRPO35.1235.1935.160.000.000.19340.0
Stage-I w. PT\mathcal{P}^{T}36.8836.8836.880.100.000.00201.3
GroundPO-4B37.2937.1837.240.100.200.20296.7
Reviewer_36jqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors identify Privilege-Induced Behavioral Shift and introduce GroundPO, a two-stage training framework for visual grounding. In the first stage, Policy Internalization, GRPO incorporates privileged information into the teacher policy. In the second stage, Policy Distillation, OPD transfers the teacher’s grounding capability to the student without exposing it to privileged input. The authors also construct DeepGround, a dataset with training and test splits covering five reasoning-intensive grounding tracks. Results on both the proposed in-domain dataset, DEEPGROUND-TEST, and out-of-domain grounding benchmarks demonstrate the effectiveness of GroundPO.

Critical strengths
  1. The paper is well organized and easy to follow.

  2. The motivation is intuitive and grounded in empirical evidence. PrivShift is a practical failure mode of privileged OPD. The paper characterizes it clearly through leakage and length statistics, as well as qualitative examples.

  3. The proposed method is straightforward and practical. DeepGround provides both training and evaluation splits, and it is challenging, which benefits the community.

  4. Extensive evaluation across three model scales and seven benchmarks shows consistent improvements over the base models.

Critical weaknesses
  1. It is unclear whether the proposed two-stage training framework is necessary. In Table 3, most of the improvement over the untrained model π0\pi_0 comes from the first-stage GRPO training, even though the Stage I model is evaluated using only the original image and query. Without reporting the variance across different runs, it is uncertain whether the improvement from the second-stage OPD is statistically significant.

  2. In Table 2, only GroundPO and the base model are compared. i) The improvement is much smaller than that on the in-domain test benchmark in Table 1. Is this because the training and test splits share the same distribution? ii) For comprehensiveness and a fair comparison, adding the models in Table 1 would provide more information. For example, LocateAnything reports 77.6 F1@mIoU on RefCOCOg, which exceeds GroundPO-27B. Without such comparisons, it is unclear whether the large gap on DeepGround-Test reflects distribution shift or evaluation settings rather than grounding capability. Furthermore, the generalization ability of the proposed training data is unclear, which weakens the SOTA claim.

  3. Comparisons with some basic baselines are missing, e.g., SFT, vanilla OPD, and its variants, such as Vision-OPD and OPD with simple regularization that directly targets PrivShift. Such baselines would show whether the Stage I internalization is necessary, or whether a lightweight fix to OPD suffices.

  4. “Mitigating verbosity drift” may, in fact, be reasoning collapse. In Figure 12, the reasoning trace of the GroundPO model contains little substantive reasoning compared with that of the base model.

  5. Given Weaknesses 1 and 3, the novelty of the proposed method is limited, since the paper provides no insight into why the two-stage pipeline is necessary.

  6. Minor: Figure 2 is hard to follow for first-time readers. The π\pi notations are explained only in later sections and are undefined when the reader first encounters Figure 2.

Questions for authors
  1. How about replacing the Stage II OPD with continued GRPO (without privileged input) from the Stage-1 checkpoint under similar compute? Alternatively, how would a single-stage approach compare, e.g., jointly optimizing the GRPO reward (possibly with an extra length or leakage reward) and an on-policy self-distillation loss from the privilege-conditioned policy?

  2. The frequent P = R on DeepGround-Test implies that multi-target samples might be rare. What fraction of the samples are multi-target? If multi-target samples are rare, how is the multi-target grounding capability evaluated?

  3. For the Conceptual subset, there is a huge improvement for the 27B model, while the improvements for the 4B and 9B models are minor. What is the intuition behind this?

  4. What is the motivation behind the following setup: “For the 4B and 27B models, we use 4,800 hard examples selected from DEEPGROUND-TRAIN based on Qwen3.5-4B predictions with IoU < 0.75, with both the batch size and PPO mini-batch size set to 32. For the 9B model, we use all 8,192 examples from DEEPGROUND-TRAIN, with both sizes set to 64.”?

7 comments
AuthorsOpenReview ↗

W1: Effectiveness and Statistical Significance of Two-Stage Training

Thank you for focusing on the independent contribution of Stage-II and the stability of the experimental results. We have added complete experiments on Qwen3.5-4B/9B with three random experiments, and report the means and standard deviations in Tab. R1-1. Compared with Stage-I, GroundPO-4B and GroundPO-9B further improve the mean F1@mIoU by 0.97% and 1.44%, respectively, with Stage-II yielding positive gains across all three seeds. These results support the statistical significance of the gains from Stage-II. The per-seed results for GroundPO-4B/9B are provided in App. B.3 of the revised manuscript. In addition, on Qwen3.5-27B, Stage-I and GroundPO achieve F1@mIoU scores of 52.98 and 56.57, respectively, with Stage-II providing a further improvement of 3.59%. This result further demonstrates the independent contribution of Policy Distillation.

Tab. R1-1. Performance of GroundPO-4B/9B across multiple random seeds and results of GroundPO-27B on DeepGround-Test.

ModelP@mIoUR@mIoUF1@mIoU
Qwen3.5-4B24.2423.9224.08
Stage-I-4B35.60 ± 1.1135.62 ± 1.0935.61 ± 1.10
GroundPO-4B36.60 ± 0.6036.56 ± 0.5436.58 ± 0.57
Qwen3.5-9B29.3128.7529.02
Stage-I-9B39.45 ± 0.5039.39 ± 0.3239.42 ± 0.41
GroundPO-9B40.92 ± 0.7440.79 ± 0.7940.86 ± 0.76
Qwen3.5-27B38.0637.3037.68
Stage-I-27B53.0052.9552.98
GroundPO-27B56.6056.5456.57
AuthorsOpenReview ↗

W2: Cross-Benchmark Comparisons and Generalization

Thanks for the suggestions on cross-benchmark comparisons and the SOTA claim. We focus on reasoning-intensive visual grounding and accordingly designed the pipeline shown in Fig. 3 of the manuscript, from which DeepGround-Train/Test were constructed. Compared with GroundingME, PR-Bench, and Ref-Adv, they differ in aspects such as target distractors, occlusion conditions, and referring query formats; therefore, the magnitudes of improvement are not identical across benchmarks.

Comparison with specialist models. We have evaluated Rex-Omni and LocateAnything on all seven grounding benchmarks (Tab. R1-2). GroundPO-27B, trained on only 8K samples from DeepGround-Train, achieves the highest F1@mIoU on all four reasoning-intensive benchmarks and on the fine-grained benchmark RefCOCOm, although Rex-Omni and LocateAnything are trained on 22M and 138M samples, respectively. LocateAnything performs better on the conventional benchmarks RefCOCO and RefCOCOg, consistent with its large-scale training on conventional grounding datasets. Accordingly, we have narrowed the SOTA claim in the revised abstract to: “GroundPO-27B achieves state-of-the-art performance on reasoning-intensive visual grounding tasks.” One point worth highlighting is that, on RefCOCO and RefCOCOg, some annotations are coarser than the referring expression or do not match its target granularity. GroundPO tends to produce finer boxes, so its predictions can match the expression better yet receive lower IoU against the original annotation, as discussed in App. C.5.

Generalization of GroundPO. In the revised Tab. 5, GroundPO improves V* Bench overall accuracy by +1.57%/+0.53%/+1.57% and the average accuracy on four GUI grounding benchmarks by +1.53%/+1.55%/+1.05% at 4B/9B/27B. Although GroundPO is trained only on DeepGround-Train, its gains carry over to general visual understanding and GUI grounding.

Tab. R1-2. F1@mIoU of specialist models and GroundPO.

ModelDeepGround-TestGroundingMEPR-BenchRef-AdvRefCOCOmRefCOCORefCOCOg
Rex-Omni5.214.9241.4236.3136.2667.5073.05
LocateAnything16.2523.4754.0743.0034.1576.2677.71
GroundPO-4B37.2443.9859.2849.3639.1870.1872.80
GroundPO-9B40.9348.3661.7051.6340.7272.2175.36
GroundPO-27B56.5758.2662.7153.4244.0573.3076.37
AuthorsOpenReview ↗

W3, W5 & Q1: Methodological Design and Alternative Training Strategies

Thank you for suggesting additional standard training methods and alternatives to the two-stage strategy. Using Qwen3.5-4B and DeepGround-Train, we have added experiments with SFT, vanilla GRPO, vanilla OPD under three privilege configurations, Vision-OPD, continued GRPO in place of Stage-II, and single-stage joint GRPO+OPD optimization. SFT is trained on trajectories generated by the Stage-I model. The results are shown in Tab. R1-3, and the comparisons have been added to Sec. 5.3 of the revised manuscript.

Vanilla OPD and Vision-OPD. Directly applying privileged OPD does not transfer the teacher’s advantage to the student: all three vanilla OPD variants show privilege leakage, verbosity drift, or both. With PT\mathcal{P}^{T}, vanilla OPD reaches the highest precision (38.91), but 59.56% of its responses are truncated before a box is produced, so recall drops to 8.91. SFT and vanilla GRPO give more balanced P/R gains, and Stage-I, which internalizes PT\mathcal{P}^{T}, outperforms both. Under Vision-OPD’s original crop-based privilege, the student obtains an F1@mIoU of 0. The crop-conditioned teacher only observes the target region and thus cannot supervise localization in the full image. This shows that the privilege design of existing visually privileged OPD does not carry over directly to visual grounding.

Alternatives in Q1. As suggested by the reviwer, starting from the Stage-I checkpoint, continuing GRPO without privilege under a comparable training budget yields 34.92 F1@mIoU, below Stage-I (36.88). Jointly optimizing LGRPO\mathcal{L}^{GRPO} and LOPD\mathcal{L}^{OPD} in a single stage (OPD weight 0.5, following [1]) yields 28.09, with 15.13% truncation and 13.51% repetition. In comparison, the complete GroundPO achieves 37.24.

Why internalization before distillation. OPD matches the teacher’s full token-level distribution, so the student imitates the behavior of the privileged teacher, including behaviors that only make sense when the privilege is visible, such as referring to the “red box” or enumerating all confusable entities described in PT\mathcal{P}^{T}. A regularizer can suppress some of these symptoms, but the teacher distribution itself remains shifted. Policy Internalization changes the teacher instead: GRPO with privileged input rewards only a correct and complete box under a fixed generation budget, so the teacher keeps the knowledge in PT\mathcal{P}^{T} while returning to generation behavior the student can reproduce (Tab. 4). Stage-II then distills from this adapted teacher. Consistent with this view, every simpler combination of the same components in Tab. R1-3 underperforms GroundPO.

Tab. R1-3. Comparison of grounding performance and generation behavior across training strategies on Qwen3.5-4B.

MethodP@mIoU (%)R@mIoU (%)F1@mIoU (%)Leakage (%)Trunc. (%)Rep. (%)Avg. length (tokens)
Qwen3.5-4B24.2423.9224.080.101.601.80249.3
Vision-OPD (crop-based)0.000.000.000.000.000.0055.7
GRPO + OPD w. PV\mathcal{P}^{V}30.9625.7128.091.7315.1313.511031.5
Continued GRPO after Stage-I34.9234.9234.920.480.000.10252.0
vanilla OPD w. PV\mathcal{P}^{V}31.8723.8427.2857.3012.8011.101239.8
vanilla OPD w. PV,T\mathcal{P}^{V,T}32.8824.4128.0289.509.208.201048.0
vanilla OPD w. PT\mathcal{P}^{T}38.918.9114.504.2059.5641.443090.0
SFT33.2333.0733.150.000.670.67151.4
vanilla GRPO35.1235.1935.160.000.000.19340.0
Stage-I w. PT\mathcal{P}^{T}36.8836.8836.880.100.000.00201.3
GroundPO-4B37.2937.1837.240.100.200.20296.7

Reference

[1] KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning (Xu et al., 2025)

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Contributions, Rebuttal Responses, and Manuscript Revisions

We sincerely thank all reviewers for their constructive feedback and recognition of our work. We summarize our contributions and the additional evidence provided in response to their comments.

Main Contributions and Reviewer-Recognized Strengths

  1. Reasoning-Intensive Visual Grounding and DeepGround. We introduce DeepGround, with 8,192 training samples and 1,044 test samples across five reasoning tracks. 36jq, SEoq, and wFAY recognize the value of its challenging data and careful construction.
  2. Privilege-Induced Behavioral Shift (PrivShift). We for the first time systematically investigate privileged OPD for referring expression comprehension and identify privilege leakage and verbosity drift under vanilla OPD. 36jq and SEoq highlight the empirical motivation and characterization of these behaviors.
  3. Two-Stage GroundPO. GroundPO uses textual privilege and Policy Internalization to establish a behaviorally stable teacher, followed by Policy Distillation to a student without privileged input. 36jq and SEoq acknowledge its practical value, while wFAY notes that it requires no architectural changes or extra inference inputs.
  4. Effectiveness and Generalization. Trained on only 8K samples, GroundPO-27B leads the compared models on four reasoning-intensive grounding benchmarks and RefCOCOm, with further gains on V* Bench and GUI grounding. 36jq and wFAY recognize the strong results across benchmarks and model scales.

Key Reviewer Concerns and Additional Evidence

  1. Methodology and Controlled Baselines (36jq-W3/W5/Q1; SEoq-W1/W2/W3/W4). We added SFT, vanilla GRPO, three privileged vanilla OPD variants, Vision-OPD, continued GRPO, and joint GRPO+OPD comparisons under matched training data. GroundPO-4B achieves 37.24 F1@mIoU, versus 35.16 for vanilla GRPO and 33.15 for SFT; direct privileged OPD exhibits PrivShift, supporting our internalization-before-distillation design (Sec. 5.3).
  2. Generalization and Performance Scope (36jq-W2; wFAY-W1/W2/W3). We evaluated Rex-Omni and LocateAnything on seven benchmarks and clarified that our SOTA claim concerns reasoning-intensive visual grounding. GroundPO also improves F1@mIoU on four additional Qwen models by 7.32–19.07 percentage points, and improves V* Bench overall accuracy and average GUI grounding accuracy across all three Qwen3.5 scales (Tab. 5).
  3. Two-Stage Effectiveness and Stability (36jq-W1; SEoq-W2/W4; wFAY-Q3). Three-seed results show mean Stage-II gains of 0.97/1.44 percentage points for 4B/9B, with a 3.59-point gain for 27B (App. B.3). Removing textual privilege from the Stage-II teacher reduces student F1@mIoU from 37.24 to 36.87, while both settings retain stable generation behavior.

Additional Clarifications (36jq-W4/W6/Q2/Q3/Q4; wFAY-Q2). We clarified single-target task scope, multi-target COCO transfer, training configurations, DeepGround quality control, and reasoning behavior, alongside figure and notation refinements.

Reviewer Feedback Following Rebuttal

The rebuttal has received positive feedback from two reviewers. SEoq acknowledged our controlled comparisons and grounding-specific analysis, described the work as a "solid and complete study," and raised their rating from 2 to 3. wFAY explicitly confirmed that their concerns had been addressed and maintained their positive rating of 3. Although 36jq has not yet provided a follow-up assessment, we have responded to each concern with extensive experimental evidence, additional analyses, and corresponding manuscript revisions.

Summary of Manuscript Revisions

The revised manuscript adds standard training baselines (Sec. 5.3, Tab. 3), truncation and repetition metrics for PrivShift (Tab. 4), updated V* Bench results and four GUI grounding benchmarks (Tab. 5), and three-seed evaluations (App. B.3). We also standardized terminology and refined Fig. 2 and the abstract's SOTA claim, updated a reasoning case study (App. C.4), and added RefCOCO/RefCOCOg qualitative examples illustrating annotation-granularity mismatches (App. C.5).

Closing Remarks

We thank the reviewers again for their valuable feedback. Together, these results strengthen the central finding that internalizing privileged grounding knowledge before distillation mitigates PrivShift while improving localization and generalization.

Area_Chair_8ofPOpenReview ↗

Official Comment by Area Chair 8ofP

Dear reviewers,

Thank you very much for your time and effort in reviewing this submission. The authors have now submitted their rebuttal, and the discussion period is nearing its end. We would greatly appreciate it if you could take some time to carefully review the authors’ responses and share your assessment of whether your concerns have been adequately addressed. Thank you again for your valuable time and contributions to the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.