acceptodds
Under review as a conference paper at ICLR 2027

RS-OPSD: Reliable Privileged On-Policy Self-Distillation For Ultra-High-Resolution Remote Sensing VQA

14 views

Abstract

Ultra-high-resolution (UHR) remote sensing visual question answering (VQA) requires models to resolve small visual evidence within extremely large images. Existing approaches typically rely on token pruning, visual search, or tool-augmented reasoning at inference time. We instead investigate whether the benefit of *zoom-in visual privilege* can be internalized into the model. We introduce **RS-OPSD**, a reliable privileged on-policy self-distillation (OPSD) framework for UHR remote sensing VQA. To provide high-quality privileged information with explicit question-relevant evidence, we construct **GeoEvidence-6K**, containing 6,750 VQA samples across seven task categories with evidence-region annotations, and develop **Human Feedback-Guided Skill Refinement (HF-SR)** for scalable annotation. To address context loss from tight crops and conflicting signals from imperfect teachers, **RS-OPSD** introduces **Context-Preserving Visual Privilege (CPVP)** and **Correctness-Aligned Distillation (CAD)**. Without any additional visual search or tool calls at inference time, **RS-OPSD** achieves state-of-the-art (SOTA) performance on XLRS-Bench, MME-RealWorld-RS, and LRS-VQA, outperforming previous SOTA models of comparable scale by an average of 4.0 percentage points. Moreover, our 2B variant, **RS-OPD-*Lite***, surpasses most 8B-scale models while achieving the fastest measured inference speed.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Remote Sensing, Ultra-High-Resolution Visual Question Answering, Vision On-Policy Self-Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LDyKOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets the “where-to-look” bottleneck in ultra-high-resolution (UHR) remote sensing VQA, where models struggle to locate and exploit question-relevant visual evidence in extremely large images. It proposes RS-OPSD, a reliable privileged on-policy self-distillation framework that internalizes zoom-in evidence into the student during training while requiring only standard full-image input at inference. The authors construct GeoEvidence-6K with explicit evidence-region annotations and introduce HF-SR for human-feedback-guided annotation refinement. Methodologically, CPVP provides the teacher with global, contextual, and fine-grained evidence views, while CAD filters unreliable teacher supervision via sample-level reliability gating and correctness-aligned token updates. Experiments report strong results on three UHR remote sensing VQA benchmarks without additional visual search or tool calls at inference.

Critical strengths
  1. The paper clearly identifies the evidence-localization bottleneck in UHR remote sensing VQA and frames training-time privilege internalization as a sensible alternative to token pruning, inference-time search, and tool use.
  2. The pilot study exposes the limitations of naive crop-only OPSD, and CPVP and CAD are introduced as targeted remedies. The ablation results support their individual and combined effectiveness, making the design logic reasonably solid.
  3. GeoEvidence-6K and the HF-SR pipeline are valuable contributions in their own right. RS-OPSD also achieves a favorable accuracy-efficiency trade-off among the compared methods, and the Lite variant is notable for surpassing most evaluated 8B-scale models with only a 2B student while maintaining low inference latency.
Critical weaknesses
  1. The paper aims to internalize privileged visual information into the student model. However, it remains unclear what inductive bias this process introduces, what capabilities are actually internalized, and under which domains or task conditions such a bias is effective.
  2. Table 3 shows that naive OPSD slightly underperforms LoRA fine-tuning (45.2 vs. 45.7), while the complete framework improves performance to 49.3. However, the absence of a full-parameter SFT baseline makes it difficult to establish whether the proposed framework offers advantages beyond conventional supervised fine-tuning.
  3. The paper claims SOTA performance on three remote sensing VQA benchmarks, but several evaluated baselines are relatively outdated. More recent closed-source models (e.g., GPT-6, Claude Opus 5.5, Gemini 3.1 Pro) and open-source models (e.g., Qwen3.5-9B, Qwen3.8-27B) are not included, making the SOTA claim less convincing.
Questions for authors

The following questions correspond directly to the critical weaknesses above, and satisfactorily addressing them would positively influence my overall assessment and potentially increase my rating.

  1. Could the authors provide a more concrete explanation of what is internalized through OPSD? Specifically, does the proposed framework induce an inductive bias toward identifying and attending to question-relevant visual regions, or does it primarily improve the model's task-specific knowledge? Furthermore, what properties of a task or domain make this form of privileged information particularly effective? It would be valuable to discuss whether the proposed approach is expected to generalize to other high-resolution visual understanding tasks beyond remote sensing VQA.
  2. Could the authors include a full-parameter SFT baseline using the same GeoEvidence-6K dataset and comparable training settings? Given that naive OPSD underperforms LoRA in the pilot study, it is important to establish whether OPSD + CPVP + CAD offers advantages over full-parameter SFT. Beyond improvements in benchmark accuracy, are there specific capabilities, generalization properties, or robustness benefits that the proposed framework can provide but conventional SFT cannot?
  3. Could the authors provide additional comparisons with stronger, more recent closed-source and open-source vision-language models, such as GPT-6, Claude Opus 5.5, Gemini 3.1 Pro, and newer Qwen variants, where applicable? In particular, comparisons with more recent models of similar parameter scales would help establish whether RS-OPSD remains competitive under a fair model-size setting, while evaluations against larger frontier models would provide useful context for the broader SOTA claim.
Reviewer_a7hxOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RS-OPSD to address the challenges of localizing fine-grained evidence and the high computational overhead of visual search during inference in ultra-high-resolution (UHR) remote sensing VQA. The authors construct GeoEvidence-6K, a dataset annotated with target regions, and introduce CPVP to enhance the Teacher's visual information through global, contextual, and local views. Additionally, they propose CAD to filter reliable distillation signals based on answer correctness. Via On-Policy Self-Distillation, the model eliminates the need for additional cropping or tool invocation during inference, achieving competitive accuracy and inference efficiency across three remote sensing VQA benchmarks.

Critical strengths
  1. Practically meaningful motivation. Addressing the difficulty of utilizing fine-grained evidence in UHR images and the high cost of visual search during inference, the approach of shifting visual privileges to the training phase is well-motivated and the overall research direction is sound.
  2. Targeted method design. CPVP mitigates the loss of context caused by local cropping, while CAD filters distillation signals to address the unreliability of Teacher predictions. Both components directly correspond to limitations observed in preliminary experiments.
  3. Notable data contribution. GeoEvidence-6K comprises 6,750 samples with human-verified evidence region annotations, providing a solid data foundation for researching visual privilege distillation in high-resolution remote sensing VQA.
Critical weaknesses
  1. Input distribution discrepancy between training and inference. According to Appendix C, the Student receives full images annotated with red bounding boxes marking target regions during training, whereas such annotations are typically absent during inference. This raises concerns that the Student may rely on explicit localization cues during training rather than learning autonomous visual evidence discovery, creating a misalignment with the paper’s stated objective.
  2. Incomplete computational cost analysis. The paper primarily reports inference latency but lacks a comparison of training GPU-hours, memory footprint, and overall training overhead against standard SFT. While the current results support claims of inference efficiency, they are insufficient to demonstrate overall computational efficiency or cost-effectiveness.
  3. The effectiveness of RS-OPD-Lite is not fully disentangled from the use of a larger teacher model. While the main RS-OPSD framework employs self-distillation with an 8B model, RS-OPD-Lite relies on an 8B teacher to train a 2B student. It remains unclear whether the proposed privileged distillation strategy is equally effective in a 2B self-distillation setting.
  4. Although the ablation study demonstrates the overall effectiveness of CPVP, the individual contributions of global, contextual, and fine-grained evidence remain unclear. A more detailed analysis would help explain whether all three views are necessary.
  5. CAD combines sample-level teacher reliability gating with token-level correctness alignment. However, the ablation study only evaluates CAD as a whole, making it difficult to determine the individual contribution of each component.
Questions for authors

I would be willing to raise my score if the authors satisfactorily address the above concerns.

Reviewer_Y8gPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces RS-OPSD, which uses training-time zoom-in visual privilege for OPSD in UHR remote sensing VQA. Inference with the trained model requires no additional visual search or tool calls. The authors construct GeoEvidence-6K with evidence-region annotations and introduce HF-SR to improve the annotation process. CPVP provides the teacher with Global Evidence, Contextual Evidence, and Fine-grained Evidence. CAD uses GT answers to assess teacher reliability and filters distillation signals according to student-answer correctness. Experiments report improvements on three benchmarks, while RS-OPD-Lite demonstrates the framework's application to a smaller student model.

Critical strengths
  1. RS-OPSD outperforms the compared methods on all three benchmarks while requiring no additional visual search or tool calls at inference time, giving it practical value. Its improvements over LoRA fine-tuning on the same data also support the effectiveness of the training framework.

  2. GeoEvidence-6K pairs manually verified questions and answers with explicit evidence-region annotations, providing a reusable data resource for studying privileged distillation in UHR remote sensing VQA.

Critical weaknesses
  1. Vision-OPD (Yuan et al., 2026), which already studies on-policy self-distillation with training-only visual privilege; RS-OPSD appears to adapt this principle to remote sensing and add multi-view inputs and correctness gates.

  2. Could the authors clarify how the Naïve OPSD setup differs from Vision-OPD in teacher updates, distillation objective, and the tokens used for distillation, and identify which parts of CPVP and CAD constitute the specific methodological advance over Vision-OPD?

  3. Table 3 evaluates CAD only as a combined module, so it does not distinguish the contribution of student-answer correctness feedback from that of the teacher reliability gate.

  4. Table 6 appears to misreport the WeaveEarth subcategory results. The original paper reports Color/Count/Position as 51.39/27.65/62.60, which corresponds to Position/Color/Count values of 62.60/51.39/27.65, rather than 51.4/27.7/62.6.

Questions for authors

Please see Weaknesses

1 comment
AuthorsOpenReview ↗

Follow-up on Rebuttal Discussion

Dear Reviewer Y8gP,

We hope you are doing well. Since the discussion period is scheduled to close tomorrow, we would like to kindly follow up on our rebuttal responses.

We would be very happy to address any further questions or suggestions you may have and engage in additional discussion. We will do our best to respond promptly and thoroughly before the discussion period concludes.

Thank you again for your time and consideration.

Best regards,
The Authors of Submission 15361

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Progress and Reviewer Feedback

We would like to summarize our rebuttal progress and the reviewers' feedback, particularly the additional experiments and clarifications provided.

1. Positive Feedback from the Reviewers

We are encouraged that the reviewers recognized several contributions of our work:

  • Practical motivation and inference efficiency: Reviewers Y8gP and a7hx acknowledged the value of addressing fine-grained evidence localization without additional inference-time search or tool calls.
  • Methodological design: Reviewers a7hx and LDyK recognized that CPVP and CAD address limitations identified in our preliminary experiments, supported by targeted ablations.
  • Dataset contribution: All three reviewers recognized GeoEvidence-6K and its human-verified evidence annotations as a valuable research resource.
  • Empirical results: Reviewer LDyK also acknowledged the favorable accuracy-efficiency trade-off and the potential of the Lite variant.

Importantly, both Reviewer a7hx and Reviewer LDyK explicitly acknowledged our rebuttal efforts and increased their ratings. Reviewer a7hx raised the score from 3 (Weak Acceptance) to 4 (Clear Acceptance), noting that our responses addressed most concerns. Reviewer LDyK raised the score from 2 (Weak Rejection) to 3 (Weak Acceptance), recognizing the additional SFT, OOD, and frontier-model evaluations.

2. Addressing Reviewer Y8gP's Concerns

Reviewer Y8gP has not yet provided a follow-up response to our rebuttal. Nevertheless, we have made substantial efforts to address all four concerns raised in the review. In light of the positive feedback from Reviewers a7hx and LDyK, we believe that our additional explanations and experiments have also addressed several key concerns raised by Reviewer Y8gP, particularly those overlapping with the other reviewers' comments.

  • Relationship to Vision-OPD and methodological novelty (W1–W2): We addressed this concern through a systematic analysis of direct adaptation, domain-specific challenges, methodological innovations, and supporting ablations. First, we showed that Naïve OPSD, which adapts Vision-OPD to UHR remote-sensing VQA, achieves 45.2, slightly below LoRA fine-tuning at 45.7, indicating that directly transferring the existing principle is insufficient in our setting. Second, we analyzed the differences between natural-image HR VQA and UHR remote-sensing VQA, particularly the need to integrate fine-grained evidence with surrounding context and global spatial relationships. Our diagnostic further showed that crop-only privilege provides a limited improvement of 6.06 points, with the crop-conditioned teacher still making errors on approximately 30% of examples. Motivated by these limitations, we introduced CPVP to construct context-preserving multi-view visual privilege and CAD to transfer teacher preferences under sample-level reliability and token-level correctness constraints. Their combination improves the average from 45.2 to 49.3. These explanations and experiments directly address the methodological novelty concerns also raised by Reviewer LDyK.

  • Individual contributions of CAD (W3): This concern directly overlaps with Reviewer a7hx's W5. We added separate ablations showing that reliability gating and correctness alignment individually achieve 46.1 and 46.6, respectively, while their combination reaches 47.0. These results demonstrate their complementary contributions. Reviewer a7hx explicitly acknowledged that our responses had addressed most of their concerns.

  • WeaveEarth reporting error (W4): We acknowledged and corrected the column-ordering error in Table 6, reporting Position/Color/Count as 62.60/51.39/27.65, respectively. The corrected values are marked in blue in the revised manuscript.

3. Additional Rebuttal Progress

We also addressed the remaining concerns raised by Reviewers a7hx and LDyK through additional experiments, including unboxed student training, training-cost profiling, 2B self-distillation, full-parameter SFT comparison, OOD evaluation, qualitative case studies, and expanded frontier-model comparisons across all three benchmarks. Both reviewers positively acknowledged these additions and increased their ratings.

In particular, following Reviewer LDyK's further suggestions, we added qualitative comparisons between the base model and RS-OPSD on identical full-image inputs, illustrating how the student utilizes question-relevant visual evidence after privileged distillation. We also completed additional frontier-model evaluations across all three benchmarks.

4. Concluding Remarks

We have made substantial efforts to strengthen the paper through additional experiments, quantitative analyses, qualitative case studies, and manuscript revisions. We hope that these additions, together with the positive feedback from the other reviewers, will help clarify our responses to Reviewer Y8gP's comments and provide further support for the contributions of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.