acceptodds
Under review as a conference paper at ICLR 2027

TraceEarth: From Semantic Matching to Spatial Tracing for Training-Free UHR Remote Sensing VQA

Abstract

Ultra-high-resolution (UHR) remote sensing visual question answering (VQA) requires capturing global context and local details within a limited visual token budget, making efficient acquisition of question-relevant local evidence critical.Existing methods improve UHR perception through visual token compression or selection, question-guided region retrieval, or iterative zooming and search. However, these methods mainly determine what visual content to retain or inspect based on semantic relevance or visual importance, while explicit spatial cues in the question are rarely converted into direct constraints on the search space. This can lead to redundant visual processing, localization errors, and missed critical evidence. We argue that remote sensing questions should not only identify ”which regions are relevant,” but directly indicate ”where the target is.” Converting spatial cues into executable visual search priors shifts broad semantic search toward efficient, precise target localization. We therefore propose TraceEarth, a training-free framework for spatially guided visual evidence acquisition in UHR remote sensing VQA. TraceEarth decouples spatial cues from full-question semantics, shifting visual evidence acquisition from semantic matching to spatial tracing. Building on this framework, we further propose a spatial cue guided evidence tracing method that combines image-referenced localization, object-referenced tracing, relation-preserving evidence construction, and candidate verification using frozen models and existing visual tools, progressively acquiring question-relevant evidence from global scenes to local targets. TraceEarth achieves 60.03% and 61.10% accuracy on the UHR benchmarks MME-RealWorld-RS and XLRS-Bench Lite, respectively, and 72.61% on the remote sensing benchmark LHRS-Bench. Consistent gains across base vision-language models demonstrate its effectiveness and generalizability without additional training. Our code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Remote Sensing Visual Question Answering, Spatial Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VqdCOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a progressive approach to ultra-high-resolution (UHR) remote sensing visual question answering by decoupling linguistic information at different granularities. Specifically, it analyzes the structure of question descriptions and develops a coarse-to-fine visual search and localization process based on the extracted linguistic cues. Although the motivation is reasonable, several concerns remain regarding the methodological novelty, benchmark dependency, and experimental justification.

Critical strengths
  1. The paper presents a reasonable approach to decoupling linguistic information at different granularities, shifting from conventional uniform region partitioning and subsequent selection toward a text-guided, coarse-to-fine visual search process.
  2. The paper provides a useful analysis of the prominent question construction patterns in UHR remote sensing VQA benchmarks, highlighting the prevalence of explicit spatial cues and their potential value for visual evidence localization.
Critical weaknesses
  1. Exploiting target descriptions, reference objects, and spatial relationships for visual localization is already a well-established practice in remote sensing vision-language research. For example, Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation (IEEE TGRS 2025) explicitly decouples object descriptions from spatial position information for fine-grained visual grounding. More notably, ProVG: Progressive Visual Grounding via Language Decoupling for Remote Sensing Imagery (ACM MM 2026) introduces a progressive localization framework based on decoupled linguistic cues. Similarly, GeoMag: A Vision-Language Model for Pixel-level Fine-Grained Remote Sensing Image Parsing (ACM MM 2025) explores prompt-guided region cropping and multi-granularity visual processing. While these studies differ in their specific tasks and implementations, they have already explored several key principles underlying the proposed approach. Therefore, the authors should clarify whether TraceEarth introduces any fundamentally new mechanisms beyond existing language-guided region retrieval, relational grounding, and progressive visual search methods.
  2. Benchmark-driven design rather than a fundamental paradigm shift: The proposed framework appears more closely aligned with a benchmark-driven engineering strategy than a fundamentally new transition from semantic matching to spatial tracing.
Questions for authors
  1. How does the proposed method perform when spatial cues in the questions are removed, paraphrased, or expressed with uncertainty? Can the framework robustly handle diverse and implicit spatial descriptions, or is its effectiveness largely restricted to the explicit spatial prompts and question patterns represented in the evaluated benchmarks?
6 comments

Dear Reviewer VqdC,

Thank you for your thorough review and feedback on our manuscript. We truly appreciate the time and effort you dedicated to understanding our work, and your constructive comments. Below, we respond to each of the concerns you raised. We hope this addresses your concerns and helps further improve your evaluation of our work.

[Weaknesses 1 : Distinction from Prior Spatial Grounding Methods]

We thank the reviewer for highlighting these related studies, and we fully acknowledge their contributions. While, we would like to clarify that TraceEarth differs fundamentally from these works in both its motivation and technical mechanisms. While FIANet, ProVG, and GeoMag primarily leverage linguistic and spatial cues to guide image-text alignment, progressive visual grounding, or region-level perception, TraceEarth transforms question-derived spatial cues and their dependencies into executable constraints that directly control visual evidence acquisition in UHR remote sensing VQA. Specifically, the roles of spatial cues determine which visual operations are performed, while their dependencies determine the execution order across image space and object space. This shifts local visual evidence acquisition from semantic matching to spatial tracing, rather than merely improving target localization or region selection. The key mechanisms are detailed below.

  • Answer-Role-Aware Spatial Execution. TraceEarth determines the execution strategy according to the role of spatial information in the question and candidate answers. Spatial Intent Routing (SIR) distinguishes known localization constraints, candidate image positions requiring verification, and unknown object relations requiring prediction. For example, “the building to the right of the field” provides a known localization constraint, whereas “Is the building to the left or right of the field?” requires inferring the relation itself. Treating the latter as a known search constraint would implicitly assume the answer. Thus, SIR distinguishes which spatial information can guide evidence acquisition and which must instead be inferred from visual evidence, enabling different execution paths for different VQA prediction targets.

  • Answer-Conditioned Evidence Construction and Verification. TraceEarth further translates these execution decisions into distinct evidence acquisition and verification mechanisms. For image-position questions, candidate locations are converted into independent local hypotheses, each evaluated through a Yes/No verification query using the frozen VLM. When the highest verification scores are insufficiently separated, the framework falls back to full-image inference. For object-relation questions, TraceEarth instead jointly localizes the relevant entities and preserves their relative spatial layout, allowing the unknown relation to be inferred from visual evidence. These mechanisms directly connect spatial tracing to the prediction target of VQA, rather than treating region localization as the final objective.

  • Spatial Disambiguation through Executable Tracing. Spatial cues are highly prevalent in UHR remote sensing VQA. As shown in Section 3 and Figure 2, explicit spatial cues appear in 95.26% of MME-RealWorld-RS and 91.17% of XLRS-Bench Lite questions. Repeated similar objects make semantic descriptions insufficient for precise localization. TraceEarth converts image-level directions and object-relative relations into executable spatial constraints, progressively narrowing the search space and reducing ambiguity among similar target instances. As illustrated in Figure 3, image-space localization and reference-object-guided tracing follow the spatial dependencies expressed in the question, rather than relying on generic coarse-to-fine visual search.

In summary, TraceEarth introduces an answer-role-aware, dependency-driven spatial execution framework that transforms question-derived spatial information into executable tracing and answer-conditioned evidence verification. Its methodological distinction lies in determining how spatial information should be used, constructing evidence according to the prediction target, and exploiting spatial dependencies to reduce localization ambiguity in UHR imagery. We will further clarify these distinctions from FIANet, ProVG, and GeoMag in Section 2.2 of the revised manuscript.

[Weakness 2: Beyond Benchmark-Driven Design — From Semantic Matching to Spatial Tracing]

We thank the reviewer for this important comment. We would like to clarify that TraceEarth is not designed around benchmark-specific question patterns, but proposes a distinct visual evidence acquisition strategy that addresses the intrinsic challenges of UHR remote sensing VQA and can be integrated into compatible existing visual search methods in a plug-and-play manner. As clarified in our response to Weakness 1, the transition from semantic matching to spatial tracing lies in using question-derived spatial roles and dependencies to directly govern how visual evidence is acquired and verified in UHR remote sensing VQA, rather than merely introducing another region-cropping heuristic.

  • From Semantic Matching to Spatial Tracing. Existing training-free UHR visual search methods commonly determine which regions to inspect through semantic relevance, visual feedback, or model-generated search decisions. While effective, these approaches do not necessarily translate the explicit spatial dependencies in a question into direct execution constraints. In contrast, TraceEarth shifts evidence acquisition from identifying semantically relevant regions to tracing where the target should be located. By converting image-level directions and object-relative relations into executable spatial constraints, TraceEarth determines the visual search path according to the spatial dependencies expressed in the question. This represents a substantive methodological distinction in evidence acquisition, rather than merely another implementation of region selection.

  • Motivation from UHR Remote Sensing VQA Rather Than Benchmark-Specific Heuristics. UHR remote sensing imagery presents inherent challenges, including extensive spatial coverage, numerous visually similar object instances, and limited visual token budgets. Semantic descriptions alone may therefore be insufficient to uniquely identify the target, whereas spatial constraints can substantially reduce localization ambiguity. As shown in Section 3 and Figure 2, explicit spatial cues occur in 95.26% (3,561/3,738) of MME-RealWorld-RS questions and 91.17% (2,808/3,080) of XLRS-Bench Lite questions. Importantly, TraceEarth interprets spatial information from the semantics of questions and answer candidates rather than relying on benchmark-specific keywords or fixed templates. As demonstrated by the evaluations on MME-RealWorld-RS, XLRS-Bench Lite, and LHRS-Bench, the framework achieves consistent improvements across different remote sensing VQA benchmarks, supporting its applicability beyond a single benchmark.

  • Preserving Baseline Accuracy Without Explicit Spatial Cues. Importantly, TraceEarth does not require every question to contain explicit directional descriptions or executable spatial constraints. When no executable spatial cue is identified, the framework falls back to the original image and vanilla VLM inference rather than forcing an inappropriate tracing operation. We further evaluate the 177 naturally occurring MME-RealWorld-RS questions without explicit spatial cues, obtaining the following results:

    Subset# QuestionsVanilla Qwen3-VL-8B (%)TraceEarth (%)Difference (pp)
    Without explicit spatial cues17755.3755.370.00

    Both methods achieve exactly the same 55.37% accuracy, demonstrating that TraceEarth preserves the baseline's aggregate performance on questions without explicit spatial guidance. This further shows that spatial tracing is applied when appropriate, rather than imposed on every question.

  • Plug-and-Play Compatibility with Existing Visual Search Methods. Furthermore, TraceEarth is not restricted to a standalone inference pipeline and can serve as a complementary module for existing visual search frameworks. As already demonstrated in Section 5.7 and Table 5, integrating TraceEarth into RAP improves average accuracy from 54.21% to 56.75% (+2.54 pp) while reducing average inference time from 96.57 s to 70.59 s (−25.98 s, approximately 26.9%). These results demonstrate that spatial tracing can complement an existing visual search strategy and improve its accuracy–efficiency trade-off, rather than functioning only as a benchmark-specific solution.

Overall, “from semantic matching to spatial tracing” represents a substantive change in visual evidence acquisition: from primarily evaluating region relevance to directly executing question-derived spatial constraints. This design addresses the intrinsic localization challenges of UHR remote sensing VQA, preserves baseline accuracy when explicit spatial cues are absent, and provides complementary benefits when integrated into existing visual search frameworks. We have clarified this methodological motivation and the scope of our contribution in the revised manuscript.

[Question 1: How does the proposed method perform when spatial cues in the questions are removed, paraphrased, or expressed with uncertainty? Can the framework robustly handle diverse and implicit spatial descriptions, or is its effectiveness largely restricted to the explicit spatial prompts and question patterns represented in the evaluated benchmarks?]

We thank the reviewer for raising this important question regarding the robustness and general applicability of TraceEarth. To address this concern, we conduct an additional paired evaluation of paraphrased, implicit, and uncertain spatial expressions, examine questions without explicit spatial references, and further clarify the semantic routing, conditional execution, and plug-and-play properties of our framework.

  • Spatial Cues as Constraints for Target Disambiguation. UHR remote sensing scenes often contain numerous visually similar object instances, making semantic descriptions insufficient to uniquely identify the intended target. Spatial cues help resolve this ambiguity by specifying image-level locations or relationships to reference objects, providing actionable guidance for visual evidence acquisition. For example, “the building to the right of the smaller football field near the image center” combines image-space and object-space constraints to identify a particular instance. Our analysis identifies spatial references in 3,561/3,738 questions (95.26%) on MME-RealWorld-RS and 2,808/3,080 questions (91.17%) on XLRS-Bench Lite, highlighting their prevalence in the evaluated benchmarks. While these proportions may partly reflect benchmark construction, they motivate spatially guided evidence acquisition. The following experiments further examine whether TraceEarth can interpret such constraints beyond their original wording.

  • Robustness to Diverse Spatial Expressions. To directly evaluate sensitivity to spatial wording, we conduct a controlled paired perturbation study on MME-RealWorld-RS using 200 original questions containing spatial cues. Each question is reformulated into three variants, resulting in 200 matched pairs per perturbation type:

    1. Paraphrased expressions: Rewording spatial descriptions while preserving their meaning, such as replacing “bottom right corner of the picture” with “the picture's lower-right corner”.
    2. Implicit expressions: Conveying spatial constraints indirectly, such as describing the lower-right corner as “the corner where the bottom and right edges of the image meet”, or expressing “the road is above the water” as “the water is below the road”.
    3. Uncertain expressions: Retaining the spatial direction while introducing qualifiers such as “roughly” or “approximately”.

    The evaluation comprises 800 inputs (200 original questions and 600 rewritten variants), all evaluated using the same TraceEarth framework and frozen Qwen3-VL-8B backbone. Each variant is compared against its corresponding original question under identical inference settings.

    Spatial ExpressionMatched PairsAccuracy (%)Δ vs. Original (pp)
    Original20064.50–
    Paraphrased20065.00+0.50
    Implicit20064.00−0.50
    Uncertain20063.50−1.00

    Relative to the original accuracy of 64.50%, the three perturbations yield changes of only +0.50, −0.50, and −1.00 percentage points, respectively. These small observed differences provide direct empirical evidence that TraceEarth can accommodate alternative spatial formulations, including indirect descriptions and uncertainty-qualified expressions, rather than relying exclusively on the original benchmark wording.

  • Semantic Spatial Routing Beyond Fixed Question Templates. The observed results are consistent with TraceEarth's design. Rather than relying on predefined directional keywords or fixed question-to-crop mappings, Spatial Intent Routing (SIR) uses the frozen VLM to interpret the semantic roles and dependencies of spatial information in both questions and candidate answers. Specifically, it distinguishes known spatial constraints used for localization from unknown positions or spatial relations that must be predicted, and selects the corresponding tracing or verification strategy. Image-Space Tracing (IST) handles image-referenced constraints, including natural-language directions, bounding boxes, and visual markers, while Object-Space Tracing (OST) handles reference entities and object-level spatial relationships. This semantic, task-aware design provides a mechanism for interpreting equivalent spatial meanings expressed in different forms, consistent with the small accuracy changes observed in our paired evaluation.

Reviewer_P9ruOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TraceEarth addresses the problem of how to find key local evidence in ultra-high-resolution (UHR) remote sensing visual question answering (VQA). The main idea is to extract spatial descriptions from questions and turn them into executable operations, including image cropping, reference object localization, and candidate region verification. This avoids relying only on semantic matching with the full question. Using the same Qwen3-VL-8B backbone, TraceEarth improves accuracy on MME-RealWorld-RS from 50.10% to 60.03%, a gain of 9.93 percentage points. However, the method relies on spatial cues in questions and additional localization models.

Critical strengths
  1. The paper shows that UHR VQA benchmarks rely heavily on explicit spatial cues. It also distinguishes between image-based references and object-based references.
  2. The paper develops an evidence acquisition pipeline that selects actions based on the roles of spatial information. The way it connects spatial information with visual operations is relatively novel in UHR remote sensing.
Critical weaknesses
  1. The main motivation rests on the observation that most questions in two UHR VQA benchmarks contain explicit spatial cues. This prevalence may largely reflect how these benchmarks were constructed, whereas real user queries often lack such precise location descriptions. The paper does not report results on questions without spatial cues, so the general applicability of the method remains unclear.
  2. The main tables compare against previously reported results of other methods under their original evaluation settings, which differ in prompts, answer parsing, and averaging. Such numbers are not directly comparable. Moreover, the experiments on XLRS and LHRS lack a native Qwen3-VL-8B baseline, so the gains on these benchmarks cannot be attributed to the method itself.
  3. The method relies on an additional segmentation model (RemoteSAM) for reference localization, whereas most compared methods use only the VLM. The comparison is therefore not conducted under equal model resources, and the efficiency comparison is reported only on a different backbone rather than the main Qwen3-VL-8B setting.
  4. The authors claim that the proposed method reduces irrelevant searches and prevents key evidence from being missed. However, the paper does not directly report target retention rates, localization success rates, error rates for different routing strategies, or recovery performance after failures. These metrics are closely related to the proposed mechanism.
  5. The appendix does not provide the complete prompts, and the implementation details of several key steps remain unclear.
Questions for authors
  1. Could the authors report the native Qwen3-VL-8B baseline on XLRS and LHRS, and re-evaluate the main baselines under a unified protocol?
  2. How does TraceEarth perform on questions without explicit spatial cues, and what happens when the spatial parsing or RemoteSAM localization fails?
  3. Could the authors provide the complete prompts, the key execution rules of the system, and a detailed analysis of failure cases?
  4. What is the inference cost of TraceEarth with Qwen3-VL-8B, including the RemoteSAM calls, compared with VLM-only methods under the same hardware?
7 comments

Dear Reviewer P9ru,

Thank you for your thorough review and feedback on our manuscript. We truly appreciate the time and effort you dedicated to understanding our work, and your constructive comments. Below, we respond to each of the concerns you raised. We hope this addresses your concerns and helps further improve your evaluation of our work.

[Weakness 1 & Question 2: Regarding the General Applicability of Spatial Tracing and Its Failure Handling Mechanisms]

We thank the reviewer for raising this important concern regarding the applicability and failure-handling mechanisms of TraceEarth. We further clarify its conditional execution, performance on questions with and without spatial references, and compatibility with existing visual search frameworks.

  • Applicability to Questions With and Without Spatial References. Spatial references help distinguish visually similar objects in UHR remote sensing imagery. Our LLM-based semantic annotation of question texts and candidate answers identifies spatial references in 3,561/3,738 questions (95.26%) on MME-RealWorld-RS and 2,808/3,080 questions (91.17%) on XLRS-Bench Lite. Importantly, TraceEarth does not require every question to contain actionable spatial constraints. When no executable spatial constraint is identified, the framework retains the original full-image VQA inference path.

    We further compare the performance of vanilla Qwen3-VL-8B and TraceEarth on MME-RealWorld-RS subsets with and without detected explicit spatial references:

    Question Subset# QuestionsVanilla (%)TraceEarth (%)Δ (pp)
    With spatial references3,56149.8460.26+10.42
    Without spatial references17755.3755.370.00
    Overall3,73850.1060.03+9.93

    TraceEarth improves accuracy by approximately 10.42 percentage points on questions containing spatial references, while maintaining the vanilla backbone's 55.37% accuracy on the 177 questions without detected explicit spatial references. These results indicate that TraceEarth can exploit spatial guidance when available without degrading aggregate accuracy on the evaluated subset where explicit spatial references are absent.

  • Failure Handling and Fallback Behavior. TraceEarth incorporates three fallback mechanisms: (1) invalid spatial parsing leads to full-image VQA inference; (2) unsuccessful RemoteSAM localization retains the IST-generated crop when available, or the original image otherwise; and (3) uncertain image-position verification triggers full-image inference when the gap between the top two candidate scores falls below τ = 0.03.

    We further quantify the candidate-verification fallback on MME-RealWorld-RS:

    Position QuestionsFallback CasesFallback RateAccuracy on Fallback Cases
    1,25736028.64%55.00% (198/360)

    These results show that the uncertainty-triggered fallback is actively employed during evaluation, allowing inference to continue when candidate-level evidence is inconclusive, although it cannot resolve every verification error.

  • Plug-and-Play Compatibility. As already demonstrated in Section 5.7 and Table 5 of our original submission, TraceEarth can be integrated into RAP as a complementary spatial guidance module. This integration improves accuracy from 54.21% to 56.75% (+2.54 pp) while reducing average inference time from 96.57 s to 70.59 s (−25.98 s). These results provide additional evidence that spatial tracing can complement existing visual search frameworks without replacing their original search and reasoning mechanisms.

Overall, TraceEarth exploits spatial constraints when available while retaining the original VQA pathway when actionable constraints are absent or tracing cannot proceed. The subset-level comparisons, fallback statistics, and RAP integration results provide complementary evidence of its conditional applicability and compatibility with existing visual search methods. We have further clarified these mechanisms and their supporting experimental results in the revised manuscript.

[Weakness 2 & Question 1: Regarding Backbone-Controlled Evaluation and Fair Comparisons Across Benchmarks]

We thank the reviewer for highlighting the importance of backbone-controlled evaluation and fair comparisons across benchmarks. To address these concerns, we supplement the native Qwen3-VL-8B results on XLRS-Bench Lite and LHRS-Bench, provide additional comparisons with representative methods using the same answering backbone, and reproduce several external baselines on LHRS-Bench.

  • Backbone-Controlled and Backbone-Aligned Comparisons Across Benchmarks. We evaluate vanilla Qwen3-VL-8B and TraceEarth under the same evaluation protocol on MME-RealWorld-RS, XLRS-Bench Lite, and LHRS-Bench. We further include ZoomSearch and WeaveEarth with the Qwen3-VL-8B answering backbone to provide broader cross-method comparisons.

    MethodMME-RS (%)XLRS-Lite (%)LHRS (%)
    Vanilla Qwen3-VL-8B50.1046.3668.84
    ZoomSearch (Qwen3-VL-8B)44.7845.1364.04
    WeaveEarth (Qwen3-VL-8B)47.3847.1470.87
    TraceEarth (Ours)60.0361.1072.61
    Gain over Vanilla (pp)+9.93+14.74+3.77

    With the underlying Qwen3-VL-8B backbone held fixed, TraceEarth improves over vanilla inference by 9.93, 14.74, and 3.77 percentage points, respectively. These matched comparisons demonstrate that the improvements arise from incorporating TraceEarth rather than merely adopting a stronger backbone.

  • Reproduced Evaluations and Baseline Transparency. To strengthen evaluation fairness, we additionally reproduce GeoEyes, GeoVista, ZoomSearch, and WeaveEarth on LHRS-Bench. GeoEyes and GeoVista are evaluated using their officially released trained checkpoints, achieving 67.68% and 58.12%, respectively. ZoomSearch is evaluated with Qwen3-VL-8B and achieves 64.04%, while our reproduction of WeaveEarth achieves 70.87%, compared with 72.61% for TraceEarth. Thus, TraceEarth outperforms the reproduced WeaveEarth and ZoomSearch baselines by 1.74 pp and 8.57 pp, respectively.

  • Evaluation Fairness and Controlled Experimental Evidence. We recognize that cross-paper comparisons may involve differences in prompts, image preprocessing, answer parsing, and other evaluation configurations. Therefore, we distinguish strictly matched vanilla-backbone comparisons, backbone-aligned cross-method comparisons, and evaluations using independently trained checkpoints. As additional controlled evidence, Section 5.7 and Table 5 report a plug-and-play evaluation in which RAP and RAP + TraceEarth use identical question samples, the same LLaVA-OV-7B backbone, and matched experimental settings. Incorporating TraceEarth improves accuracy from 54.21% to 56.75% (+2.54 pp), with the gain calculated exclusively from these paired experiments.

Overall, the backbone-controlled results establish consistent improvements across all three benchmarks, while the reproduced LHRS-Bench evaluations and matched RAP integration experiment provide complementary evidence under clearly specified conditions. We have updated the relevant comparisons in the revised manuscript and marked independently reproduced results with † to distinguish them from literature-reported baselines. We have highlighted the corresponding revisions in yellow in the revised manuscript.

[Weakness 3 & Question 4: Regarding Additional Model Resources and End-to-End Inference Efficiency]

We thank the reviewer for raising this important concern regarding additional model resources and inference efficiency. We provide controlled comparisons to examine the contribution of RemoteSAM, report end-to-end inference costs using our main Qwen3-VL-8B backbone under identical hardware, and further clarify the accuracy–efficiency trade-off of TraceEarth.

  • Additional Model Resources and Their Contribution. TraceEarth employs RemoteSAM as an additional reference-object localization model, while keeping the underlying VLM frozen. To investigate whether the improvements arise simply from introducing RemoteSAM, we conduct a RemoteSAM-only object-cropping control without TraceEarth's spatial intent routing and cross-space tracing.

    MethodColor (%)Count (%)Position (%)Overall (%)
    Vanilla Qwen3-VL-8B59.7621.8568.0150.10
    RemoteSAM-only Crop57.1315.4252.1941.79
    TraceEarth (Ours)69.3232.3877.7260.03

    RemoteSAM-only cropping performs below the vanilla backbone on all three subtasks, whereas TraceEarth consistently improves upon it. These results demonstrate that the additional localization model and simple object-centric cropping alone are insufficient to explain TraceEarth's performance gains, highlighting the contribution of spatially guided evidence acquisition.

  • End-to-End Inference Cost under Identical Hardware. We additionally evaluate the inference efficiency of TraceEarth and VLM-only methods using the Qwen3-VL-8B-Instruct backbone on a single NVIDIA RTX A6000 GPU. TraceEarth's end-to-end latency includes RemoteSAM localization, spatial intent routing, image-space/object-space tracing, candidate verification when applicable, and final answer prediction.

    MethodAdditional ModelAccuracy (%)Time (s/question)
    Vanilla Qwen3-VL-8BNone50.103.2
    Generic Agentic Cropping (VLM-only)None53.2112.5
    TraceEarth (Ours)RemoteSAM60.0329.1

    Compared with vanilla Qwen3-VL-8B, TraceEarth incurs approximately 9.1× inference latency (3.2 s vs. 29.1 s), while improving MME-RealWorld-RS accuracy from 50.10% to 60.03% (+9.93 pp). These results explicitly quantify the additional computational cost and corresponding accuracy gains. Importantly, our efficiency claim concerns multi-stage visual evidence acquisition rather than latency superiority over single-pass VLM inference.

  • Efficiency Relative to Multi-Stage Visual Search Methods. As reported in Section 5.6 and Figure 4, we compare TraceEarth with representative multi-stage visual search methods using the same LLaVA-OV-7B backbone. TraceEarth achieves 58.05% accuracy with 28.24 s average inference time, compared with RAP (54.21%, 96.57 s), ZoomSearch (45.31%, 72.63 s), and ZoomEye (44.87%, 103.85 s). TraceEarth achieves both the highest accuracy and the lowest latency among these evaluated methods, demonstrating the efficiency benefits of spatially guided evidence acquisition relative to the compared multi-stage search frameworks.

  • Plug-and-Play Accuracy and Efficiency Improvements. Furthermore, Section 5.7 and Table 5 demonstrate that TraceEarth can improve an existing visual search framework. Under the same LLaVA-OV-7B backbone and evaluation settings, integrating TraceEarth into RAP improves accuracy from 54.21% to 56.75% (+2.54 pp) while reducing average inference time from 96.57 s to 70.59 s (−25.98 s, approximately 26.9%). These results demonstrate that spatial tracing can complement existing visual search strategies and improve their overall accuracy–efficiency trade-off.

Overall, the RemoteSAM-only control shows that the additional localization model alone cannot explain TraceEarth's improvements, while the end-to-end latency measurements explicitly quantify its computational overhead. Together with the existing efficiency and plug-and-play evaluations, these results demonstrate that TraceEarth trades additional computation for substantial accuracy gains over vanilla VLM inference while providing efficiency benefits relative to the evaluated multi-stage visual search methods. We will incorporate the new controlled results and latency measurements into the revised manuscript and clarify the corresponding efficiency claims.

Reviewer_31uWOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses ultra-high-resolution (UHR) remote sensing VQA, where the evidence needed to answer a question occupies a small part of a very large image while the visual token budget is limited. The authors first observe that most questions in two UHR RS benchmarks contain explicit spatial cues, and argue that such cues should not be folded into a single semantic query but decoupled from question semantics and used as executable constraints on the visual search space. They propose TraceEarth, a training-free, prompt-driven pipeline over a frozen VLM with four ingredients:

  1. Spatial Intent Routing (SIR), which decides whether the spatial information is the prediction target (image position / object relation), a known localization condition, or a relation to be inferred, and selects the corresponding tracing path;
  2. Image-Space Tracing (IST), which converts image-referenced cues (natural-language directions, bounding boxes, visual markers) into crops via a small skill library (3×3 grid crop, bbox crop, marker crop);
  3. Object-Space Tracing (OST), which localizes a reference entity with RemoteSAM and expands the region along a known relation, or, when the relation itself must be predicted, builds a relation-preserving crop enclosing the relevant entities;
  4. Candidate-conditioned verification for position questions, where each candidate answer is normalized to an image region and turned into a local yes/no question whose Yes/No logit gap scores the candidate, with fallback to the full image when the top-two gap is below τ = 0.03.

Validation uses Qwen3-VL-8B as the main backbone on MME-RealWorld-RS, XLRS-Bench Lite and LHRS-Bench, plus Qwen3-VL-4B, LLaVA-OV-7B and Qwen3-VL-32B for backbone generalization; an ablation of the three stages, an accuracy–latency comparison, a plug-and-play integration into RAP, and a qualitative case study. Reported overall accuracies are 60.03%, 61.10% and 72.61% respectively.

Critical strengths

S1. A simple, well-motivated recipe with large and consistent gains at zero training cost. Over the same frozen backbone, TraceEarth improves MME-RealWorld-RS average accuracy by +9.93 (Qwen3-VL-8B, 50.10 → 60.03), +10.43 (LLaVA-OV-7B), +7.97 (Qwen3-VL-4B) and +7.94 (Qwen3-VL-32B), with the same sign on all three subsets. A ~8–10 point gain from an inference-time wrapper on an already strong 8B model, replicated across two model families and four scales, is unusual and hard to attribute to a lucky backbone. This matters because the practical message — an off-the-shelf RS VLM can be materially improved without new annotations or fine-tuning — is the paper's most valuable and best-supported claim.

S2. The image-space / object-space and target-vs-constraint decomposition is conceptually clean and reusable. It pinpoints a real failure mode of semantic-retrieval and token-selection methods in UHR RS (repeated same-category instances make category semantics non-identifying), and it yields a decision procedure that is independent of the specific backbone. The cumulative ablation (50.10 → 53.91 → 56.66 → 60.03) is consistent with the three stages contributing complementary constraints, and the plug-and-play result on RAP (+2.54 average accuracy, −25.98 s) indicates the components are modular rather than a single tuned monolith.

Critical weaknesses

W1. The gains are not attributed to spatial tracing, because the obvious alternative explanations are uncontrolled. TraceEarth adds two things at once: (a) substantially more inference compute (a routing call, parsing calls, one verification call per candidate, and a final answering call per question), and (b) zoomed-in crops centred on the referenced object or candidate region. Either factor alone can produce a large jump on a VQA benchmark. Three controls are missing and, in my view, are required for the paper's thesis:

  • a generic agentic-crop baseline with the same backbone and a comparable tool-call budget, in which the VLM itself decides where to crop (V-style, or "reason about where the target is, then crop"), with no image-space/object-space decoupling;
  • an object-centric crop baseline that simply runs RemoteSAM on the queried entity and crops its box at the same zoom factor, with no cue parsing and no routing;
  • a matched-compute baseline, e.g. self-consistency over k samples or long chain-of-thought prompting with k× tokens.

As written, the only comparison against the unmodified base model is Table 3, which shows that something in the pipeline helps but cannot distinguish the mechanism from generic zooming plus test-time compute. This is the single most consequential issue, because it decides whether the contribution is a mechanism ("spatial tracing") or a well-engineered zoom heuristic.

W2. The reported evidence is selectively favourable and in places internally inconsistent, which weakens the "state-of-the-art" and "compatible" claims.

  • On XLRS-Bench Lite, TraceEarth is below the best baseline on all five columns where it is not best: OSR 31.60 vs. GeoVista 42.40, Plan 37.00 vs. GeoLLaVA 65.00, AR 74.00 vs. ZoomSearch 81.00, CR 67.50 vs. 81.00, and SR 45.00 vs. C2F 58.33. OSR and SR are precisely the object-spatial-relation tasks that object-space tracing is designed for, so this is in direct tension with the paper's thesis. None of it is discussed; the text cites only the favourable 70.72% on OP. Moreover, the 61.10 average is sample-size weighted, so the subsets on which the method is weakest appear to contribute little to the headline number; a macro-average over task types should be reported next to it.
  • On MME-RealWorld-RS, Count is 32.38, i.e. fourth behind RAP (40.25), ZoomSearch (39.15) and ZoomEye (35.56) — a ~8-point deficit on one of only three subsets, again undiscussed, while the abstract states "state-of-the-art" as an average-only claim.
  • Table 5 vs. Table 1: for RAP on the same benchmark, Table 5 reports Color 64.18 / Count 39.15 / Position 58.95 (Avg 54.21) whereas Table 1 reports 64.53 / 40.25 / 57.62 (Avg 54.20). The per-subset values differ while the averages coincide, so the +2.54 accuracy and −25.98 s deltas are computed against a row that cannot be reconciled with the main table without explanation.
  • Missing base-model rows: Qwen3-VL-8B vanilla appears neither in Table 2 nor in the LHRS column of Table 1, and the LHRS comparison includes no recent UHR method (only GPT-4o, LLaVA-v1.5, Qwen2.5-VL, GeoChat, RADAR). Consequently 72.61% on LHRS-Bench cannot be read as an improvement over either the backbone or the state of the art; it may simply reflect a stronger base model.

Taken together, a reader cannot separate "the method is better" from "the comparison is favourable", and two of the three headline numbers are not verifiable against a controlled baseline.

Questions for authors
  1. Please reconcile Table 5 with Table 1 for RAP (Color 64.18 vs. 64.53; Count 39.15 vs. 40.25; Position 58.95 vs. 57.62). Are the two evaluations on identical question sets, and are the +2.54 / −25.98 s deltas computed against the same configuration?
  2. Please report vanilla Qwen3-VL-8B on XLRS-Bench Lite and on LHRS-Bench, and add the strongest recent UHR methods (GeoEyes, WeaveEarth, ZoomSearch, GeoVista) to the LHRS column. What is the actual gain over your own backbone on those two benchmarks?
  3. Can you provide the three controls in W1 (generic agentic crop, RemoteSAM-only object crop, matched-compute self-consistency/CoT)? If the gain survives them, the paper's central claim is established; if not, would you consider reframing the contribution as an efficient zoom-and-verify pipeline?
  4. Why does TraceEarth rank below GeoVista on OSR (31.60 vs. 42.40) and below C2F on SR (45.00 vs. 58.33), given that object-space tracing is designed for spatial relations? Is the relation-preserving crop removing the context needed to judge relations, and does enlarging the crop or concatenating the full image recover the gap?
  5. Why is Count below RAP/ZoomSearch on MME-RealWorld-RS? Does the pipeline systematically lose counts, e.g. by truncating instances at crop boundaries or by discarding global context?
  6. How exactly is the 95.3% / 91.2% prevalence measured (annotation protocol, inter-annotator agreement, absolute counts)? What is the corresponding value on a benchmark that is not generated from positional templates?
  7. What are the sensitivities to τ, the 3×3 grid and the 0.5× margin, and were they tuned on a held-out split? What fraction of position questions take the fallback branch, and what accuracy does that branch achieve?
10 comments

Dear Reviewer 31uW,

Thank you for your thorough review and feedback on our manuscript. We truly appreciate the time and effort you dedicated to understanding our work, and your constructive comments. Below, we respond to each of the concerns you raised. We hope this addresses your concerns and helps further improve your evaluation of our work.

[Weakness 1 & Question 3: Regarding the Attribution of Performance Gains to Spatial Tracing]

We thank the reviewer for this important suggestion. To investigate whether the improvements of TraceEarth arise primarily from generic image cropping, additional object localization, or increased test-time computation, we conducted three additional controlled experiments using the same Qwen3-VL-8B backbone on MME-RealWorld-RS.

MethodColor (%)Count (%)Position (%)
Vanilla Qwen3-VL-8B59.7621.8568.01
RemoteSAM-only Crop57.1315.4252.19
Generic Agentic Crop65.3123.4070.20
8-Sample Voting60.0621.4568.09
TraceEarth69.3232.3877.72
  • RemoteSAM-Only Object Cropping. We evaluated a baseline that uses RemoteSAM for object localization and cropping without TraceEarth's spatial intent routing and cross-space tracing mechanisms. Its accuracies are 57.13%, 15.42%, and 52.19% on Color, Count, and Position, respectively, substantially below TraceEarth's 69.32%, 32.38%, and 77.72%. This indicates that simply introducing an additional localization model and object-centric cropping is insufficient to explain the improvements achieved by TraceEarth.

  • Generic Agentic Cropping. We also evaluated a generic agentic-crop baseline in which the VLM selects the image region to enlarge without TraceEarth's explicit image-space/object-space decomposition and role-aware routing. This baseline achieves 65.31% on Color, 23.40% on Count, and 70.20% on Position. Although generic zooming improves some categories relative to vanilla inference, its performance remains below TraceEarth across all three subsets, particularly for position prediction. These results highlight the importance of distinguishing spatial constraints from spatial prediction targets and selecting the corresponding evidence acquisition strategy, rather than applying generic cropping uniformly.

  • Additional Test-Time Computation through 8-Sample Voting. To examine whether repeated VLM inference alone accounts for the improvements, we independently sampled eight answers per question from the original full image using Qwen3-VL-8B with temperature 0.7 and top-p 0.9, and aggregated the predicted answer options by majority voting. This baseline achieves 60.06% on Color, 21.45% on Count, and 68.09% on Position, remaining close to the vanilla model and consistently below TraceEarth. Moreover, 823 of 3,738 questions (22.02%) produced different valid answer options across the eight samples, confirming that the experiment introduced stochastic variation rather than merely repeating deterministic predictions. Based on the measured single-pass vanilla latency of approximately 3.2 s, eight samples correspond to an estimated 25.6 s, close to TraceEarth's measured end-to-end latency of 29.1 s, which includes RemoteSAM. We characterize this as an approximately latency-matched sampling control, subject to verification using the actual cumulative sampling time.

  • Implications for the Central Claim. The three controls target complementary alternative explanations: object-centric cropping, generic agent-driven zooming, and additional stochastic inference. None reproduces TraceEarth's improvements across Color, Count, and Position. Together with the original ablation study (OST: 53.91%; IST+OST: 56.66%; full TraceEarth: 60.03%), these findings provide further evidence that TraceEarth's gains are associated with role-aware spatial intent routing and executable cross-space tracing, rather than generic zooming or increased test-time sampling alone.

Therefore, we believe the results support retaining spatial tracing as the central methodological contribution, rather than reframing TraceEarth solely as a generic zoom-and-verify pipeline.

[Weakness 2.1 & Question 4: Regarding Category-Level Performance and Spatial Reasoning Limitations on XLRS-Bench Lite.]

We thank the reviewer for highlighting the category-level performance gaps on XLRS-Bench Lite and for raising the important question of whether relation-preserving cropping may discard useful global context. We provide further clarification regarding the category-level evaluation, backbone-controlled improvements, task characteristics, and the effectiveness of context preservation.

  • Category-Balanced Evaluation. To further assess TraceEarth's performance across different task categories, we additionally compute the unweighted macro-average accuracy over all eight subtasks of XLRS-Bench Lite. TraceEarth achieves 55.11%, outperforming representative competing methods with complete category-level results, including GeoEyes (53.70%), GeoLLaVA-8K (52.34%), and GeoVista (50.74%). Notably, TraceEarth maintains the highest macro-average among the compared methods with complete eight-category results, demonstrating that its effectiveness is not solely attributable to the sample-size weighting used in the reported 61.10% overall accuracy. We will include the macro-average alongside the original overall accuracy in the revised manuscript to provide a more comprehensive and balanced evaluation.

  • Backbone-Controlled Improvements on OSR and SR. To better interpret the performance gaps, we additionally evaluate vanilla Qwen3-VL-8B on the corresponding subtasks. The original backbone achieves 29.00% on OSR and 41.67% on SR, whereas TraceEarth improves these results to 31.60% (+2.60 pp) and 45.00% (+3.33 pp), respectively. We acknowledge that these improvements are relatively modest and do not close the gaps to GeoVista and C2F. Nevertheless, the controlled comparisons indicate that TraceEarth provides positive improvements over its own backbone on both subtasks, rather than degrading their performance.

  • Distinction Between Object Spatial Relations and Spatiotemporal Reasoning. Importantly, OSR and SR evaluate different reasoning capabilities. OSR focuses on identifying objects and determining their spatial relationships, which directly aligns with the motivation of Object-Space Tracing (OST). In contrast, SR involves spatiotemporal reasoning, including regional changes and comparisons across different time points. Such tasks require temporal comparison and change reasoning beyond the static object-level spatial dependencies primarily addressed by OST. Therefore, while the OSR results highlight a direct limitation in complex spatial relationship understanding, the SR gap also reflects reasoning requirements that TraceEarth is not specifically designed to optimize.

  • Relation-Preserving Cropping and Global–Local Context Analysis. When the object-level spatial relation itself is the prediction target, TraceEarth does not use this unknown relation to guide cropping. Instead, it localizes the relevant entities and constructs a compact enclosing region that preserves their original relative spatial layout while reducing irrelevant background. To examine whether broader scene context can further improve relation reasoning, we introduce a Global+Local setting that provides both the original full image and the spatially traced local evidence to the same frozen Qwen3-VL-8B backbone, with the remaining inference settings unchanged.

    MethodVisual EvidenceOSR Acc. (%)
    Vanilla Qwen3-VL-8BOriginal full image29.00
    TraceEarth (Ours)Standard spatial tracing31.60
    TraceEarth + Global ContextSpatial tracing + original full image29.40

    Interestingly, adding the original full image reduces OSR accuracy from 31.60% to 29.40% (−2.20 pp). This finding suggests that focused, relation-preserving local evidence can be more effective than simply combining local evidence with the entire scene. While broader contextual information may still be useful for individual questions, indiscriminately introducing the full image does not improve spatial reasoning under the evaluated setting. The result supports our design choice of reducing irrelevant visual content while preserving the spatial layout of question-relevant entities.

Overall, the category-balanced macro-average and backbone-controlled improvements demonstrate the effectiveness of spatial tracing across different XLRS-Bench Lite tasks. Furthermore, the Global+Local experiment shows that simply adding global visual context does not resolve the remaining OSR gap, highlighting the value of focused spatial evidence acquisition.

[Weakness 2.2 & Question 5:Regarding counting performance on MME-RealWorld-RS and its potential limitations.]

We thank the reviewer for raising this important concern regarding the counting performance of TraceEarth. We acknowledge that TraceEarth achieves lower Count accuracy than RAP and ZoomSearch in the reported comparison. However, our controlled evaluations show that spatial tracing does not systematically degrade counting accuracy relative to the corresponding original VLMs or the evaluated integrated framework.

  • Consistent Counting Improvements over the Original Backbones. On MME-RealWorld-RS, TraceEarth improves Count accuracy from 21.85% to 32.38% (+10.53 pp) on Qwen3-VL-8B and from 22.51% to 37.27% (+14.76 pp) on LLaVA-OV-7B. These improvements indicate that spatially guided visual tracing can facilitate counting by focusing on question-relevant visual evidence, rather than inherently impairing counting performance.

  • Complementarity with Existing Visual Search Frameworks. Our plug-and-play experiment in Table 5 further shows that integrating TraceEarth into RAP improves Count accuracy from 39.15% to 39.72% (+0.57 pp), while also improving overall accuracy from 54.21% to 56.75% (+2.54 pp) under the same evaluation setting. We clarify that the 39.15% RAP Count result is obtained from our reproduced baseline used for the paired comparison in Table 5, whereas 40.25% in Table 1 is the previously reported result. This controlled comparison suggests that spatial tracing can complement existing visual search strategies without necessarily compromising their counting capability.

  • Limitations of Localized Evidence for Global Counting. We recognize that counting presents a particular challenge for localized visual tracing. While spatial cropping can improve the visibility of small objects and resolve target ambiguity, questions requiring exhaustive counting across a large scene may depend on evidence distributed over multiple regions. In such cases, insufficient crop coverage or objects intersecting crop boundaries could result in missed instances. Importantly, TraceEarth does not force spatial cropping when no actionable spatial constraint is available; instead, it retains the original full-image inference path. Nevertheless, our current results do not establish that all instance-omission errors are eliminated. This potential tension between precise localization and comprehensive instance coverage remains a limitation that we will clarify in the revised manuscript.

Overall, although TraceEarth does not achieve the highest Count accuracy among all compared methods, its consistent gains over the original backbones and its positive contribution when integrated into RAP support the value of spatial tracing for counting in the evaluated settings. Further improving the coverage of spatially distributed instances remains an important direction for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.