acceptodds
Under review as a conference paper at ICLR 2027

EfficientViS: Adaptive Parallel Visual Search for High-Resolution Remote Sensing Visual Grounding

Abstract

High-resolution remote sensing visual grounding (RSVG) requires both global spatial context and fine-grained evidence for small targets. Full-scene inference preserves context but loses local details after resizing, whereas iterative visual search recovers such details through sequential crop-and-ground calls at substantial computational cost. We propose EfficientViS, an adaptive parallel framework for efficient high-resolution RSVG. EfficientViS consists of two components: 1) One-shot Complementary Region Planning (OCRP), which predicts diverse candidate regions from a single global image–text representation before local grounding; and 2) Adaptive Candidate Parallel Inference (ACPI), which selects a query-adaptive observation budget and packs selected crops into a single specialist forward pass for efficient grounding. We further introduce a two-stage training strategy: Stage I trains a crop-robust specialist grounder to improve local grounding, while Stage II applies supervised-to-PPO training to optimize the accuracy–cost trade-off. Extensive experiments on three benchmarks (RSVG-HR, AVVG, and XLRS-Bench) demonstrate that EfficientViS delivers 4.3–4.8× higher throughput than the corresponding full-scene specialist baselines and more than 14× higher throughput than sequential search.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Grounding, Remote Sensing, High-Resolution Vision-Language, Visual Search, Adaptive Region Planning, Parallel Inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9ukpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper targets high-resolution remote sensing visual grounding. To avoid the latency of sequential, iterative visual search and the detail loss of full-image downsampling, it proposes EfficientViS to decouple candidate region planning from local verification. The model predicts candidate crops from a downsampled global view in one shot and evaluates them in parallel.
Contributions OCRP: Generates an ordered set of candidate regions from a single downsampled global representation using spatial scoring and neighborhood suppression.
ACPI: Predicts an adaptive observation budget via a utility head and packs the selected crops into a single specialist forward pass for parallel local grounding.
Training & Theory: Employs a two-stage training strategy (crop-jitter adaptation followed by supervised bootstrapping and PPO policy tuning), backed by an information-theoretic bound arguing that local feedback provides limited planning value.
Validation Evaluated on three remote sensing datasets spanning 1K to 10K resolutions (RSVG-HR, AVVG, XLRS-Bench) with LPVA and LQVG backbones. The method yields moderate localization gains (1.2 to 3.9 points in [email protected]) while providing reported 4.3–4.8x throughput improvements over full-scene baselines and over 14x over sequential search.

Critical strengths

Tangible inference throughput gains: The main merit of the paper lies in its practical throughput advantage over serialized visual search. Rather than invoking the specialist model sequentially across iterative steps, evaluating candidate crops in a single packed batch yields reported 4.3–4.8x speedups over full-scene baselines and over 14x over sequential search. This provides a pragmatic engineering solution to the severe latency bottleneck in high-resolution visual grounding.

Critical weaknesses

Circular theoretical bound: The information-theoretic argument against local feedback trivially begs the question. It assumes that residual spatial information is negligible to conclude that feedback has bounded utility. In practice, downsampling 8K images to a 256x256 thumbnail heavily degrades small targets, making local observations the primary source of discriminative signal. The core theoretical premise simply fails in real ultra-high-resolution settings.

Limited technical novelty: Strip away the "visual search" terminology, and the framework reduces to a standard two-stage cascade: coarse candidate selection on a downsampled thumbnail followed by batched crop verification. Layering PPO and a six-term reward on top of this feed-forward pipeline introduces heavy engineering complexity without offering new algorithmic insights into the localization problem.

Questions for authors

In Table 2, the baseline runs on 1024x1024 images, whereas EfficientViS processes a 256x256 thumbnail plus an average of 1.6–2.5 crops of 256x256—roughly one-fifth of the total pixels. How much of the reported 4.5x speedup is simply an artifact of processing far fewer pixels rather than parallel inference? What are the baseline's accuracy and FPS under an equivalent pixel or FLOPs budget?
The theoretical bound assumes that local observations provide negligible residual information over the global view. However, [email protected] on XLRS-Bench (8K–10K) is only 11.97%. What is the standalone recall of the 256x256 planner on XLRS? Specifically, what percentage of the ~88% failure cases stem from the planner missing the target entirely across the proposed crops?

7 comments
AuthorsOpenReview ↗

Response to Reviewer 9ukp (1/4)

Thank you for your thorough review and for recognizing EfficientViS's throughput gains. We addressed the scope of our information theoretic bound, the distinction from existing global and local methods, the sources of speedup, and planner failures on XLRS-Bench to address your concerns, as detailed below.

Response to W1: Conditional Meaning of the Information Theoretic Bound and Feedback Free Planning

Thank you for your careful examination of our theoretical analysis. Your comment helped us realize that the phrasing in Section 3.2 of the original manuscript could mislead readers into assuming that our theorem relies on a prior assumption that "the residual spatial information in ultra-high-resolution imagery is inherently small." Below, we clarify this point from three perspectives: mathematical logic, functional decoupling across modules, and the practical behavior of lightweight planners.

1. Meaning of the Bound

Corollary 1 follows from conditional mutual information, Pinsker's inequality, and Jensen's inequality. For any planning utility normalized to [0,1][0, 1] and any nonnegative conditional mutual information Ij\mathcal{I}_j between the next target region and previous local observations given ZZ, it gives the following unconditional bound.

0≤Vfb,j∗−Vff,j∗≤2Ij0 \le V_{\text{fb},j}^* - V_{\text{ff},j}^* \le \sqrt{2\mathcal{I}_j}

The bound does not claim that Ij\mathcal{I}_j is small in every ultra high resolution scene. It states that when the global cross modal representation ZZ already narrows the plausible regions, prior local observations O<jO_{<j} may add limited information about where to search next. In that case, even an ideal Bayes optimal feedback planner has a bounded utility gain over feedback free planning. When tiny targets or weak reference cues are lost in the global view while the planner can use the local information effectively, Ij\mathcal{I}_j can be large.

2. Global Planning and Local Patch Evidence

Local detail is still used for localization and it is simply not fed back to change the remaining crop plan. The global representation ZZ determines where to look by proposing complementary regions and an observation budget before local inference. The second stage extracts small patches directly from the original high resolution image and sends them to the specialist grounder for fine grained discrimination and final box regression.

3. Lightweight Planner and Controlled Sequential Comparison

Corollary 1 describes an ideal Bayes optimal planner. Our deployed planner has only 1.08M parameters, less than 1% of the specialist model. Unlike MLLMs, which can draw on broad pretrained knowledge and reasoning to reassess their predictions, the lightweight planner has limited capacity to detect and correct early search errors. In dense scenes with similar looking objects, an early false positive can mislead this lightweight planner and affect every later crop (as shown in Figure 6). To test this effect, we compare EfficientViS with a sequential baseline using the same backbone, planner, crop size, and training hyperparameters. The sequential baseline feeds each crop's local features back to select the next region. The only changed factor is the planning process. Table 1 reports target center coverage by object size.

Table 1: Comparison of target center coverage rates across different object scales on XLRS-Bench.

GT Longer Side (px)InstancesEfficientViSSequential InferenceCoverage Difference
≤32\le 3245638.60%31.58%+7.02%
(32,64](32, 64]1,67832.72%29.26%+3.46%
(64,128](64, 128]1,83031.09%28.25%+2.84%
(128,256](128, 256]1,13031.59%29.38%+2.21%
>256> 2561,21527.90%31.03%-3.13%
Overall6,30931.54%29.50%+2.04%

For targets no larger than 256 px, which make up 80.7% of the dataset, parallel planning has higher center coverage in every size group. Its overall recall is 31.54% compared with 29.50% for sequential search, with the largest difference on targets no larger than 32 px. This is consistent with the risk that a lightweight sequential planner follows an early false positive in a visually ambiguous crop. Parallel planning considers complementary regions before any local prediction can misdirect it.

For targets larger than 256 px, sequential search has 3.13 points higher center coverage. Local feedback can therefore help follow large objects across regions. The results show where feedback is useful while confirming higher overall recall for parallel planning.

AuthorsOpenReview ↗

Response to Reviewer 9ukp (2/4)

Response to W2: Explanation of Technological Innovation

Thank you for raising this concern. The main contribution of EfficientViS is to decouple region planning from local verification, allowing high resolution local evidence to be processed in parallel without waiting for feedback from earlier crops.

Most existing visual search methods, including V*, GeoViS, ZoomEarth, and GeoSearcher, follow a sequential process. They use local observations both to decide where to look next and to perform final localization. Although this recovers fine details and improves grounding, each new region decision must wait for the preceding local inference. Repeated crop selection and grounding therefore limit computational efficiency.

To address this bottleneck, we explore decoupling global region planning from local fine verification, drawing inspiration from biological visual attention. Rapid feedforward visual processing provides early semantic information within approximately 150 ms, while global scene information helps guide subsequent selective inspection [1, 2]. The visual system can also distribute attention across several spatially separated locations. Studies of multiple attentional spotlights and multifocal tracking suggest a limited capacity, often around four attended targets [3 to 5]. Together with Inhibition of Return, which discourages revisiting recently attended locations, these mechanisms motivate planning a small set of spatially complementary candidates before detailed verification.

The reason humans exhibit sequential eye movements during fine verification is largely due to the physical constraint of a biological single fovea [6]. While existing visual search methods naturally inherit this sequential paradigm, our work takes a complementary perspective of "one-shot global gist planning + adaptive parallel local verification." GPU computation offers a way to inspect multiple image patches concurrently and this motivates combining complementary attention allocation with hardware parallelism. The candidate plan is therefore determined before local verification, avoiding the repeated waits between local inference and the next region decision.

This separation preserves the role of fine visual evidence. The specialist still receives small patches cropped directly from the original high resolution image and uses their textures, boundaries, and other local details for final localization. Only the use of intermediate local observations to redirect subsequent region selection is removed. We implement it through the following three aspects.

[1] Speed of processing in the human visual system. Nature, 381(6582), 520 to 522.

[2] Visual search in scenes involves selective and nonselective pathways. Trends in Cognitive Sciences, 15(2), 77 to 84.

[3] Parallel and serial neural mechanisms for visual search in macaque area V4. Science, 308(5721), 529 to 534.

[4] Multiple spotlights of attentional selection in human visual cortex. Neuron, 42(4), 677 to 686.

[5] Tracking multiple targets with multifocal attention. Trends in Cognitive Sciences, 9(7), 349 to 354.

[6] Optimal eye movement strategies in visual search. Nature, 434(7031), 387 to 391.

continued in Part 3/4

AuthorsOpenReview ↗

Response to Reviewer 9ukp (3/4)

1. Complementary Global Region Planning with OCRP

OCRP plans complementary candidate regions from a single global image and text representation before local grounding begins. Independent Top K selection can concentrate crops around the same distractor. Neighborhood suppression instead conditions each new candidate on the regions already selected, while continuous offsets refine its position. This spreads a limited observation budget across different spatial hypotheses without requiring local feedback. At the same budget of K = 4, complementary suppression improves RSVG-HR [email protected] from 86.57% to 88.38%, a gain of 1.81 percentage points.

2. Adaptive Budgeting and Parallel Verification with ACPI

Decoupling removes the need to inspect crops sequentially, but using the maximum crop count for every query would still waste computation. ACPI predicts the utility of ordered candidate prefixes and selects one to four crops before any local inference. It then packs the retained crops into a single specialist forward pass. On RSVG-HR, it matches the 88.58% accuracy of fixed K = 4 with only 1.63 crops on average, increasing throughput from 77.36 FPS to 122.01 FPS, a 57.7% improvement.

3. Policy Optimization with Training Only PPO

We optimize region selection and budget allocation using the final grounding outcome and computational cost. PPO handles the discrete planning decisions, while differential returns credit each additional crop according to its contribution. The six reward terms balance final IoU, target coverage, and local grounding quality against overlap, observation cost, and latency. Compared with the supervised adaptive baseline, PPO improves [email protected] by 0.40, 1.20, and 1.85 percentage points on RSVG-HR, AVVG, and XLRS-Bench, respectively. These rewards and policy updates are used only during training, so they introduce no extra computation at inference.

OCRP coordinates candidate regions, ACPI sets the crop budget, and PPO optimizes both decisions for localization quality and computational cost. This allows EfficientViS to preserve global context and fine local evidence while improving how candidate regions are coordinated, how many observations are acquired, and how the policy balances localization quality with computational efficiency.

Response to Q1: Attribution of the Speedup and Equal Compute Comparison

Thank you for raising this question. We would first like to emphasize that reducing computation through adaptive allocation of high-resolution visual processing is itself a key part of our methodological contribution. Specifically, EfficientViS achieves its speedup through two complementary mechanisms: processing fewer pixels through selective local crops and processing those crops in parallel. We matched the per sample GFLOPs of full scene LQVG to EfficientViS across all three benchmarks to separate these effects.

The table reports results on the same NVIDIA RTX 5090. The LQVG input resolutions were adjusted to 400 × 400, 448 × 448, and 544 × 544 to match EfficientViS compute.

Table 2: Accuracy and throughput comparison between the LQVG baseline and EfficientViS under equal computational budgets.

DatasetMethod and Input SettingCompute (GFLOPs/sample)Throughput FPS ↑\uparrowAccuracy [email protected] (%) ↑\uparrow
RSVG-HRLQVG (downsampled to 400×400400 \times 400)36.083153.5475.95
EfficientViS36.123122.0188.58 (+12.63)
AVVGLQVG (downsampled to 448×448448 \times 448)42.539142.9727.56
EfficientViS43.717136.8432.96 (+5.40)
XLRS-BenchLQVG (downsampled to 544×544544 \times 544)69.606129.366.15
EfficientViS70.214128.4611.97 (+5.82)

Under matched compute, downsampled LQVG is faster because it avoids crop scheduling and coordinate fusion. This shows that pixel reduction contributes to the speedup. However, directly reducing the full-scene resolution causes severe loss of fine-grained visual features, leading to a sharp drop in localization accuracy. EfficientViS preserves those details in local crops and significantly improves [email protected] by 12.63, 5.40, and 5.82 points across the three benchmarks.

Therefore, the core advantage of EfficientViS lies in combining efficient allocation of computing resources with parallel execution, thereby achieving a better balance between positioning accuracy and inference efficiency.

Reviewer_RJGdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

EfficientViS addresses high-resolution remote-sensing visual grounding by separating global region planning from local verification. One-shot Complementary Region Planning proposes diverse regions from a downsampled global image-text representation, while Adaptive Candidate Parallel Inference chooses a query-dependent crop budget and evaluates the crops in one packed specialist pass. A crop-adaptation stage is followed by supervised initialization and PPO refinement. Across RSVG-HR, AVVG, and XLRS-Bench with LPVA and LQVG, the method reports consistent [email protected] gains, 4.3-4.8x throughput over full-scene specialist baselines, and much larger gains over sequential search, supported by component, budget, crop-size, reward, and failure-mode analyses.

Critical strengths
  1. The one-shot planning/parallel verification decomposition is simple, relevant, and practically consequential. It directly removes the serial dependency that makes visual search slow while retaining high-resolution evidence.
  2. The evaluation spans three image scales and two specialist grounders, and the ablations cover complementarity, adaptive budgeting, crop adaptation, PPO terms, and candidate capacity. The consistent gains on XLRS-Bench are particularly persuasive.
Critical weaknesses
  1. The paper reports single aggregate accuracy values without repeated seeds or confidence intervals. Several headline gains on RSVG-HR are around one point, so the absence of variability estimates makes their robustness difficult to judge.
  2. The very large speedup over a single full-scene specialist pass is not decomposed sufficiently. EfficientViS includes global feature extraction plus an average of more than one local crop, yet is reported as over four times faster; per-stage latency, batch size, warm-up, and input-resolution effects are needed to establish that the comparison reflects the algorithmic design rather than implementation asymmetry.
Questions for authors
  1. Please report variance across independent training seeds, at least for the LQVG baseline and full EfficientViS on the three benchmarks. Would the reported gains remain statistically distinguishable?
  2. Can the authors provide a latency breakdown for preprocessing, global encoding, planning, crop construction, packed specialist inference, and fusion, together with memory and effective batch size? Please also explain why the full pipeline is more than four times faster than one full-scene specialist forward pass.
3 comments

Response to W1/Q1: Multi-Seed Evaluation and Statistical Reliability

Thank you for asking about statistical reliability. We evaluated LQVG and EfficientViS with three independent random seeds on RSVG-HR, AVVG, and XLRS-Bench. All runs used the same dataset splits and evaluation protocol. The table reports the mean [email protected] and standard deviation (Mean ± SD) across the three seeds.

DatasetLQVG (Mean ± SD)EfficientViS (Mean ± SD)Gain
RSVG-HR87.17 ± 0.2088.51 ± 0.12+1.34
AVVG30.74 ± 0.0532.93 ± 0.03+2.19
XLRS-Bench8.00 ± 0.0411.94 ± 0.03+3.94

EfficientViS has higher mean accuracy and lower standard deviation than LQVG on all three benchmarks. On RSVG-HR, its mean [email protected] is 88.51%, a gain of 1.34 percentage points, with a standard deviation of 0.12%. The gains on AVVG and XLRS-Bench are 2.19 and 3.94 percentage points, respectively. These results show that the improvements persist across the three seeds.

Response to W2/Q2: Detailed Latency Breakdown and Explanation of the Inference Speedup

Thank you for requesting a detailed analysis of inference efficiency. We profiled AVVG, where EfficientViS has the largest throughput gain over LQVG.

(1) Inference configuration and latency breakdown.

For a fair comparison, both LQVG and EfficientViS were evaluated using the same NVIDIA RTX 5090 GPU, BF16 precision, and an inference batch size of 16. We performed several forward passes for warm-up and recorded the subsequent forward pass.

We measure model forward computation to compare the two architectures. Preprocessing, including data loading, is excluded from the forward latency and GPU memory measurements. Latency is reported in milliseconds per sample.

Component / MetricLQVGEfficientViS
Preprocessing (ms/sample, excluded)4.9834.978
Global encoding (ms/sample)9.2822.241
Region planning (ms/sample)--0.130
Crop construction (ms/sample)--0.032
Specialist inference (ms/sample)24.8565.031
Score fusion (ms/sample)--0.031
Total forward latency (ms/sample)34.1387.465
Peak GPU memory (GiB)29.9910.94

EfficientViS reduces global encoding latency from 9.282 ms to 2.241 ms and specialist inference latency from 24.856 ms to 5.031 ms. Region planning, crop construction, and score fusion add 0.193 ms per sample. Total forward latency falls from 34.138 ms to 7.465 ms. Peak GPU memory usage falls from 29.99 GiB to 10.94 GiB, a reduction of approximately 63.5%.

(2) Explanation of the inference speedup.

The original LQVG processes the entire scene using a 1024 × 1024 input, requiring the specialist model to extract features and perform grounding over the full image. In contrast, EfficientViS first performs global planning on a 256 × 256 downsampled view and then selects informative 256 × 256 local crops for fine-grained grounding.

EfficientViS determines the region plan and adaptive observation budget before local inference. It then packs all selected crops into a single specialist forward pass. This avoids repeated local inference between sequential region decisions. This reduces global encoding and specialist inference costs with little additional overhead for planning and fusion.

Summary

Thank you again for your careful review and suggestions. We hope our responses address your concerns and welcome any further feedback.

Reviewer_RJGdOpenReview ↗

Response by Reviewer

Thank you for your responses. My concerns have been addressed. I will keep my positive score.

Thank you for your positive feedback and continued support. We sincerely appreciate your time and constructive suggestions, which have helped us improve our work.

Reviewer_yw1mOpenReview ↗
Rating
3(+1)
Confidence
1(−2)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EfficientViS, an adaptive parallel visual search framework for high-resolution remote sensing visual grounding (RSVG). It introduces two components: One-shot Complementary Region Planning (OCRP) and Adaptive Candidate Parallel Inference (ACPI) for efficient high-resolution image understanding. Experiments on RSVG-HR, AVVG, and XLRS-Bench with two specialist grounders (LPVA and LQVG) report consistent improvements in [email protected] and throughput gains. However, several important concerns regarding the methodological design and experimental setup need to be clarified.

Critical strengths
  1. The paper addresses an important challenge in high-resolution remote sensing visual grounding: recovering fine-grained target information without the considerable inference overhead of repeatedly processing large images. The proposed framework achieves throughput improvements while maintaining or improving grounding accuracy. The improvements are observed with two different specialist grounders across three benchmarks.

  2. Relatively comprehensive component-wise evaluation. The paper includes ablations on complementary region selection, adaptive observation budgets, PPO rewards, crop sizes, candidate capacity, and specialist adaptation. In particular, the comparisons with fixed-budget inference and the additional Top-K experiments strongly validate the rationality of the proposed design.

Critical weaknesses
  1. Unclear training and evaluation protocols. The paper does not clearly specify the training and test splits used for the three benchmarks. Were EfficientViS and all baselines trained on the corresponding official training sets under consistent settings? Since some MLLM results are taken from previous studies, the fairness and comparability of these results also need clarification.

  2. Concerns about the localization accuracy of the global planner. EfficientViS downsamples large images (e.g., 8192×8192) to 256×256 for global planning, using a 32×32 planning grid. Despite continuous offset refinement, accurately locating small targets from such a low-resolution representation remains challenging. Moreover, four 256×256 crops cover less than 0.4% of an 8192×8192 image. Although Figure 5 reports Planner Recall, I would like to see a more detailed analysis across different target sizes, image resolutions, and referring-expression types.

  3. Limitations of fixed-size crops. The fixed 256×256 crops may be insufficient when the target itself is larger than the crop, has a large aspect ratio, or requires spatial context from distant reference objects. Although the paper acknowledges this limitation, it neither adapts crop sizes to target scales nor explains how large targets spanning multiple crops can be reconstructed. I consider this particularly important for real-world high-resolution remote sensing imagery.

  4. Unclear contribution of local visual search. Since the global planner is directly supervised using ground-truth target centers, it already learns to locate targets from the low-resolution image. How much additional benefit does local visual search actually provide? A planner-only grounding baseline that directly predicts bounding boxes from global features would help clarify whether the main improvement comes from the supervised planner or the proposed local grounding strategy.

  5. I am not fully convinced that completely removing local feedback is appropriate for ultra-high-resolution imagery. A downsampled global view may only identify plausible regions, while local observations can provide critical information to guide subsequent exploration. For images exceeding 10,000 or even 20,000 pixels, relying on one-shot planning with at most four crops seems particularly restrictive. Although the theoretical derivation appears reasonable, the assumption that local feedback provides little additional information is not empirically validated. A comparison with a hybrid parallel-sequential search strategy would help clarify the applicability and limitations of the proposed approach.

  6. Although EfficientViS improves [email protected] from 8.03% to 11.97% on XLRS-Bench, the absolute accuracy remains very low, with nearly 88% of instances failing to reach IoU 0.5. This raises concerns about its practical effectiveness for ultra-high-resolution grounding. I believe a more detailed failure analysis, particularly regarding planner errors, insufficient crop coverage, and local grounding failures, would be more informative than primarily emphasizing inference efficiency at such a low accuracy level.

Questions for authors

Please see the weaknesses.

11 comments
AuthorsOpenReview ↗

Response to Reviewer yw1m (1/5)

Thank you for your thorough and constructive review. We have conducted additional analyses and experiments to address your six concerns, as detailed below.

Response to W1: Training and Evaluation Protocols

We follow the official training and test splits for all benchmarks and specialist comparisons use consistent splits and evaluation protocols.

1.1. Official Dataset Splits and Consistent Evaluation

The dataset statistics are summarized in Table 1.

Table 1. Official dataset splits and image resolutions.

DatasetImage resolutionTraining samplesTest samplesTotal samples
RSVG-HR1024 × 10242,1514992,650
AVVG4000 × 225026,4826,13932,621
XLRS-Bench4096 × 4096 to 10000 × 100006,3096,31012,619

The official XLRS-Bench test split contains 6,310 examples. One example has an invalid coordinate annotation and is excluded from our instance-level diagnostic analyses, leaving 6,309 valid test examples. We found in the statistics that the number of AVVG datasets is not consistent with the report by GeoGround [1] due to data version differences. In the revised version, we will change the reported number in the main text to the actual number.

1.2. MLLM Results

We report MLLM results from original papers or benchmark reports (e.g., GeoGround [1] and XLRS-Bench [2]), consistent with common RSVG practice. We do not retrain them because dataset-specific fine-tuning would change their general-purpose setting and may cause catastrophic forgetting [3]. Some weights and training pipelines are also unavailable and our goal is to evaluate EfficientViS as an efficient search framework for specialist grounders.

[1] GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding. arXiv preprint arXiv:2411.11904, 2024.

[2] XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery? Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14325 to 14336, 2025.

[3] Investigating the Catastrophic Forgetting in Multimodal Large Language Model Fine-Tuning. Conference on Parsimony and Learning (CPAL), PMLR 234:202 to 227, 2024.

AuthorsOpenReview ↗

Response to Reviewer yw1m (2/5)

Response to W2: Global Planner Recall and Fine-Grained Diagnostics

Thank you for raising this concern. We measure planner coverage by center recall, which counts a hit when a retained crop contains the ground-truth center.

2.1. Planner Center Recall by Target Size

Table 2a. Center recall by target size on XLRS-Bench.

GT longer side (px)SamplesSample shareCenter hitsCenter recall
≤ 324567.23%17638.60%
(32, 64]1,67826.60%54932.72%
(64, 128]1,83029.01%56931.09%
(128, 256]1,13017.91%35731.59%
> 2561,21519.26%33927.90%
Overall6,309100.00%1,99031.54%

Table 2b. Center recall by relative target area on XLRS-Bench.

Relative target area< 0.005%0.005% to 0.02%0.02% to 0.1%0.1% to 0.5%≥ 0.5%Overall
Samples1,7351,8971,4628203956,309
Center recall26.80%30.89%35.16%37.20%30.38%31.54%

Recall is highest for targets no larger than 32 px at 38.60% and remains between 31.09% and 32.72% for targets from 32 to 256 px. This indicates that global scene and language cues can still guide the planner toward tiny objects that occupy few pixels in the overview. By relative area, it peaks at 37.20% for targets occupying 0.1% to 0.5% of the image, then falls to 30.38% for targets occupying at least 0.5%, showing that large targets are harder to cover with sparse candidate regions.

2.2. Planner Center Recall by Image Resolution

Table 2c. Center recall by original image resolution on XLRS-Bench.

Original resolutionSamplesCenter hitsCenter recall
4096 × 40961,70886050.35%
5616 × 374427010237.78%
7360 × 49121,11632128.76%
10000 × 100003,09863320.43%
Other resolutions1177463.25%
Overall6,3091,99031.54%

Center recall decreases as image resolution grows, from 50.35% at 4096 × 4096 to 20.43% at 10000 × 10000. Compressing a larger image into the same global view can smooth weak target features and local reference cues, which makes it harder for the planner to identify promising regions.

2.3. Planner Center Recall by Language Cue

Table 2d. Center recall by language cue on XLRS-Bench.

Language cueSamplesCenter hitsCenter recall
Absolute location6,1021,92231.50%
Relative spatial relations6,2141,96031.54%
Visual attributes6,0131,91531.85%
Color descriptions5,6341,82732.43%
Shape descriptions3,3781,00929.87%
Size descriptors75916321.48%
Quantity descriptors3,18198330.90%

Recall is similar for absolute location, spatial relations, and visual attributes, ranging from 31.50% to 31.85%. Recall also remains near 30% for color, shape, and quantity cues. This consistency suggests that OCRP uses more than directional words alone. Size descriptors have lower recall at 21.48%. Phrases such as “the larger building” require comparisons among nearby objects, and these details are difficult to distinguish in the downsampled global view.

AuthorsOpenReview ↗

Response to Reviewer yw1m (3/5)

Response to W3: Fixed Crop Size

Thank you for raising the crop-size concern. The 256 × 256 crop covers most benchmark targets and preserves efficient batched inference.

3.1. Target Size and Crop Coverage

We measure the longer side of each ground-truth box. For AVVG, we count an object once when multiple expressions refer to it. The table shows the resulting distribution across the three test sets.

Table 3. Distribution of GT longer-side lengths across the three benchmark test sets.

GT longer sideRSVG-HRAVVGXLRS-Bench
≤ 64 px291 (58.32%)7 (0.30%)2,134 (33.82%)
(64, 128] px171 (34.27%)842 (36.47%)1,830 (29.01%)
(128, 256] px35 (7.01%)1,055 (45.69%)1,130 (17.91%)
> 256 px2 (0.40%)405 (17.54%)1,215 (19.26%)
Cumulative ≤ 256 px497 (99.60%)1,904 (82.46%)5,094 (80.74%)

Targets no larger than 256 px account for 99.60% of RSVG-HR, 82.46% of AVVG, and 80.74% of XLRS-Bench, so this crop can fully contain most targets. The crop-size ablation also shows that accuracy improves when the crop size increases from 128 px to 256 px. Increasing further to 320 px or 384 px adds little accuracy and reduces throughput. The target size distribution and crop-size ablation therefore support using 256 × 256 windows to balance target coverage and processing speed.

3.2. The Reason for Fixed Windows

We previously considered predicting variable crop sizes, but estimating original object scale from a 256 × 256 overview is uncertain. A scale error can either include excessive background or truncate the target. Variable windows also need resizing or padding before batching, which weakens the consistent inputs used by ACPI and decreases efficiency. Fixed windows with an adaptive number of crops preserve batched inference while letting the observation budget respond to query complexity.

Response to W4: Contribution of High-Resolution Local Grounding

Thank you for suggesting this ablation. The Planner-only comparison shows that high-resolution local grounding improves [email protected] by 29.06, 12.58, and 7.99 percentage points on RSVG-HR, AVVG, and XLRS-Bench, respectively. The two stages provide complementary functions. Supervision with target centers teaches the planner where to search, but downsampling removes the textures and edges needed for precise box regression. High-resolution crops restore those details, allowing the specialist grounder to distinguish similar objects and refine their boxes.

Table 4. Grounding accuracy ([email protected]) of the Planner-only baseline and full EfficientViS.

MethodRSVG-HRAVVGXLRS-Bench
Planner-only baseline59.5220.383.98
EfficientViS88.5832.9611.97
Absolute improvement+29.06+12.58+7.99

Response to W5: One Shot Planning, Local Feedback, and 10K Images

Thank you for questioning one-shot planning at very high resolution. Controlled comparisons indicate that parallel planning is generally faster and more accurate than sequential planning for lightweight planners.

5.1. Planning Strategy and 10K Results

Table 5a. [email protected] (%) on the 10K and lower resolution subsets of XLRS-Bench.

Resolution groupSamplesLQVGEfficientViSAbsolute gainRelative gain
10K3,0983.327.17+3.84+115.5%
Below 10K3,21112.5816.60+4.02+31.9%

EfficientViS improves accuracy in both resolution groups. On the 10K subset, it more than doubles LQVG accuracy, from 3.32% to 7.17%, with a gain of 3.84 percentage points. On the lower resolution subset, accuracy increases from 12.58% to 16.60%, a gain of 4.02 points. These results support the usefulness of globally guided local verification even at 10K resolution. The planner uses scene context to select candidate regions, and the specialist grounder accesses original image detail through local patches that avoid the loss of fine features during full image resizing.

5.2. Candidate Budget

We set M = 4 as a practical upper limit on the candidate pool, while ACPI determines how many crops to verify for each query and it averages 1.63, 2.41, and 2.53 crops per image on RSVG-HR, AVVG, and XLRS-Bench, respectively. The budget ablation shows diminishing recall gains as M increases from 3 to 5, so larger M offers limited benefit relative to their computational cost. M=4 is a trade-off between accuracy and computational efficiency.

continued in Part 4/5

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to All Reviewers: Summary of Revisions

Dear Reviewers and Area Chair,

We thank Reviewers yw1m, RJGd, and 9ukp for their careful reviews and constructive suggestions. We have revised the manuscript and highlighted the main changes in blue, primarily in the appendix. The summary below identifies the revisions and their locations.

1. Clarified the contribution and theoretical scope. We explain how EfficientViS separates global region planning from local verification, drawing inspiration from biological attention. Small patches from the original image still provide fine details for localization. Only feedback from intermediate local predictions to subsequent region selection is removed. The bound concerns the Bayes-optimal expected value of feedback and indicates limited benefit when residual information is small. We explain the roles of complementary planning, adaptive budgeting, and policy optimization. Locations: Appendix A.3.4.

2. Added a controlled sequential comparison. With the same backbone, planner architecture, specialist, crop size, and training hyperparameters, EfficientViS improves [email protected] by 0.60%, 1.62%, and 1.92%, with 1.34×, 1.42×, and 1.39× throughput. On XLRS-Bench, overall center recall rises from 29.50% to 31.54%, with gains in all four target-size groups up to 256 pixels. We discuss how early false positives can misdirect a lightweight sequential planner. Location: Appendix A.4.5 (Tables 12 and 13).

3. Examined the sources of speedup and measured inference costs. At approximately matched GFLOPs, EfficientViS improves [email protected] over downsampled full-scene LQVG by 12.63%, 5.40%, and 5.82%. The downsampled baselines are faster at these budgets, confirming that reduced pixel processing contributes to the original speedup and selective crops preserve more detail for localization. AVVG profiling with the same RTX 5090, BF16 precision, and batch size of 16 shows that forward latency falls from 34.138 to 7.465 ms per query and peak GPU memory from 29.99 to 10.94 GiB. Planning, cropping, and fusion together add 0.193 ms per query. Locations: Appendix A.4.3 (Table 10) and Appendix A.4.6 (Table 14).

4. Expanded the analysis of resolution, crop coverage, and language cues. Table 11 now separates XLRS-Bench into 10K and lower-resolution subsets. EfficientViS improves [email protected] from 3.32% to 7.17% on 10K images and from 12.58% to 16.60% below 10K. We also report target-size distributions and center recall across language cues, explain the fixed crop size, and distinguish reaching the target center from observing its full extent. These analyses clarify both the value of local verification and the limits of global planning. Locations: Appendix A.4.4 (Table 11) and Appendices A.5.1 and A.5.2 (Tables 15 and 16).

5. Added a quantitative failure analysis and recovery directions. Among 5,554 failures on 6,309 valid XLRS-Bench instances, planner misses account for 77.76%, insufficient crop coverage for 11.99%, and local grounding or fusion errors for 10.24%. This identifies region selection as the main bottleneck. We discuss multi-scale windows, aggregation of evidence across crops, and additional search triggered by low confidence as future directions. Locations: Appendix A.6.1 (Table 17) and Appendix A.6.2.

6. Clarified evaluation protocols and corrected dataset reporting. We document the official splits and shared specialist evaluation protocol, explain the sources of published MLLM results, correct AVVG to 32,621 referring instances, and record the exclusion of one invalid XLRS-Bench annotation from instance-level diagnostics. Locations: Section 4.1 and Appendix A.1.1 (Table 6).

Our individual replies additionally report three-seed results for LQVG and EfficientViS, with mean accuracy gains of 1.34%, 2.19%, and 3.94% (Reviewer RJGd, W1/Q1), and a Planner-only comparison that tests the contribution of high-resolution local grounding (Reviewer yw1m, W4).

We sincerely thank the reviewers for their constructive feedback and valuable suggestions, which have helped us improve the clarity and rigor of our work.

Summary of the Review Discussion

Dear Program Chairs, Senior Area Chairs, Area Chairs, and Reviewers,

Thank you for the time and care you devoted to reviewing EfficientViS and coordinating the discussion. We were grateful for the reviewers' recognition of EfficientViS's results and the additional evidence provided during the discussion. Reviewer yw1m found the Planner-only comparison and failure analysis helpful, appreciated the clearer distinction between the two speedup comparisons, and noted that the rebuttal addressed most of their concerns. Reviewer 9ukp recognized the throughput gains and noted that the comparison at approximately matched compute strengthened the empirical case. Reviewer RJGd indicated that our responses addressed their concerns about the consistency of the accuracy gains and the cost of inference, and kept their positive assessment.

In our replies, we reported results from three independent seeds to check the consistency of the accuracy gains and used a Planner-only comparison to assess the contribution of local verification. The revised manuscript adds a comparison at approximately matched compute and a breakdown of runtime and memory use (Tables 10 and 14). Appendix A.4.5 explains how the sequential baseline uses local feedback and clarifies the two speedup comparisons. EfficientViS achieves 1.34 to 1.42 times the throughput of the sequential baseline with the same architecture. The speedup of more than 14 times over GeoViS-Search reflects the change in the full search procedure.

We also added analyses by image resolution and target size and a quantitative breakdown of failures (Tables 11, 13, and 17). On XLRS-Bench, EfficientViS improves grounding accuracy over full-image LQVG on 10K images and has higher center recall than sequential search in all four size groups up to 256 pixels. We clarified the conditions under which the theoretical bound motivates planning complementary regions before local verification (Appendix A.3.4).

Thank you again for your constructive feedback. We would be grateful if you would consider our responses, the additional evidence, and the revisions to the manuscript in your assessment.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.