- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper targets high-resolution remote sensing visual grounding. To avoid the latency of sequential, iterative visual search and the detail loss of full-image downsampling, it proposes EfficientViS to decouple candidate region planning from local verification. The model predicts candidate crops from a downsampled global view in one shot and evaluates them in parallel.
Contributions
OCRP: Generates an ordered set of candidate regions from a single downsampled global representation using spatial scoring and neighborhood suppression.
ACPI: Predicts an adaptive observation budget via a utility head and packs the selected crops into a single specialist forward pass for parallel local grounding.
Training & Theory: Employs a two-stage training strategy (crop-jitter adaptation followed by supervised bootstrapping and PPO policy tuning), backed by an information-theoretic bound arguing that local feedback provides limited planning value.
Validation
Evaluated on three remote sensing datasets spanning 1K to 10K resolutions (RSVG-HR, AVVG, XLRS-Bench) with LPVA and LQVG backbones. The method yields moderate localization gains (1.2 to 3.9 points in [email protected]) while providing reported 4.3–4.8x throughput improvements over full-scene baselines and over 14x over sequential search.
Critical strengths
Tangible inference throughput gains: The main merit of the paper lies in its practical throughput advantage over serialized visual search. Rather than invoking the specialist model sequentially across iterative steps, evaluating candidate crops in a single packed batch yields reported 4.3–4.8x speedups over full-scene baselines and over 14x over sequential search. This provides a pragmatic engineering solution to the severe latency bottleneck in high-resolution visual grounding.
Critical weaknesses
Circular theoretical bound: The information-theoretic argument against local feedback trivially begs the question. It assumes that residual spatial information is negligible to conclude that feedback has bounded utility. In practice, downsampling 8K images to a 256x256 thumbnail heavily degrades small targets, making local observations the primary source of discriminative signal. The core theoretical premise simply fails in real ultra-high-resolution settings.
Limited technical novelty: Strip away the "visual search" terminology, and the framework reduces to a standard two-stage cascade: coarse candidate selection on a downsampled thumbnail followed by batched crop verification. Layering PPO and a six-term reward on top of this feed-forward pipeline introduces heavy engineering complexity without offering new algorithmic insights into the localization problem.
Questions for authors
In Table 2, the baseline runs on 1024x1024 images, whereas EfficientViS processes a 256x256 thumbnail plus an average of 1.6–2.5 crops of 256x256—roughly one-fifth of the total pixels. How much of the reported 4.5x speedup is simply an artifact of processing far fewer pixels rather than parallel inference? What are the baseline's accuracy and FPS under an equivalent pixel or FLOPs budget?
The theoretical bound assumes that local observations provide negligible residual information over the global view. However, [email protected] on XLRS-Bench (8K–10K) is only 11.97%. What is the standalone recall of the 256x256 planner on XLRS? Specifically, what percentage of the ~88% failure cases stem from the planner missing the target entirely across the proposed crops?
7 comments
Response to Reviewer 9ukp (2/4)
Response to W2: Explanation of Technological Innovation
Thank you for raising this concern. The main contribution of EfficientViS is to decouple region planning from local verification, allowing high resolution local evidence to be processed in parallel without waiting for feedback from earlier crops.
Most existing visual search methods, including V*, GeoViS, ZoomEarth, and GeoSearcher, follow a sequential process. They use local observations both to decide where to look next and to perform final localization. Although this recovers fine details and improves grounding, each new region decision must wait for the preceding local inference. Repeated crop selection and grounding therefore limit computational efficiency.
To address this bottleneck, we explore decoupling global region planning from local fine verification, drawing inspiration from biological visual attention. Rapid feedforward visual processing provides early semantic information within approximately 150 ms, while global scene information helps guide subsequent selective inspection [1, 2]. The visual system can also distribute attention across several spatially separated locations. Studies of multiple attentional spotlights and multifocal tracking suggest a limited capacity, often around four attended targets [3 to 5]. Together with Inhibition of Return, which discourages revisiting recently attended locations, these mechanisms motivate planning a small set of spatially complementary candidates before detailed verification.
The reason humans exhibit sequential eye movements during fine verification is largely due to the physical constraint of a biological single fovea [6]. While existing visual search methods naturally inherit this sequential paradigm, our work takes a complementary perspective of "one-shot global gist planning + adaptive parallel local verification." GPU computation offers a way to inspect multiple image patches concurrently and this motivates combining complementary attention allocation with hardware parallelism. The candidate plan is therefore determined before local verification, avoiding the repeated waits between local inference and the next region decision.
This separation preserves the role of fine visual evidence. The specialist still receives small patches cropped directly from the original high resolution image and uses their textures, boundaries, and other local details for final localization. Only the use of intermediate local observations to redirect subsequent region selection is removed. We implement it through the following three aspects.
[1] Speed of processing in the human visual system. Nature, 381(6582), 520 to 522.
[2] Visual search in scenes involves selective and nonselective pathways. Trends in Cognitive Sciences, 15(2), 77 to 84.
[3] Parallel and serial neural mechanisms for visual search in macaque area V4. Science, 308(5721), 529 to 534.
[4] Multiple spotlights of attentional selection in human visual cortex. Neuron, 42(4), 677 to 686.
[5] Tracking multiple targets with multifocal attention. Trends in Cognitive Sciences, 9(7), 349 to 354.
[6] Optimal eye movement strategies in visual search. Nature, 434(7031), 387 to 391.
continued in Part 3/4
Response to Reviewer 9ukp (3/4)
1. Complementary Global Region Planning with OCRP
OCRP plans complementary candidate regions from a single global image and text representation before local grounding begins. Independent Top K selection can concentrate crops around the same distractor. Neighborhood suppression instead conditions each new candidate on the regions already selected, while continuous offsets refine its position. This spreads a limited observation budget across different spatial hypotheses without requiring local feedback. At the same budget of K = 4, complementary suppression improves RSVG-HR [email protected] from 86.57% to 88.38%, a gain of 1.81 percentage points.
2. Adaptive Budgeting and Parallel Verification with ACPI
Decoupling removes the need to inspect crops sequentially, but using the maximum crop count for every query would still waste computation. ACPI predicts the utility of ordered candidate prefixes and selects one to four crops before any local inference. It then packs the retained crops into a single specialist forward pass. On RSVG-HR, it matches the 88.58% accuracy of fixed K = 4 with only 1.63 crops on average, increasing throughput from 77.36 FPS to 122.01 FPS, a 57.7% improvement.
3. Policy Optimization with Training Only PPO
We optimize region selection and budget allocation using the final grounding outcome and computational cost. PPO handles the discrete planning decisions, while differential returns credit each additional crop according to its contribution. The six reward terms balance final IoU, target coverage, and local grounding quality against overlap, observation cost, and latency. Compared with the supervised adaptive baseline, PPO improves [email protected] by 0.40, 1.20, and 1.85 percentage points on RSVG-HR, AVVG, and XLRS-Bench, respectively. These rewards and policy updates are used only during training, so they introduce no extra computation at inference.
OCRP coordinates candidate regions, ACPI sets the crop budget, and PPO optimizes both decisions for localization quality and computational cost. This allows EfficientViS to preserve global context and fine local evidence while improving how candidate regions are coordinated, how many observations are acquired, and how the policy balances localization quality with computational efficiency.
Response to Q1: Attribution of the Speedup and Equal Compute Comparison
Thank you for raising this question. We would first like to emphasize that reducing computation through adaptive allocation of high-resolution visual processing is itself a key part of our methodological contribution. Specifically, EfficientViS achieves its speedup through two complementary mechanisms: processing fewer pixels through selective local crops and processing those crops in parallel. We matched the per sample GFLOPs of full scene LQVG to EfficientViS across all three benchmarks to separate these effects.
The table reports results on the same NVIDIA RTX 5090. The LQVG input resolutions were adjusted to 400 × 400, 448 × 448, and 544 × 544 to match EfficientViS compute.
Table 2: Accuracy and throughput comparison between the LQVG baseline and EfficientViS under equal computational budgets.
| Dataset | Method and Input Setting | Compute (GFLOPs/sample) | Throughput FPS | Accuracy [email protected] (%) |
|---|---|---|---|---|
| RSVG-HR | LQVG (downsampled to ) | 36.083 | 153.54 | 75.95 |
| EfficientViS | 36.123 | 122.01 | 88.58 (+12.63) | |
| AVVG | LQVG (downsampled to ) | 42.539 | 142.97 | 27.56 |
| EfficientViS | 43.717 | 136.84 | 32.96 (+5.40) | |
| XLRS-Bench | LQVG (downsampled to ) | 69.606 | 129.36 | 6.15 |
| EfficientViS | 70.214 | 128.46 | 11.97 (+5.82) |
Under matched compute, downsampled LQVG is faster because it avoids crop scheduling and coordinate fusion. This shows that pixel reduction contributes to the speedup. However, directly reducing the full-scene resolution causes severe loss of fine-grained visual features, leading to a sharp drop in localization accuracy. EfficientViS preserves those details in local crops and significantly improves [email protected] by 12.63, 5.40, and 5.82 points across the three benchmarks.
Therefore, the core advantage of EfficientViS lies in combining efficient allocation of computing resources with parallel execution, thereby achieving a better balance between positioning accuracy and inference efficiency.
Response to Reviewer 9ukp (1/4)
Thank you for your thorough review and for recognizing EfficientViS's throughput gains. We addressed the scope of our information theoretic bound, the distinction from existing global and local methods, the sources of speedup, and planner failures on XLRS-Bench to address your concerns, as detailed below.
Response to W1: Conditional Meaning of the Information Theoretic Bound and Feedback Free Planning
Thank you for your careful examination of our theoretical analysis. Your comment helped us realize that the phrasing in Section 3.2 of the original manuscript could mislead readers into assuming that our theorem relies on a prior assumption that "the residual spatial information in ultra-high-resolution imagery is inherently small." Below, we clarify this point from three perspectives: mathematical logic, functional decoupling across modules, and the practical behavior of lightweight planners.
1. Meaning of the Bound
Corollary 1 follows from conditional mutual information, Pinsker's inequality, and Jensen's inequality. For any planning utility normalized to [0,1] and any nonnegative conditional mutual information Ij between the next target region and previous local observations given Z, it gives the following unconditional bound.
0≤Vfb,j∗−Vff,j∗≤2Ij
The bound does not claim that Ij is small in every ultra high resolution scene. It states that when the global cross modal representation Z already narrows the plausible regions, prior local observations O<j may add limited information about where to search next. In that case, even an ideal Bayes optimal feedback planner has a bounded utility gain over feedback free planning. When tiny targets or weak reference cues are lost in the global view while the planner can use the local information effectively, Ij can be large.
2. Global Planning and Local Patch Evidence
Local detail is still used for localization and it is simply not fed back to change the remaining crop plan. The global representation Z determines where to look by proposing complementary regions and an observation budget before local inference. The second stage extracts small patches directly from the original high resolution image and sends them to the specialist grounder for fine grained discrimination and final box regression.
3. Lightweight Planner and Controlled Sequential Comparison
Corollary 1 describes an ideal Bayes optimal planner. Our deployed planner has only 1.08M parameters, less than 1% of the specialist model. Unlike MLLMs, which can draw on broad pretrained knowledge and reasoning to reassess their predictions, the lightweight planner has limited capacity to detect and correct early search errors. In dense scenes with similar looking objects, an early false positive can mislead this lightweight planner and affect every later crop (as shown in Figure 6). To test this effect, we compare EfficientViS with a sequential baseline using the same backbone, planner, crop size, and training hyperparameters. The sequential baseline feeds each crop's local features back to select the next region. The only changed factor is the planning process. Table 1 reports target center coverage by object size.
Table 1: Comparison of target center coverage rates across different object scales on XLRS-Bench.
For targets no larger than 256 px, which make up 80.7% of the dataset, parallel planning has higher center coverage in every size group. Its overall recall is 31.54% compared with 29.50% for sequential search, with the largest difference on targets no larger than 32 px. This is consistent with the risk that a lightweight sequential planner follows an early false positive in a visually ambiguous crop. Parallel planning considers complementary regions before any local prediction can misdirect it.
For targets larger than 256 px, sequential search has 3.13 points higher center coverage. Local feedback can therefore help follow large objects across regions. The results show where feedback is useful while confirming higher overall recall for parallel planning.