acceptodds
Under review as a conference paper at ICLR 2027

The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understanding

Abstract

Multimodal large language models (MLLMs) must balance local detail against scene context when interpreting ultra-high-resolution (UHR) remote sensing (RS) imagery within a limited visual-input budget. Existing selection-based methods either prune tokens and select patches through relevance scoring, or crop actively through repeated inspection. Neither strategy directly redistributes pixels within a continuous full-scene view: the first retains selected tokens or patches, and the second re-encodes a crop detached from its surroundings. Our pilot study finds that a frozen MLLM already produces useful question-guided spatial requests, yet crop-based inspection of the selected regions does not consistently improve its answers. We therefore formulate UHR understanding as a question of where to spend a fixed pixel budget. Based on this, we introduce GazeEarth, a simple-yet-effective training-free framework that couples question-guided region selection with full-scene foveated observation. The MLLM selects evidence cells from an indexed overview; a deterministic, topology-preserving warp resamples the original image onto a fixed-size canvas, enlarging their shared neighborhood while compressing the periphery; the same frozen model answers from this focused view, using at most two MLLM calls and no external selector or iterative search. Across three UHR remote sensing benchmarks and four frozen backbones, GazeEarth improves benchmark-averaged accuracy by 4.6 to 9.4 percentage points over direct answering and 3.4 to 4.3 over overview answering, outperforming task-trained methods. Our analyses show that existing MLLMs can guide where to look in UHR images on their own, and that what they can infer from the selected evidence depends on how that evidence is presented.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal large language models, ultra-high-resolution image understanding, active visual perception, foveated vision, training-free inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_svEJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GazeEarth targets UHR remote sensing VQA under a fixed input budget. Two probes set it up: a frozen MLLM can usually name the grid cell a question points at, but feeding back that crop costs 9.2 points on MME-RealWorld-RS against a plain resized overview. Presentation, not localisation, is the claimed bottleneck. The model picks up to two cells of a 4x4 overview, a separable piecewise linear warp enlarges their enclosing rectangle inside a 2048 pixel canvas while compressing the periphery, and the same model answers from that view. Training free, two calls, three benchmarks, four backbones.

Critical strengths
  1. Table 7 is the useful experiment: one region, four renderings, same answerer.
  2. Cheap, training free, unchanged across four backbones, and the statistics are done properly.
Critical weaknesses
  1. Backbones differ throughout Tables 3 and 4 (ZoomEarth 3B, most RS models 7B, GazeEarth on Qwen3-VL-8B), so beating task trained methods proves less than claimed. Against Overview on the same model the gains are +8.38, +3.42 and +1.01, the last not surviving correction, and 4.07 of the 8.34 point mean gain over Direct is canvas normalisation, which is plain resizing.
  2. With δ = 0.25, α = 1.5 and β = 0.55 the focus grows about 1.5x linearly over a uniform resize, taking a 20 pixel object on a 10k scene to roughly 6 pixels. Thin basis for 8 to 10 points, and counting agrees: 21.78 on MME-RealWorld-RS, up 0.43 over Direct, behind ZoomEarth at 31.66. Stratify by target size, and report how often a request ends as global_only or brief_incomplete rather than a real focus. Two distant cells also give an enclosing rectangle covering most of the scene, so the case that motivates K = 2 gets almost no enlargement.
  3. fx and fy are applied independently, so two cross shaped bands come out stretched on one axis and squeezed on the other, and aspect ratio is a category cue for runways and ships. The prompt never says the canvas is geometrically non uniform, yet many questions ask about position on it. Section 3.3 overstates things: a product of two monotone maps is a homeomorphism by construction, and Eq. 6 is an identity.
  4. Two-Gaze and Three-Gaze reselect on an already warped canvas and warp it again, so later rounds undo earlier ones and nothing accumulates. Degradation is what that design predicts; the XLRS-Bench fall from 53.51 to 46.27 reads as composition damage, not evidence that one fixation suffices. A fair counterpart would carry information across fixations and stop when it has enough. AdaptiveNN (Wang et al., Emulating human like adaptive vision for efficient and flexible machine visual perception, Nature Machine Intelligence 2025) does that, with sequential fixations, incremental aggregation and an explicit stopping rule under an adjustable budget. Even a paragraph of discussion would support the single gaze claim better than the present baseline.
  5. The 4x4 grid, K = 2, α = 1.5 and the 1536 pixel selector are each the argmax of the mean over the evaluation sets, the sweeps are not monotone (XLRS-Bench: 50.97, 53.51, 50.49), and the spread rivals the margin over the baselines. No held out split. The 77.74% alignment is partly circular too, since reference cells come from spatial phrases in the question itself.
Questions for authors

See Weakness

11 comments
AuthorsOpenReview ↗

Response to Reviewer svEJ Part (1/8)

Thank you for recognizing our informative ablations, rigorous statistical evaluation, and efficient, training-free design. We address your concerns in their original order below.

W1 · Backbones, significance, and gain baselines

(1) Backbone comparisons. Thank you for your constructive advices. To address the backbone difference, we added experiments on Qwen2.5-VL-3B, used by ZoomEarth, and Qwen2.5-VL-7B, used by GeoLens. GazeEarth remains training-free on both and achieves competitive performance on the same backbones as these trained methods.

Table S1. Direct and GazeEarth on the same 3B and 7B backbones; accuracy (%).

ModelLRS-GROMME-RealWorld-RSXLRS-BenchMean accuracy (%)
Qwen2.5-VL-3B24.3132.5636.2331.03
GazeEarth-3B39.7536.4642.9939.73
Gain+15.44+3.90+6.76+8.70
Qwen2.5-VL-7B48.5540.9041.9843.81
GazeEarth-7B53.3744.3852.9950.25
Gain+4.82+3.48+11.01+6.44

GazeEarth improves over Direct on all three benchmarks, with mean gains of 8.70 percentage points (pp) on Qwen2.5-VL-3B and 6.44 pp on Qwen2.5-VL-7B. We then reproduced the training of ZoomEarth and GeoLens on Qwen3-VL-8B to compare the methods on the same newer backbone.

Table S2. Trained methods and GazeEarth on Qwen3-VL-8B; accuracy (%).

MethodBackboneLRS-GROMME-RealWorld-RSXLRS-BenchReported mean
GeoLensQwen3-VL-8B57.4652.0142.0150.49
ZoomEarthQwen3-VL-8B55.8431.2535.0340.71
GazeEarthQwen3-VL-8B61.0052.4353.51—

In Table S2, — denotes an unreported mean. The comparison on the same backbone shows that training-free GazeEarth scores higher than both retrained methods on all three benchmarks. The retraining results also indicate that the original training data and recipes do not transfer readily to Qwen3-VL-8B. Some results even fall below Direct: on MME-RealWorld-RS, retrained ZoomEarth scores 31.25%, compared with 42.11% for Direct on the same backbone. As discussed in Section 2, current MLLMs already have strong visual capabilities. Together, the retraining results and GazeEarth's training-free gains support our central argument in Section 2. How visual evidence is presented may therefore matter more for UHR understanding than task-specific training. We agree that the benefit of our presentation mechanism depends on the vision encoder's perceptual capacity and will discuss this dependence. Nevertheless, the consistent improvements across the frozen 3B, 7B, and 8B backbones support the broader applicability of GazeEarth.

(2) Statistical significance. As you note, the LRS-GRO gain over Overview does not remain statistically significant after Holm correction (Table 21 in the paper). We agree and have revised the corresponding statement. The gains on XLRS-Bench and MME-RealWorld-RS remain significant under the same test. Thank you again for your detailed discovery.

(3) Ablation baselines. Direct serves as a raw-input baseline, with no image processing by our pipeline, including resizing. Overview applies uniform canvas normalization. We agree that the gain descriptions should distinguish normalization from the additional benefit of directed focus, and have revised them accordingly.

W2 · Magnification, target size, counting, and request status

(1) Magnification, target size, and counting. We agree that the nominal focus strength alone cannot explain the gain over Direct, which also includes canvas normalization. The comparison with Overview measures the additional benefit of directed focus. We have revised the main text to distinguish these contributions.

We stratified questions by target size in the source image, before resizing or warping. For a target box of width ww and height hh, we define its size as s=whs=\sqrt{wh}. Following the COCO thresholds, Small uses s<32s<32 px, Medium uses 32≤s<9632\leq s<96 px, and Large uses s≥96s\geq96 px. Tiny (s<16s<16 px) is reported separately as a subset of Small. For XLRS-Bench, Table S3 uses the same 1,660 coordinate-resolved questions and original-pipeline prediction file as the coordinate-subset results in Tables S4 and S5. These questions account for 53.90% of the benchmark. On LRS-GRO, 7,394 questions (75.96%) have valid annotated object or region boxes. MME-RealWorld-RS does not provide target boxes for this analysis.

AuthorsOpenReview ↗

Response to Reviewer svEJ Part (2/8)

Table S3. Answer accuracy by target size in the source image.

BenchmarkNative target sizeQuestionsOverview acc. (%)GazeEarth acc. (%)Gain (pp)
XLRS-BenchSmall (<32 px)32335.9143.96+8.05
XLRS-BenchMedium (32–<96 px)70937.3846.54+9.17
XLRS-BenchLarge (≥96 px)62857.0165.13+8.12
XLRS-BenchAll coordinate-resolved questions1,66044.5253.07+8.55
XLRS-Bench↳ Tiny (<16 px), included in Small9327.9636.56+8.60
LRS-GROSmall (<32 px)3167.7464.52−3.23
LRS-GROMedium (32–<96 px)86557.6959.19+1.50
LRS-GROLarge (≥96 px)6,49853.8955.19+1.29
LRS-GRO↳ Tiny (<16 px), included in Small0———

On the coordinate-resolved XLRS-Bench subset, gains are comparable across Small, Medium, and Large targets. On LRS-GRO, GazeEarth improves on Medium and Large targets. The Small group contains only 31 questions: GazeEarth answers 20 correctly and Overview answers 21.

Your 20-pixel example identifies a real resolution limit. Resizing a 10,000-pixel image axis to 2,048 pixels reduces a 20-pixel target to about 4.1 pixels along that axis. Even with the full 1.5× axial enlargement, it spans only about 6.1 pixels. On the 93 XLRS Tiny questions, GazeEarth answers 34 correctly (36.56%), compared with 26 (27.96%) for Overview. This recovers eight additional answers, but absolute accuracy remains low. These questions represent 5.60% of the coordinate-resolved subset and 3.02% of the full XLRS benchmark. We found no Tiny targets in the annotated LRS-GRO subset; MME lacks the boxes needed to estimate their prevalence. Meanwhile, Table 11 in our article also verifies this point. In this table, we can see that Self-MLLM still achieved the best on 537 questions without specific direction evidence from LRS-GRO. Among them, Center hit reached 64.99%, Box coverage reached 55.65, which is significantly better than other methods, and the answer accuracy rate of Oracle Gaze is similar (Self- MLLM: 66.29, Oracle Gaze: 70.20).

Exhaustive counting exposes a related coverage limitation. On the 1,226 MME counting questions, GazeEarth answers 267 correctly (21.78%), compared with 225 (18.35%) for Overview. Trained ZoomEarth remains stronger on this category, as you note. When instances are scattered across the image, a single focus enlarges one neighborhood while other instances remain at overview resolution. Trained methods with multi-step zooming can cover more instances; limited coverage is a cost of our single-gaze, training-free design. The selector also declares brief_incomplete more often on counting questions than on other MME questions (15.25% versus 5.53%). We will discuss this limitation and identify counting-oriented scanning as a direction for future work.

AuthorsOpenReview ↗

Response to Reviewer svEJ Part (3/8)

(2) Request outcomes and actual enlargement. We counted global_only and brief_incomplete requests, their proportions, and their conditional answer accuracies across every question in all three benchmarks. The statistics also include valid selections, invalid outputs, and XLRS-Bench requests whose focus is determined by question coordinates. We separately inspected the final rendered views to identify all cases without actual enlargement.

Table S4. Request outcomes and conditional answer accuracy across all three benchmarks.

Request outcomeXLRS-Bench, n (%)Accuracy (%)MME-RealWorld-RS, n (%)Accuracy (%)LRS-GRO, n (%)Accuracy (%)
Coordinate-derived focus1,660 (53.90)53.070 (0.00)—0 (0.00)—
Valid model selection1,193 (38.73)54.403,295 (88.15)56.338,884 (91.27)59.95
global_only90 (2.92)46.6754 (1.44)40.74489 (5.02)79.75
Declared brief_incomplete112 (3.64)52.68326 (8.72)16.56117 (1.20)53.85
Invalid response25 (0.81)68.0063 (1.69)44.44244 (2.51)65.16
Full benchmark3,080 (100.00)53.513,738 (100.00)52.439,734 (100.00)61.00
global_only or declared brief_incomplete202 (6.56)50.00380 (10.17)20.00606 (6.23)74.75
No realized enlargement349 (11.33)52.44477 (12.76)24.531,498 (15.39)75.43

The first five rows are mutually exclusive request outcomes, and the last two rows are summaries. Every proportion uses the full benchmark as its denominator; accuracy is conditional on the questions in that row. The no-enlargement summary also includes invalid responses and valid selections that produced no enlargement. Request status and realized enlargement are therefore different quantities.

Only XLRS-Bench contains questions that trigger coordinate-derived focus from coordinates supplied in the prompt. The compared baselines use the same coordinate-handling policy as GazeEarth. Table S5 reports the control that forces grid selection on every XLRS-Bench question.

Table S5. Effect of forcing grid selection on all XLRS questions.

XLRS evaluation groupQuestionsCoordinate policy (%)Forced grid selection (%)Paired difference (pp)
Full benchmark308053.5150.13−3.38
Questions with resolved coordinates166053.0746.81−6.26
Questions without resolved coordinates142054.0154.010.00

Forcing grid selection lowers accuracy on coordinate-bearing questions and leaves the other questions unchanged. We will report this measurable contribution separately from autonomous gaze selection.

Bypass and conditional routing are common implementation practices in related UHR work. GazeEarth's coordinate bypass is one instance of this practice. Public implementations of ZoomEarth, GeoLens, and WeaveEarth also provide such options; the exact files and line numbers are listed below.

Reviewer_Y5m6OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GazeEarth, a training-free framework for ultra-high-resolution remote sensing visual question answering that pairs a frozen multimodal large language model with a deterministic image warping operator. In the first stage, the model inspects a downsampled overview overlaid with a 4x4 grid to select up to two target cells, which are then enlarged within a fixed-size full-scene canvas via a separable piecewise-linear warp before a second call to the same model generates the final answer. Experiments across three remote sensing benchmarks (LRS-GRO, MME-RealWorld-RS, and XLRS-Bench) and four frozen backbones compare this pipeline against direct inference, uniform downsampling, crop-based active vision baselines, and external retrievers.

Critical strengths

The paper separates the problem of locating relevant regions from the problem of presenting those regions in context, showing through controlled comparisons against uniform overview baselines that isolated cropping can degrade spatial reasoning on context-heavy benchmarks. The proposed pipeline is evaluated consistently across three benchmarks and four frozen backbones, and the appendix provides detailed breakdowns of latency and canvas normalization effects.

Critical weaknesses
  1. Section 3.2 and Appendix B.1 reveal that whenever a question contains bounding-box coordinates, a hard-coded parser extracts the box, expands it into a contextual square, and bypasses model-guided gaze selection entirely. The paper neither reports the fraction of benchmark questions handled by this rule-based bypass nor applies the same coordinate extraction to the baselines, confounding the claimed accuracy and latency gains of self-directed gaze.
  2. Although the selection prompt requests up to two grid cells to support two-region comparisons, Stage 2 replaces multiple selected cells with their single smallest enclosing rectangle before applying a three-segment axial warp capped at 55 percent of the canvas. When two requested cells lie far apart, their enclosing box spans most of the scene, which either magnifies irrelevant background between them or hits the 55 percent span cap and prevents magnification along that axis.
  3. In the main localization probe, reference regions are derived from explicit directional phrases in the questions (such as "top right corner"), whereas external retrievers like SigLIP2 score isolated cell crops that lack global frame coordinates. Furthermore, the multi-gaze ablation in Table 7 re-warps the already downsampled and compressed canvas rather than resampling from the original high-resolution image, creating a degraded baseline that cannot fairly evaluate iterative selection.
  4. Appendix A specifies that the selector outputs only compact JSON without reasoning and the answerer receives an ungridded image without the selector's conversation history. However, Figures 1, 3, 4, and 15 through 20 display first-person reasoning traces that explicitly reference grid identifiers and pixel reallocation actions, presenting author-written workflow summaries as if they were actual model outputs.
Questions for authors
  1. Report the exact proportion of questions in each benchmark that trigger the bounding-box coordinate bypass, and evaluate GazeEarth when model-guided grid selection is enforced on all questions.
  2. Report the frequency of non-adjacent two-cell selections across the three benchmarks and quantify the effective area magnification factor achieved on those samples after enclosing-rectangle merging.
  3. Evaluate Two-Gaze and Three-Gaze when subsequent selections accumulate or refine regions in source coordinates and resample directly from the original image rather than re-warping the intermediate canvas.
  4. Replace the synthesized first-person narratives in Figures 1, 3, 4, and 15 through 20 with the verbatim JSON outputs from the selection stage and the raw text responses from the answering stage.
7 comments
AuthorsOpenReview ↗

Response to Reviewer Y5m6 Part (1/4)

Thank you for recognizing our insightful problem formulation, well-controlled comparisons, and comprehensive empirical evaluation. We address your requested protocol controls and implementation details below.

W1/Q1 · Coordinate bypass and inference costs

The compared baselines use the same coordinate-handling policy as GazeEarth. Thus, unequal coordinate handling does not confound the accuracy comparison. We also quantified the coordinate route and completed the requested forced-selector control. Only XLRS-Bench triggers the coordinate bypass: 1,660 of 3,080 questions (53.90%) contain coordinates supplied by the question itself. MME-RealWorld-RS and LRS-GRO have no such cases. Disabling the coordinate shortcut in region selection and otherwise keeping the inference procedure unchanged reduces full XLRS-Bench accuracy from 53.51% to 50.13% (−3.38 pp). This route makes a measurable contribution, which we will report separately from autonomous gaze selection.

Table Y1. Enforcing grid selection on XLRS-Bench.

Evaluation groupQuestionsOriginal pipeline accuracy (%)Forced grid selection (%)Change (pp)
Full benchmark3,08053.5150.13−3.38
Questions with coordinates1,66053.0746.81−6.26
Questions without coordinates1,42054.0154.010.00

The loss is confined to coordinate-bearing questions; questions without coordinates are unaffected.

For the cost comparison, all reported latency measurements use the complete inference flow, including both selection and answering, without bypassing the selector. Coordinate bypass therefore provides no speed advantage in this latency comparison. For completeness, the actual calls across full-benchmark runs with all routes enabled are reported below.

Table Y2. Coordinate bypass and realized One-Gaze calls.

BenchmarkQuestionsCoordinate bypass, n (%)Selector callsAnswerer callsTotal callsCalls/question
XLRS-Bench3,0801,660 (53.90%)1,4203,0804,5001.461
MME-RealWorld-RS3,7380 (0%)3,7383,7387,4762.000
LRS-GRO9,7340 (0%)9,7349,73419,4682.000

These call counts describe runs with all routes enabled, separately from the full-pipeline runs used to measure latency.

Bypass and conditional routing are common implementation practices in related UHR work. GazeEarth's coordinate bypass is one instance of this practice. Public implementations of ZoomEarth, GeoLens, and WeaveEarth also provide such options; the exact files and line numbers are listed below.

Table Y3. Bypass and conditional routing in related UHR implementations.

MethodImplementation and lines
ZoomEarthsrc/eval/infer.py, lines 233-247
GeoLensevaluation/xlrs_tool_usage/run_geolens_tool_agent_xlrs.py, lines 39-50 and 271-276
WeaveEarthinference_scripts/eval_lrs_vqa_qwen3vl.py, lines 717-729

We will discuss these implementation details alongside the separate coordinate-ablation and full-pipeline latency results.

AuthorsOpenReview ↗

Response to Reviewer Y5m6 Part (2/4)

W2/Q2 · Enclosing rectangles and non-adjacent selections

We measured how much enlargement remains when two selected cells are far apart. We considered all requests selecting two cells in the same image and defined non-adjacent cells as sharing neither an edge nor a corner. For each non-adjacent request, we computed the area of the merged focus rectangle after warping relative to its area under uniform resizing.

Table Y4. Frequency and realized area magnification of non-adjacent two-cell selections.

BenchmarkSame-image two-cell requestsNon-adjacent requestsShare of same-image two-cell requestsShare of full benchmarkMedian area magnification on non-adjacent requestsNon-adjacent requests with no area gain
XLRS-Bench38019952.37%6.46%1.50×64/199 (32.16%)
MME-RealWorld-RS1605534.38%1.47%1.50×14/55 (25.45%)
LRS-GRO2,07396246.41%9.88%1.50×337/962 (35.03%)

The median area magnification is 1.50× on all three benchmarks. However, 25.45%–35.03% of non-adjacent requests receive no area gain, confirming the enlargement limit you identify. The enclosing rectangle retains both targets and the scene between them. Under a fixed canvas budget, this requires a trade-off between target detail and inter-target context.

To test how this trade-off affects answering, we compared three presentations on all two-cell questions in each benchmark (Table Y5). We held the selected cells, answer model, prompt, and final canvas size fixed and changed only the presentation. Linked Overview + Union Crop pairs the global view with the smallest rectangle enclosing both cells and the intervening region. Linked Overview + Separate Crops pairs the global view with the two isolated cells placed side by side.

Table Y5. Accuracy on all two-cell questions (%).

BenchmarkQuestionsGazeEarth WarpLinked Overview + Union CropLinked Overview + Separate Crops
XLRS-Bench38044.4741.3240.79
MME-RealWorld-RS16045.6340.6335.63
LRS-GRO2,07365.6564.3063.43

GazeEarth Warp achieves the highest accuracy on all three benchmark subsets. These results support our argument that continuous warping effectively balances pixel allocation between the targets and the context connecting them. We will add this analysis to the manuscript.

W3/Q3 · Localization fairness and cumulative multi-gaze

For a fair comparison, we appended each crop's global position to the text input supplied to SigLIP2 and RemoteCLIP. They also received the spatial phrases in the question. We agree that this differs from an MLLM's joint view of the whole scene, but that difference arises from the architectures themselves. The ability to map spatial language to visual regions is exactly what a selector needs. The comparison therefore supports using the MLLM itself for region selection, since it can associate the spatial query with the full image layout.

We also conducted a localization probe on 537 target questions without explicit location cues from 159 LRS-GRO images. Each selector was required to choose exactly two cells per question (Table Y6). Ground-truth annotation boxes were withheld from every selector and used only for evaluation. The question-only and random controls test how much localization depends on visual input.

AuthorsOpenReview ↗

## Response to Reviewer Y5m6 Part (3/4)

Table Y6. Controlled localization probe.

GridSelector input/modelCenter hit (%)Box coverage (%)Normalized center error ↓
4×4Image + question (MLLM)77.8472.400.0902
4×4Question only (MLLM)11.1712.000.2669
4×4Random12.5012.500.2364
4×4RemoteCLIP51.0247.160.1406
4×4SigLIP271.8866.730.0981
8×8Image + question (MLLM)56.4248.250.0946
8×8Question only (MLLM)2.051.740.3517
8×8Random3.1253.1250.2421
8×8RemoteCLIP39.8433.820.1258
8×8SigLIP255.8747.800.1026
16×16Image + question (MLLM)21.7916.520.1505
16×16Question only (MLLM)1.300.970.3084
16×16Random0.780.780.2436
16×16RemoteCLIP19.7413.900.1708
16×16SigLIP222.5318.510.1382

Image + question (MLLM) outperforms Question only (MLLM) and Random at every grid size, supporting the contribution of visual scene content. The center-hit rate nevertheless falls as the grid becomes finer. The MLLM exceeds both retrievers at 4×4 and 8×8, and exceeds RemoteCLIP but slightly trails SigLIP2 at 16×16. These limits motivate coarse 4×4 selection with padding and retained scene context.

We agree with your concern about repeatedly warping an intermediate canvas. Following your suggestion, we evaluated cumulative Two-Gaze and Three-Gaze with earlier selections retained in source coordinates. Every new view is resampled directly from the original high-resolution image. Iteration stops when no new region is selected or the selector returns a non-selection status, preserving the previous answer.

Table Y7. Accuracy of cumulative source-resampled multi-gaze (%).

BenchmarkOne-Gaze (%)Two-Gaze (%)Three-Gaze (%)
XLRS-Bench53.5151.6951.79
MME-RealWorld-RS52.4350.2950.27
LRS-GRO61.0057.7857.80

On XLRS-Bench, cumulative Two-Gaze improves by 5.42 points over the original recursive variant, but remains 1.82 points below One-Gaze. Two-Gaze corrects 92 wrong answers but overturns 148 correct ones on XLRS-Bench. Our interpretation is that longer context across gazes increases the model's reasoning and perceptual burden. In addition, only 4.16%, 0.94%, and 0.59% of questions reach a third answer on the three benchmarks, respectively, explaining the small changes from Two-Gaze to Three-Gaze.

To quantify the additional inference work, Table Y8 reports extra calls relative to One-Gaze. The complete One-Gaze counts are provided in Table Y2.

Table Y8. Additional calls relative to One-Gaze, separated by stage.

BenchmarkSettingExtra selector callsExtra answerer callsTotal extra callsExtra calls/question
XLRS-BenchTwo-Gaze3,0801,7654,8451.573
XLRS-BenchThree-Gaze4,8451,8936,7382.188
MME-RealWorld-RSTwo-Gaze3,7384494,1871.120
MME-RealWorld-RSThree-Gaze4,1874844,6711.250
LRS-GROTwo-Gaze9,73479510,5291.082
LRS-GROThree-Gaze10,52985211,3811.169

The additional gazes increase inference cost, yet One-Gaze achieves higher accuracy on all three benchmarks. These results support the effectiveness of GazeEarth's design: the MLLM's own localization can identify useful evidence for answering within a single gaze.

Reviewer_AviTOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies ultra-high-resolution (UHR) remote sensing understanding with frozen MLLMs. A pilot study first decouples evidence localization from evidence presentation: a frozen MLLM's spatial requests align with question-relevant regions (77.74%) better than CLIP-based retrievers, yet presenting the selected region as an isolated crop does not reliably improve answering (−9.2pp on MME-RealWorld-RS). Motivated by this, the authors propose GazeEarth, a training-free framework in which the frozen MLLM selects up to two cells on a 4×4 indexed overview, a deterministic topology-preserving piecewise-linear warp reallocates a fixed pixel budget to enlarge the selected neighborhood within a single full-scene canvas, and the same model answers from this foveated view, using at most two MLLM calls. GazeEarth attains the best overall scores on LRS-GRO, MME-RealWorld-RS, and XLRS-Bench across four frozen backbones.

Critical strengths
  1. The pilot study (Sec. 2, Appendix C) is well designed: separating localization from presentation is a useful diagnostic, and the negative result—that question-guided cropping can degrade answering—is informative for the "thinking with image" line of work.
  2. The method is simple, training-free, deterministic, and cheap (≤2 calls, ~5s), with a clean formulation as budget-preserving foveated resampling.
  3. The experimental hygiene is above average: same-backbone Direct/Overview controls, observer substitution, presentation ablations, cost analysis, an honest decomposition of the total gain (+4.07pp from canvas normalization, +4.27pp from directed focus), and an oracle-gaze upper bound.
Critical weaknesses
  1. Backbone confound in the headline claim. The claim of outperforming task-trained UHR methods compares GazeEarth on the newest 8B backbone (Qwen3-VL) against methods on older/smaller backbones (ZoomEarth: 3B; GeoLens/Coarse-to-Fine: 7B), with some rows taken from original papers (†). In the authors' own tables, the plain Direct baseline of Qwen3-VL already exceeds several of these trained methods. The clean, same-backbone contribution is +4.27pp over Overview—solid, but considerably more modest than the abstract-level claim.
  2. The central diagnosis is overgeneralized. (a) The 77.74% alignment is computed only on eligibility-filtered questions whose reference cells are parsed from explicit spatial cues in the question, at a coarse 4×4 granularity; on questions without location cues, the center-hit rate against ground-truth boxes is only 64.99%. "The answerer can already guide where to look" is thus unsupported in the no-cue regime. (b) "Crops do not help" is benchmark-asymmetric: Gaze+Crop improves over Overview on XLRS-Bench (+5.4pp) and is flat on LRS-GRO, with the −9.2pp drop concentrated in MME-RealWorld-RS—the benchmark dominated by the context-dependent categories that foveation later benefits most. The diagnosis and the remedy close on the same sub-population, which weakens the claim of a general bottleneck.
  3. Roughly half of the total gain is not from gaze. Of the mean +8.34pp over Direct, +4.07pp comes from uniform canvas normalization (Overview); the net effect of question-guided focus is +4.27pp. The abstract's "4.6–9.4pp" framing invites over-interpretation.
  4. The warp trades one distortion for another, and this is never isolated. Topology (order/connectivity) is preserved, but metric geometry is not: distances, areas, and directions across the focus boundary are altered. No experiment measures this cost; consistently, MME-RS Position is slightly below Overview and XLRS OSR remains weak (37.80). Moreover, the min-enclosing-rectangle focus dilutes allocation when two distant cells are selected; counting/global questions are routed back to the global view by a prompt rule; and two/three-gaze variants hurt XLRS-Bench by ~7pp without discussion or explanation, which sits awkwardly with the "active focus" framing. There is no limitations section.
Questions for authors
  1. What fraction of questions per benchmark receive "global_only" or "brief_incomplete" (i.e., the pipeline degenerates to Overview), and what is the accuracy conditioned on each request status?
  2. Can you provide a controlled comparison that separates framework from backbone—e.g., GazeEarth and Direct/Overview on the same backbones as the trained baselines?
  3. Why does iterative gazing degrade XLRS-Bench by ~7pp? Please provide an error analysis; this bears directly on whether "active" focus is the right framing.
  4. Is there any evaluation of the geometric distortion introduced by the warp on metric spatial reasoning (distance/area/direction), or a control against a distortion-free presentation at matched detail (Linked-Global+Crop is already close on XLRS-Bench: 52.56 vs. 53.51)?
  5. Does the localization conclusion survive a finer grid (8×8 or 16×16), where cell selection approaches true grounding?
8 comments
AuthorsOpenReview ↗

Response to Reviewer AviT Part (1/6)

Thank you for recognizing our insightful diagnosis, rigorous experimental design and informative ablations, and simple, efficient, training-free approach. We address each concern below.

W1/Q2 · Backbone comparisons

To address the backbone confound, we added experiments on Qwen2.5-VL-3B (the ZoomEarth backbone) and Qwen2.5-VL-7B (the GeoLens backbone). GazeEarth remains training-free on both models and achieves competitive performance on the backbones used by the trained methods.

Table A1. Direct and GazeEarth on the same 3B and 7B backbones; accuracy (%).

ModelLRS-GROMME-RealWorld-RSXLRS-BenchMean accuracy (%)
Qwen2.5-VL-3B24.3132.5636.2331.03
GazeEarth-3B39.7536.4642.9939.73
Gain+15.44+3.90+6.76+8.70
Qwen2.5-VL-7B48.5540.9041.9843.81
GazeEarth-7B53.3744.3852.9950.25
Gain+4.82+3.48+11.01+6.44

GazeEarth improves accuracy across all three benchmarks on both backbones. The mean gains over Direct are 8.70 percentage points on 3B and 6.44 points on 7B. We then reproduced the training of ZoomEarth and GeoLens on Qwen3-VL-8B to compare all three methods on the same newer backbone.

Table A2. Trained methods and GazeEarth on Qwen3-VL-8B; accuracy (%).

MethodBackboneLRS-GROMME-RealWorld-RSXLRS-BenchReported mean
GeoLensQwen3-VL-8B57.4652.0142.0150.49
ZoomEarthQwen3-VL-8B55.8431.2535.0340.71
GazeEarthQwen3-VL-8B61.0052.4353.51—

Without task-specific training, GazeEarth scores higher than both retrained methods on all three benchmarks in our Qwen3-VL-8B reproduction (Table A2). Its advantage in these comparisons therefore persists when the backbone is held fixed.

The retraining results indicate that the original training data and recipes do not transfer readily to Qwen3-VL-8B. In some cases, accuracy even falls below Direct: on MME-RealWorld-RS, retrained ZoomEarth scores 31.25%, compared with 42.11% for Direct on the same backbone. As discussed in Section 2, current MLLMs already have strong visual capabilities. Together, the retraining results and GazeEarth's training-free gains support our central argument in Section 2. How visual evidence is presented may therefore matter more for UHR understanding than task-specific training. We agree that the benefit of our presentation mechanism depends on the vision encoder's perceptual capacity and will discuss this dependence in the revised paper. Nevertheless, the consistent improvements across the frozen 3B, 7B, and 8B backbones support the broader applicability of GazeEarth.

W2 · Scope of localization and cropping

(a) Coarse localization with and without explicit location cues. Our claim is that the answerer can use the question and image to identify relevant regions on a coarse grid. We evaluated all three selectors on the same 537 LRS-GRO questions without explicit location cues, using dataset-provided target-box centers as the reference. We matched the number of selected cells for each question on a 4×4 grid and retained empty selections in the denominator. Self-MLLM achieved a center-hit rate of 64.99% (349/537), compared with 52.33% (281/537) for SigLIP2 and 37.99% (204/537) for RemoteCLIP (Table 11). Uniform random selection under the same per-question cell budget has an expected hit rate of 6.41%. This comparison supports useful coarse localization without explicit location cues.

For questions with explicit spatial cues, Table 1 measures alignment with the region specified in the question. The mean rates across the three benchmarks are 77.74% for Self-MLLM and 43.09% for SigLIP2. Both external retrievers received the spatial phrases and each patch's global position. Mapping these cues to visual regions is part of the selector's task. The MLLM can interpret the question together with the full scene, and the comparison supports using this capability for region selection.

To isolate the contribution of visual input, we also ran a controlled probe on the same 537 box-annotated questions, requiring exactly two cells per question (Table A14). This fixed budget differs from the matched, variable cell budgets used in Table 11. At 4×4, the center-hit rate is 77.84% with image and question, 11.17% with the question alone, and 12.50% for random selection. As detailed under Q5, the MLLM exceeds both retrievers at 4×4 and 8×8, but this advantage does not persist at 16×16.

AuthorsOpenReview ↗

Response to Reviewer AviT Part (2/6)

(b) Benchmark dependence of cropping. We did not claim "Crop does not help." Our statement in Section 2 is: "Cropping what it selects does not reliably help." Selecting a relevant region is only part of the problem; answering also depends on how local detail and scene context are presented together.

As you note, many MME-RealWorld-RS questions require global context. Overview retains that context with a smaller visual input burden, which explains its advantage over crop-only presentation in this setting. XLRS-Bench contains many questions about fine-grained targets, so local detail is also important and cropping can help. These differences are consistent with our claim that cropping is not reliably beneficial across benchmarks.

GazeEarth's largest gain over Overview occurs on XLRS-Bench (Table 21 in the paper), where cropping itself helps. We further tested continuous presentation on the 380 XLRS-Bench two-cell questions, holding the selected regions, answer model, prompt, and final canvas size fixed. The continuous warp outperforms both a linked overview plus a union crop and a linked overview plus separate crops (Table A6). Its benefit therefore also appears in a setting where cropping is useful.

W3 · Gain decomposition

Direct serves as a raw-input baseline, with no image processing by our pipeline, including resizing. We agree that the contributions of canvas normalization and directed focus should be distinguished more clearly. We will state the reference baseline alongside each headline gain in the revised manuscript, separating canvas normalization from the additional benefit of directed focus over Overview.

W4/Q1/Q3/Q4 · Geometry, two-cell focus, request status and iterative gaze

Metric geometry and distortion. We agree that warping changes distances, areas, and angles. It preserves connectivity and pointwise left/right and up/down ordering. We therefore chose the warp parameters carefully to balance enlargement against geometric deformation.

Comparisons with distortion-free presentations do not support attributing the spatial-category results you cite solely to warp distortion. On XLRS-Bench, GazeEarth outperforms Linked-Global+Crop in every category except OP and achieves 53.51% overall versus 52.56%. This indicates that the benefit of its presentation outweighs the cost of deformation in these comparisons. You describe XLRS OSR as weak, but GazeEarth's 37.80% is the highest score among all methods and variants in Tables 4 and 15. On MME Position, GazeEarth scores slightly below Overview. Our explanation is that the global view already provides the location cues needed for these questions, with a lower visual input burden. Consistent with this explanation, Table 17 shows that the other presentations with added detail, including crop-based methods, also trail Overview. Consequently, this result cannot simply be attributed to warp distortion.

We further compared three warp settings on position-related questions: no additional warp, the default α=1.5\alpha=1.5, and a stronger warp with α=2.0\alpha=2.0.

Table A3. Position-related answering under three warp settings; accuracy (%).

BenchmarkPosition-related categoryQuestionsGazeEarth, α=1.5 (%)No additional warp (%)α=2.0 (%)
XLRS-BenchObject Spatial Relationship (OSR)50037.8028.8027.20
MME-RealWorld-RSPosition1,25771.1272.3966.91
LRS-GROObject relative position / context92121.7211.9411.94

The default α=1.5\alpha=1.5 performs best on XLRS OSR and LRS-GRO relative position/context. On XLRS OSR, it scores 37.80%, compared with 28.80% without additional warp and 27.20% with α=2.0\alpha=2.0. On MME Position, the corresponding scores are 71.12%, 72.39%, and 66.91%. These results support our chosen balance between preserving spatial information and making details visible.

AuthorsOpenReview ↗

Response to Reviewer AviT Part (3/6)

We also quantified anisotropy and evaluated shape-sensitive recognition under the default warp. Let sxs_x and sys_y be the local axial scale factors relative to uniform Overview. We define R=max⁡(sx/sy,sy/sx)R=\max(s_x/s_y,s_y/s_x); for example, R=1.50R=1.50 means one axial scale is 1.50 times the other.

Table A4. Anisotropy in two-cell selections.

BenchmarkTwo-cell questionsMerged focus with R>1.10R>1.10Median focus RRSource-image area in cross-shaped strips with R>1.10R>1.10
XLRS-Bench380259/380, 68.16%1.466×39.27%
MME-RealWorld-RS160129/160, 80.63%1.498×44.93%
LRS-GRO2,0731,401/2,073, 67.58%1.467×38.35%

Across the benchmarks, 67.58% to 80.63% of two-cell requests have merged-focus anisotropy above R=1.10R=1.10, with median RR ranging from 1.466× to 1.498×. We then compared GazeEarth with Overview on elongated or geometry-sensitive categories.

Table A5. Shape-sensitive LRS-GRO categories.

LRS-GRO target categoryQuestionsOverviewGazeEarthΔ (pp)
Ships22573.78%75.56%+1.78
Bridges16978.11%78.70%+0.59
Runways1693.75%93.75%0.00
Roads1250.00%50.00%0.00
Total42275.59%76.78%+1.18

Across these 422 questions, GazeEarth matches or exceeds Overview in every listed category, with an aggregate gain of 1.18 percentage points. Thus, the complete GazeEarth presentation maintains or improves overall recognition accuracy relative to Overview in these tested categories. Section 3.3 was intended to state that gaze does not increase the visual budget. We will revise it to distinguish that property from metric invariance, since distances, angles, aspect ratios, and absolute canvas positions can change.

Two-cell focus. We agree that widely separated cells can dilute per-target magnification. For non-adjacent pairs, we measured the merged focus area after warping relative to its area under uniform resizing. The median ratio is 1.50× on each benchmark, while 25.45%–35.03% of these requests receive no area gain. This quantifies the enlargement limit you describe. The enclosing rectangle preserves both selected regions and the intervening scene in one continuous view, balancing target detail against inter-target context. To test this trade-off, we compared the continuous warp with union and separate crops, each paired with a linked Overview, on all two-cell questions across the three benchmarks. We held the selected cells, answer model, prompt, and final canvas size fixed and changed only the presentation.

Table A6. Accuracy (%) on all two-cell selections.

BenchmarkQuestionsWarpLinked Overview + Union CropLinked Overview + Separate Crops
XLRS-Bench38044.4741.3240.79
MME-RealWorld-RS16045.6340.6335.63
LRS-GRO2,07365.6564.3063.43

Continuous Warp outperforms both crop-based presentations on all three benchmark subsets. These results support our argument that it effectively balances pixel allocation between the targets and the context connecting them. We will add this analysis to the paper. More effective multi-focus rendering for widely separated targets is also a direction for future work.

Request status and coordinate handling. The compared baselines use the same coordinate-handling policy as GazeEarth. To account for all execution routes, we also quantified coordinate bypass and evaluated a forced-grid control. Only XLRS-Bench uses this route: 1,660 of 3,080 questions (53.90%) provide coordinates in the question itself. MME-RealWorld-RS and LRS-GRO have no such cases. Disabling this shortcut in region selection and otherwise following the same inference procedure changes full XLRS-Bench accuracy from 53.51% to 50.13% (−3.38 points). The coordinate route therefore makes a measurable contribution, which we will report separately from autonomous gaze selection.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response Part (1/2)

We thank the reviewers for recognizing our rigorous experiments and informative ablations, particularly the controlled comparisons of visual presentations (AviT, Y5m6, svEJ). They also highlight our conceptual insight in separating localization from contextual presentation (AviT, Y5m6). Further strengths include our simple, efficient, training-free design (AviT, svEJ) and comprehensive evaluation across three benchmarks and four frozen backbones (Y5m6, svEJ). We address the shared concerns below.

  1. Backbones and gain attribution (AviT-W1/Q2/W3; svEJ-W1). We added same-backbone 3B and 7B comparisons and retrained GeoLens and ZoomEarth on Qwen3-VL-8B. GazeEarth improves over Direct on the smaller backbones and, without task-specific training, outperforms both retrained methods on 8B. These methods' original training data and recipes do not transfer readily; retrained ZoomEarth even falls below Direct on MME-RealWorld-RS. Taken together, these comparisons support our argument that visual presentation may matter more than task-specific training when the backbone already has strong visual capabilities. We also clarify that Direct is a raw-input baseline without image processing by our pipeline, including resizing. We distinguish normalization gains from the additional benefit of directed focus. We agree that the LRS-GRO gain over Overview does not reach corrected statistical significance.

  2. Localization and cropping (AviT-W2/Q5; Y5m6-W3; svEJ-W5). Our claim is that cropping the selected region does not reliably help: its usefulness depends on how detail and scene context are presented together. For localization, external retrievers received the same spatial phrases and each patch's global position. The comparison therefore tests their ability to ground these cues. On questions without location cues, Self-MLLM outperforms both external retrievers and random selection under matched per-question cell budgets. A separate fixed-two-cell probe confirms the contribution of visual input and identifies limits at finer grids. These findings support coarse 4×4 selection with padding and retained scene context.

AuthorsOpenReview ↗

General Response Part (2/2)

  1. Geometry and two-region presentation (AviT-W4/Q4; Y5m6-W2/Q2; svEJ-W2/W3). We quantified anisotropy and added shape-sensitive and position-related controls. Under the default setting, GazeEarth matches or exceeds Overview in every tested shape-sensitive category. It also achieves the highest accuracy among the tested warp settings on XLRS-Bench and LRS-GRO position-related questions. On MME Position, it scores slightly below the no-additional-warp setting. For two selected regions, the enclosing rectangle deliberately retains both targets and the context between them. New area measurements quantify the resulting limit: some non-adjacent pairs receive no enlargement. We then compared visual presentations on all two-cell questions in each benchmark, holding the selected cells, answer model, prompt, and final canvas size fixed. Continuous warping outperforms both linked crop alternatives in these comparisons. The size and counting analyses clarify where a single focus helps and where its coverage is insufficient.

  2. Cumulative multi-gaze (AviT-W4/Q3; Y5m6-W3/Q3; svEJ-W4). "Active" means that the model determines where to look and thereby guides the presentation; it does not require iterative search. We nevertheless added cumulative gaze with retained source-coordinate regions, direct rendering from the original image, and early stopping. This improves on the original recursive variant on XLRS-Bench, but still trails One-Gaze. Error-transition statistics show that additional gazes overturn more correct answers than they repair, while early stopping explains the small third-gaze changes. Our interpretation is that accumulated context increases the model's reasoning and perceptual burden. These results support One-Gaze as an effective tested inference budget.

  3. Coordinate use and execution routes (AviT-Q1; Y5m6-W1/Q1; svEJ-W2). Bypass and conditional routing are common implementation practices in related UHR work, including ZoomEarth, GeoLens, and WeaveEarth. The compared baselines use the same coordinate-handling policy as GazeEarth. We quantify its contribution on XLRS-Bench through a forced-grid control and report it separately from autonomous gaze selection. Our latency measurements use the complete selector-and-answerer path. We also report request outcomes and conditional answer accuracy, distinguishing declared non-selection from the absence of enlargement in the final rendered view.

The detailed replies provide the new experimental results and implementation references, including the counting analysis. Extensive sensitivity tests and one configuration shared across all three benchmarks support GazeEarth's robustness. Our evaluation follows the practice reported in related UHR work. We have also revised the figures to show verbatim model outputs. We thank the reviewers for the suggestions that led to these additional tests and clarifications.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.