acceptodds
Withdrawn from ICLR 2027

Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Ultra-high-resolution (UHR) remote-sensing (RS) imagery provides fine-grained Earth-observation evidence over city-scale scenes, but poses a fundamental challenge for multimodal large language models (MLLMs): task-relevant evidence is often sparse, local, and spatially dispersed across extremely large visual contexts. A natural solution is to equip MLLMs with zoom-in tools for active local inspection. However, through a pilot study on XLRS-Bench, we find that zoom-in is only partially effective: it resolves easy and medium-level tasks with locally recoverable evidence, but saturates on hard cases requiring global search, multi-region comparison, path planning, or dispersed-evidence reasoning. Motivated by this finding, we move beyond single-tool zoom-in and introduce GeoMTVR, a large-scale Geospatial Multi-Tool Visual Reasoning dataset built from wide-area satellite imagery. GeoMTVR contains 13K UHR VQA samples with interleaved reasoning trajectories, diverse visual tool calls, and returned visual observations, enabling models to learn question decomposition, tool selection, regional inspection, object-level grounding, auxiliary visual reasoning, and cross-tool evidence integration. Beyond supervised fine-tuning, we propose a tool-attention-focused reinforcement learning algorithm that concentrates optimization on critical tool-use decisions, including when to invoke tools, which tool to select, where to apply it, and how to interpret tool outputs. By combining SFT on GeoMTVR with our RL algorithm, we develop GeoLens, a multi-tool visual reasoning MLLM for UHR RS. Experiments show that GeoLens achieves competitive or leading performance across three UHR RS benchmarks. Our results suggest that UHR RS MLLMs should evolve from passive image understanding and single-operation exploration toward active, task-adaptive, multi-tool visual reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
remote sensing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FKchOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies tool-augmented reasoning for ultra-high-resolution remote-sensing VQA. A pilot study with a zoom-in-only RL model on XLRS-Bench suggests that zoom-in mainly solves tasks with localized evidence and saturates on tasks requiring global search, multi-region comparison, or path reasoning. Motivated by this, the paper contributes (1) GeoMTVR, a dataset of about 13K UHR VQA samples whose questions come from SuperRSVQA and LRS-GRO-train, re-annotated with interleaved reasoning and crop-and-zoom, grounding, and auxiliary-line tool calls through a staged semi-automatic pipeline (Qwen3-VL-235B, GPT-5.2, GroundingDINO); (2) Reinforced Tool Attention Learning, which adds an advantage-weighted Jensen–Shannon divergence on final-layer attention at tool-call positions to the GRPO objective; and (3) GeoLens, a Qwen2.5-VL-7B model trained with SFT on GeoMTVR followed by RTAL.

Critical strengths
  1. The paper is well presented. The figures clearly illustrate the pilot study, the dataset, and the RTAL pipeline, and the layout is clean and easy to follow.
  2. GeoMTVR is a potentially useful resource for UHR remote sensing. It provides 13K VQA samples with interleaved reasoning, multi-tool calls, and returned visual observations.
Critical weaknesses
  1. Limited algorithmic novelty. RTAL is essentially Reinforced Attention Learning (RAL; Li et al., 2026), restricted to tool-call positions. Eqs. 2–8 follow RAL almost term by term, yet RAL is cited only as "inspiration". There is no comparison against RAL, so the 1.8-point gain over GRPO cannot be attributed to the tool-position mask, which is the only new component.
  2. The "beyond zooming" thesis is weakly supported. With the same SFT setup, zoom-only data and full multi-tool data differ by only 0.7 points (Table 14). The line tool appears in only about 5% of samples. No zoom-only model is trained under the same RL pipeline.
  3. The main results do not back the claims. On XLRS-Bench, the benchmark used for the pilot study and all ablations, GeoLens only ties the zoom-only GeoEyes on average and is best on 2 of 13 sub-tasks. It also shows no gain on the tasks that motivate the extra tools.
Questions for authors
  • Can you compare RTAL against vanilla RAL (attention JSD on all response tokens) under the same SFT checkpoint, data, and steps, and against RTAL with the response_reg branch removed? This is needed to attribute any gain to the tool-position mask.
  • Which objective is actually used in the reported experiments: Eq. 3 alone, or Eq. 3 plus the valid-response JSD branch described in Appendix A.2? If the latter, what is λresp\lambda_{resp}?
  • Can you report the mean and standard deviation over multiple seeds for SFT + GRPO and SFT + RTAL?
  • Can you train a zoom-in-only variant with the same backbone, SFT data size, and RTAL stage, and report per-task results, especially on RP, RCCD, and CCR, where you claim multi-tool gains?
  • Can you rerun the pilot-study Quality analysis with GeoLens on the same failure sets to show that the additional tools improve the persistently failed samples?
  • Why does GeoLens trail GeoEyes on Overall Counting and GeoLLaVA-8K on Route Planning, the task types that motivate the grounding and line tools? How often is the line tool used on RP samples?
  • Are the Qwen3-VL-235B-A22B values in Table 4 correct? Its OC and RC scores are far above those of all other models, while its RLUC and OMS scores are far below.
  • How many generated trajectories were filtered or corrected by human annotators, by how many annotators, and with what agreement?
  • How was the annotation quality of GeoMTVR evaluated? Since GPT-5.2 sees the ground-truth answer when it generates trajectories, are the intermediate steps checked to actually support that answer? Can you report trajectory correctness and bounding-box accuracy on a manually verified sample?
Reviewer_sDubOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-tool visual reasoning for ultra-high-resolution (UHR) remote sensing, motivated by the limitations of zoom-in-only methods on tasks requiring global search, spatial reasoning, and multiple regions of evidence. The authors introduce GeoMTVR, a dataset containing approximately 13K VQA samples with tool-augmented reasoning trajectories generated through a semi-automatic annotation pipeline. They then train GeoLens based on Qwen2.5-VL-7B using SFT followed by Reinforced Tool Attention Learning (RTAL), which applies an advantage-weighted attention objective to tool-related tokens. Experiments on XLRS-Bench, RSHR-Bench, and LRS-GRO-eval demonstrate competitive performance, matching GeoEyes on XLRS-Bench and outperforming it on the other two benchmarks.

Critical strengths

1. A useful dataset contribution for multi-tool remote sensing reasoning.

GeoMTVR extends existing UHR remote sensing VQA datasets by introducing interleaved reasoning trajectories and multiple visual tools, including zoom-in, grounding, and auxiliary-line drawing. The dataset is reasonably large and addresses a meaningful limitation of existing zoom-in-based methods. Although the annotation process relies heavily on existing foundation models, the resulting dataset could still be a useful resource for future research in active visual reasoning.

2. A reasonable adaptation of attention-level RL with competitive experimental results.

Although RTAL builds on the existing RAL method, the introduction of tool-call-specific attention masking and an additional attention regularization branch is a reasonable modification for multi-tool visual reasoning. I consider this a positive aspect of the method, even though its independent contribution still needs to be better established.

The paper also provides a fairly complete experimental evaluation across three UHR remote sensing benchmarks, with several ablation studies. GeoLens matches GeoEyes on XLRS-Bench and outperforms it on RSHR-Bench and LRS-GRO-eval. RTAL also achieves a 1.8 percentage-point improvement over standard GRPO on XLRS-Bench. These results suggest that the proposed adaptation is practically useful, despite its relatively limited algorithmic novelty.

Overall, I think the combination of a task-specific dataset, reasonable training modifications, and competitive empirical results makes this a meaningful contribution.

Critical weaknesses

1. Limited methodological novelty, with insufficient evidence separating the contributions of RTAL and multi-tool reasoning.

My main concern is the limited novelty of the proposed training method. RTAL largely builds on Reinforced Attention Learning (RAL), using a similar advantage-weighted attention divergence objective but applying it to tool-related tokens. This is a reasonable adaptation to multi-tool reasoning, but the algorithmic contribution appears incremental. The authors should more clearly explain what is technically new compared with the original RAL formulation. A direct comparison with RAL under the same training setup would be particularly useful.

Similarly, the three visual tools are well established in prior visual reasoning work. The additional performance gain from multi-tool supervision is relatively modest. In Table 14, moving from zoom-in-only supervision to full multi-tool supervision improves the average accuracy from 49.6% to 50.3%. Although the paper includes relevant ablations, it remains unclear how much grounding and auxiliary-line drawing individually contribute, especially on complex spatial reasoning tasks.

I would not consider the existing analysis entirely insufficient, but stronger controlled experiments would help substantiate the claimed advantages of multi-tool reasoning.

2. Insufficient analysis of annotation reliability, performance trade-offs, and inference cost.

GeoMTVR relies heavily on foundation models to generate intermediate reasoning trajectories and tool annotations. The authors state that human validation and filtering were performed, which is positive. However, the paper does not provide enough quantitative analysis of the correctness and faithfulness of these trajectories.

In particular, a correct final answer does not necessarily imply that the intermediate tool calls, grounded regions, or reasoning steps are correct. Incorrect but plausible trajectories could introduce noisy supervision. A quantitative annotation-quality evaluation would make the dataset contribution more convincing.

I also noticed substantial differences in task-level performance compared with GeoEyes. While GeoLens performs better on several reasoning-oriented tasks, its Object Classification and Object Color accuracies decrease from 59.5% to 46.5% and from 66.1% to 45.6%, respectively. These trade-offs deserve further discussion, especially since the overall XLRS-Bench macro-average remains essentially tied with GeoEyes.

Finally, the paper reports the average number of tool calls but provides limited information about practical inference efficiency. RTAL increases the average number of tool calls from 1.89 to 2.40 compared with GRPO. It would be useful to determine whether the accuracy improvement remains under comparable tool-use and computational budgets.

Questions for authors

1. How much of the RTAL improvement comes specifically from tool-focused attention optimization?

Since RTAL is closely related to RAL, could the authors compare RTAL against the original RAL objective under the same SFT initialization, training data, and RL configuration? This would help clarify whether tool-specific attention masking provides benefits beyond applying existing attention-level RL to this domain.

2. Can the authors further isolate the contributions of the individual visual tools?

Table 14 shows a relatively small improvement from zoom-in-only to full multi-tool supervision. Could the authors provide a more detailed comparison, such as zoom-in only, zoom-in + grounding, zoom-in + line, and the full tool combination? Task-level results on Route Planning, Object Spatial Relationship, and other challenging reasoning tasks would be particularly informative.

3. How reliable are the automatically generated reasoning trajectories in GeoMTVR?

The paper mentions human verification and filtering. Could the authors clarify the validation protocol and provide quantitative statistics on intermediate reasoning correctness, tool-call validity, and grounding accuracy? A manual audit on a representative subset would help determine whether the trajectories genuinely provide correct intermediate supervision rather than only supporting correct final answers.

4. Can the authors provide stronger comparisons with closely related tool-using methods?

The paper already includes several relevant baselines, such as GeoEyes, ZoomEarth, and ViLaSR. However, the comparisons would be more convincing if training configurations and inference budgets were better controlled. VICoT-Agent is another closely related remote-sensing tool-agent framework worth considering where a compatible evaluation is feasible. A discussion of the differences in tool capabilities and training setups would also help establish the contribution of GeoLens.

5. Could the authors further discuss the performance trade-offs and computational efficiency?

GeoLens improves several reasoning tasks but performs substantially worse than GeoEyes on some fundamental perception tasks. What might explain these regressions? Reporting sample-weighted accuracy in addition to the existing macro-average would provide a more complete view of overall performance.

In addition, could the authors report average inference latency, generated token counts, and tool-execution overhead? Since RTAL uses more tool calls than GRPO, it would be valuable to compare performance under matched tool-call or inference-cost budgets.

Reviewer_GGKsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies multimodal LLMs on ultra-high-resolution (UHR) remote-sensing VQA. A pilot study trains Qwen3-VL-8B-Instruct with GRPO and only a zoom-in tool. Fig. 4 tracks the MED Quality factor under three failure-set definitions across training steps. From this and a qualitative table (Table 8), the authors argue that zoom-in solves localized tasks but saturates on global counting, path planning and dispersed-evidence reasoning. The authors then build GeoMTVR, about 13K UHR VQA samples (average about 6K x 6K) with interleaved reasoning and three tools (crop_and_zoomin, grounding, line). The trajectories are produced by a Qwen3-VL-235B-A22B / GPT-5.2 / GroundingDINO pipeline (Fig. 3). They also propose RTAL, which adds to GRPO an advantage-weighted Jensen-Shannon divergence between the current and the pre-update final-layer attention at tool-call positions (Eqs. 1-4). The resulting 7B model, GeoLens (Qwen2.5-VL-7B-Instruct, SFT then RTAL), ties GeoEyes on XLRS-Bench (54.2 macro average, Table 4). It reports the best averages on RSHR-Bench (48.8, Table 5) and LRS-GRO-eval (60.7, Table 6). Ablations (Table 7 and the appendix) attribute +4.3 to SFT and +1.8 to RTAL over GRPO. No variance across runs is reported.

Critical strengths

The paper introduces GeoMTVR, a UHR remote-sensing dataset with multi-tool reasoning trajectories. GeoMTVR contains 13,078 samples with an average resolution of 6,033 x 6,017 (Table 1, Table 3). Each sample interleaves reasoning with crop-and-zoom, grounding and line-drawing calls and the returned observations. Among the remote-sensing datasets compared in Table 1, it is the only one with more than one tool type. The others offer zoom-in only or no tools. The general-domain tool-use data in Table 2 uses images of at most 588 x 588, about one tenth of GeoMTVR's resolution per side.

Critical weaknesses

W1. RTAL's auxiliary objective may contribute no gradient, and the main text gives no evidence that its gain exceeds run-to-run noise. Location: Eq. (3), L366-L374, Table 7.

Quote (L368-L369): "With the frozen pre-update attention policy p~told\tilde p^{old}_t, the per-head objective is" ... (L373-L374): "Minimizing this loss penalizes departures from the old attention policy when Aτ>0A_\tau > 0 and favors departures when Aτ<0A_\tau < 0."

The loss is AτA_\tau times DJSD(p~tθ∣p~told)D_{JSD}(\tilde p^\theta_t \\| \tilde p^{old}_t). The partial derivative of JSD with respect to p~tθ(i)\tilde p^\theta_t(i) is 12log⁡(p~tθ(i)/mi)\tfrac12\log(\tilde p^\theta_t(i)/m_i), with m=(p~tθ+p~told)/2m = (\tilde p^\theta_t + \tilde p^{old}_t)/2. It is exactly zero when θ=θold\theta = \theta_{old}. The main text does not state how many optimizer steps are taken per rollout batch. If there is only one, the gradient is taken at θ=θold\theta = \theta_{old}, and Eq. (3) adds nothing to the GRPO gradient, whatever the sign of AτA_\tau. Even with several inner updates, the objective only anchors attention to its own previous value. It contains no term that moves attention "toward task-relevant context" (L344), or that encourages "tool decisions to incorporate relevant context and previous observations" (L351-352). The only RTAL-vs-GRPO comparison in the main text is Table 7 (52.4 vs. 54.2). It gives a single number per method, without variance. The relation to Reinforced Attention Learning (Li et al., 2026), which RTAL is "inspired by" (L111), is also not delineated. RTAL is one of the three claimed contributions (L134).

W2. The "beyond zooming" thesis lacks a controlled test, and the tie with GeoEyes on XLRS-Bench does not hold under sample-weighted accuracy.

Location: L120-L122, Sec. 4.1 (L296-L299), Table 4.

Quote (L120-L122): "Experiments on UHR RS reasoning benchmarks show that GeoLens consistently outperforms direct reasoning and single-tool zoom-in baselines".

(a) The main text has no comparison of zoom-only and multi-tool training under the same backbone and data. The only such comparison is in the appendix (Table 14). It covers SFT only, reports one number per setting without variance, and shows 49.6 vs. 50.3 macro. There is no zoom-only GeoMTVR + RTAL model. So the end-to-end gain cannot be attributed to the extra tools rather than to more SFT/RL data. The comparison with GeoEyes in Table 4 does not isolate the tools either, since GeoLens also differs in training data (GeoMTVR) and RL algorithm (RTAL).

(b) The pilot explains its result as follows (L296-L299): "as easier samples are solved and leave Dfail(t)D_{fail}(t), the remaining failures become harder, and a single zoom-in operation can no longer provide sufficient evidence". This selection effect would also be expected with other tool sets, so the divergence alone does not show that zoom-in is the limiting factor. The decomposition is not repeated for the multi-tool model. The per-task claim in the Takeaway (L305-L308) is supported only qualitatively (L310-L318 and appendix Table 8), without per-task numbers. The pilot also uses Qwen3-VL-8B-Instruct (L277), while GeoLens is built on Qwen2.5-VL-7B (L419).

(c) Five XLRS-Bench subsets closely match the categories the pilot identifies as zoom-in failures: OC, OLUC, RP, RCCD and CCR. GeoLens is the best compared model on two of them (RCCD 51.7, CCR 55.0). On the other three it trails at least one baseline: OC 35.0 vs. 40.0 for InternVL3-8B, OLUC 37.0 vs. 49.0 for GeoLLaVA-8K, and RP 62.0 vs. 66.0 for GeoLLaVA-8K. The evidence that multi-tool reasoning resolves these cases is therefore mixed.

(d) Table 4 follows the XLRS-Bench convention of averaging over subsets without weighting, and GeoLens ties GeoEyes at 54.2 (L427). The subsets range from 60 to 800 questions (Table 9), and weighted by question count, GeoLens scores 49.1 and GeoEyes 56.1. GeoLens trails GeoEyes by 13.0 and 20.5 points on OCC and OCL, which together hold 1,600 of the 3,080 questions. Across the 13 subsets, GeoLens wins 7 (1,160 questions), ties 1 (100) and loses 5 (1,820) against GeoEyes. Without seeds or confidence intervals, "consistently outperforms ... single-tool zoom-in baselines" is not supported.

Questions for authors
  1. RTAL gradient. Appendix Table 13 lists train batch size 32 and PPO mini-batch size 32. This means one optimizer step per rollout batch if a single PPO epoch is used. How many PPO epochs or optimizer steps are taken per rollout batch? How is p~told\tilde p^{old}_t obtained (the same FSDP actor before the update)? Please report the norm of the gradient of the attention term relative to the GRPO term during training. If it is zero at the first inner step, where does the 1.8-point gain over GRPO come from? Table 18 sweeps λattn\lambda_{attn} over 0.05, 0.10 and 0.20 (54.0/54.2/54.1) but has no λattn=0\lambda_{attn}=0 run. The only run without the attention term (SFT+GRPO, 52.4) uses KL 0.01, whereas the default RTAL disables KL (Table 17). Please report a λattn=0\lambda_{attn}=0 run with otherwise identical settings, ideally over several seeds. Please also ablate the tool-position branch and the response-token branch (L910-914) separately.

  2. Variance. Please report mean +/- std over at least 3 seeds for SFT+GRPO and SFT+RTAL (Table 7). Please do the same for the zoom-only and multi-tool SFT rows of Table 14.

  3. Controlled single-tool baseline. Can you train a zoom-only variant of GeoMTVR with the same SFT + RTAL recipe and backbone? Alternatively, can a zoom-in-only baseline such as GeoEyes be given the same grounding and line tools? This is the comparison the title implies.

  4. Weighted accuracy. Since the XLRS-Bench subsets range from 60 to 800 questions, please also report sample-weighted accuracy for Table 4. Weighting by question count also narrows the RTAL-over-GRPO gap to 48.6 vs. 49.1 (appendix Tables 9 and 16).

  5. Checkpoint selection. The appendix mentions a validation set of 244 SuperVQA samples (L870-871), and RTAL saves a checkpoint every 10 steps (Table 13). Please state how the final GeoLens checkpoint was chosen, for example the last step or the best step on this validation set. Please also confirm that XLRS-Bench was not used for this choice.

  6. Annotation quality. The pipeline starts from the gold answer (L250-L251: "Given a question, image, and answer, we first use Qwen3-VL-235B-A22B"). Fig. 3 also shows the GPT-5.2 stage receiving "Q&A" as input. The trajectories may therefore be answer-conditioned rationalizations whose tool calls do not determine the answer. The AI-use statement says "Human annotators validated and filtered the annotations produced by the semi-automatic pipeline" (L495-496). Sec. 3 gives no protocol for this. How many annotators were involved, what was the acceptance rate, and what was their agreement? What fraction of trajectories reach the gold answer without access to it? Table 10 lists a third SFT source, "route_visual_accepted_388" (388 of the 13,078 samples), which is not described elsewhere in the paper. Please state where these route samples come from and how they were filtered. Since route planning is where the gain over GeoEyes is emphasized (L447-451), please also confirm that they do not overlap with the XLRS-Bench RP images. How is "strict image-level deduplication" (L237) implemented: by exact hashing, or by geographic or tile overlap?

  7. Baseline provenance. Several Table 4 entries cannot be produced on XLRS-Bench's subset sizes (Table 9). Qwen3-VL-235B has RC = 82.5 and RP = 37.8 on 100-question subsets. It also has OMS = 38.9 on a 60-question yes/no subset, which is below chance. In the Claude 3.7 Sonnet row, 10 of the 13 entries are similarly unattainable. Gemini 2.5 Pro and Intern-S1-mini report only an average. Were these copied from other papers or from a different protocol? This matters because L428 claims a 3.1-point margin over Qwen3-VL-235B. In Table 6, GeoEyes' Avg (56.05) equals the unweighted mean of its three columns, while no other row's Avg does. Please state how Avg is computed for each row. GeoLens uses the LRS-GRO training split in both stages (Tables 10-11). Which baselines in Table 6 were also trained on this split?

  8. General-domain tool data (Table 2). Adding about 35K general-domain samples to 13K remote-sensing samples changes two things at once. The share of remote-sensing data drops to about 27%. The added traces also use a different tool (Object Mapper) from GeoLens's three tools. The drop from 48.9 to 42.7 therefore cannot be attributed to the tool traces alone. The explanation at L213-L216 ("their tool-use traces may provide weak or even misleading supervision") is not tested. Can you add a control with the same amount of general-domain data without tool calls?

Suggestions:

  • Please discuss and, where feasible, compare with recent zoom-in and visual-search methods: Retrieval-Augmented Perception (Wang et al., ICML 2025) and Mini-o3 (Lai et al., ICLR 2026).
  • The line tool appears in 5.31% of training samples (Table 3) and is used mainly for route planning (0.830 calls per RP question vs. 0.079 overall, Table 21). Please show its isolated effect, for example on RP.

Minor issues:

  • L86-87 cites Li et al. (2026) as the "zoom-in-enabled reinforcement learning framework". That reference is "Reinforced attention learning" (L572-573), while Sec. 4.1 uses GRPO with MED data.
  • L422: "based on the our RTAL method" should be "based on our RTAL method".
  • The naming is inconsistent: SuperRSVQA, SuperRS-VQA and SuperVQA.
  • The arrow notation in Tables 14-16 ("48.9 ↓ +0.7") is unclear.
  • Units are used inconsistently. XLRS-Bench and LRS-GRO-eval results are given with % and "percentage points" (L427-L431). RSHR-Bench results are given without % and in "points" (L430, L484). L1080 omits the unit ("reduces invalid calls by 7.90"). L1082 gives "(64.5 vs. 59.5)" for the values written "(64.5% vs. 59.5%)" at L470.
  • In Tables 7, 17 and 20, the accuracy columns are given without %, while the call-rate columns carry %. No column header states the unit.
  • Table 6 reports GeoLens to one decimal (60.7) and the baselines to two (e.g., 56.05). The 4.65-point gain at L431 therefore mixes precisions.
  • Table 20 duplicates Table 7, and L774-782 duplicate L310-318.
  • Eqs. (5)-(8) mainly derive the gradient through the standard softmax Jacobian. They could move to the appendix to make room for the training details now only in A.2.2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.