acceptodds
Withdrawn from ICLR 2027

EAGLE-360: Embodied Active Global-to-Local Exploration in

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Precise target localization in environments requires integrating global spatial context with fine-grained visual evidence while accounting for projection distortion and horizontal wrap-around geometry. For multimodal large language models (MLLMs), the challenge is not only to select informative views but also to derive geometrically consistent estimates of the target direction from visual observations. We present EAGLE-360, an Embodied Active Global-to-Local Exploration framework that leverages panoramic context to guide iterative, tool-augmented target localization. We introduce Spherical Atlas Rotary Position Embedding (AtlasRoPE) for local spherical positional modeling, alongside Spherically Stratified Replay (SSR) and Spherical Direction Regularization (SDR) for geometry-aware training. This training strategy increases exposure to underrepresented directions and incorporates continuous spherical supervision, explicitly accounting for seam-adjacent and high-latitude targets without adding inference-time modules. To support this framework, we construct a curated panoramic search dataset with multi-turn trajectories interleaving reasoning, projection actions, and visual observations. We train the model through supervised fine-tuning (SFT) followed by reinforcement learning based on Group Relative Policy Optimization (GRPO) to develop its multi-turn search capabilities. Experiments show that EAGLE-360-8b leads the main comparison on EAGLE-360 with 84.01% localization accuracy, together with strong transfer to HBench, achieving 75.00% object-search success under panoramic initialization without benchmark-specific training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, Panoramic Target Localization, Active Visual Search, Spatial Reasoning, Tool Use, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GEAvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents EAGLE-360, a framework for target localization in 360-degree panoramas. The model uses the full panorama to understand the scene and can request additional perspective views before predicting the target direction.

The authors introduce three components: AtlasRoPE for spherical positional encoding, Spherically Stratified Replay for improving directional coverage in training data, and Spherical Direction Regularization for continuous direction supervision. They also construct a dataset of 21,884 reference trajectories and train the model using progressive SFT followed by GRPO-based reinforcement learning.

Experiments are conducted on a direction-balanced test set of 713 questions and H* Bench. The final model achieves 84.01% localization accuracy on EAGLE-360 and 75.00% object-search success on H* Bench. The paper also provides component ablations and experiments on the effect of visual feedback.

Critical strengths
  1. The paper addresses a meaningful problem in panoramic visual understanding. The authors consider several difficulties specific to 360-degree images, including horizontal seam continuity, polar distortion, and uneven directional coverage. The direction-balanced evaluation is useful because it shows that models performing well on horizontal directions can still struggle with polar targets. EAGLE-360 achieves a clear improvement over its SFT baseline, particularly for upward and downward localization.

  2. The experimental work is fairly extensive. The authors construct a multi-turn training dataset, evaluate several open-source and proprietary models, and include transfer experiments on H* Bench. I also appreciate the matched component ablations and the visual feedback controls. In Table 4, real tool observations outperform mismatched observations under the same initial panorama conditions. This provides evidence that the model benefits from relevant visual feedback rather than only from generating additional reasoning steps.

Critical weaknesses
  1. I am not fully convinced by the novelty and independent contribution of the proposed components. AtlasRoPE builds on existing spherical positional encoding approaches, while SSR is mainly a direction-aware data replay strategy and SDR adds an auxiliary direction prediction loss. These components are reasonable, but the overall method seems to combine several existing ideas rather than introduce a clearly new learning framework.

More importantly, the ablations in Table 3 show that SSR contributes most of the improvement. Plain SFT achieves 76.30% accuracy, and adding SSR alone increases it to 83.03%. The full AtlasRoPE+SSR+SDR configuration reaches 84.43%, while AtlasRoPE alone slightly decreases overall accuracy, although it improves angular error. The additional contributions of AtlasRoPE and SDR are relatively small. The paper would benefit from stronger evidence showing why these components are necessary beyond SSR.

  1. The contribution of the final training pipeline is not sufficiently isolated. The main EAGLE-360 model includes additional interaction-focused SFT and GRPO, but Table 3 evaluates the geometry-aware components before these stages. As a result, it is difficult to determine how much GRPO contributes to the final localization accuracy and whether reinforcement learning improves tool selection or coordinate refinement. The full SFT configuration in Table 3 reaches 84.43%, while the final system reports 84.01%, although these are not directly matched training runs.

I also have some concerns about the evaluation setting. The direction-balanced test is constructed partly through image reorientation, which changes the relationship between visual content and gravity and does not introduce independent scenes. The paper acknowledges this limitation, as well as test-set reuse during development and single-run experiments. H* Bench provides useful transfer evidence, but its task and evaluation protocols are not fully matched across methods. These factors make it harder to judge how well the reported improvements would generalize to new panoramic environments.

Questions for authors
  1. Can the authors provide a direct comparison between the interaction-focused SFT checkpoint and the final GRPO checkpoint under the same evaluation setting? It would be useful to report localization accuracy, tool-call frequency, and performance on questions requiring multiple observations. This would help clarify the contribution of reinforcement learning.

  2. SSR provides most of the improvement in Table 3. Can the authors further explain the benefit of AtlasRoPE and SDR relative to SSR alone? In particular, do these components provide consistent improvements on seam-adjacent and polar examples across different training seeds?

  3. Can the authors provide more evidence that the method generalizes to independent panoramic scenes without evaluation-time reorientation? It would also help to separate the benefit of visual feedback for improving the estimated target direction from its benefit for producing valid final answers, since the current feedback ablation shows improvements in both accuracy and answer validity.

Reviewer_HhL7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EAGLE-360, an active target-localization framework for 360-degree panoramic environments. It combines global panoramic observations with optional perspective-view inspection and introduces AtlasRoPE, Spherically Stratified Replay (SSR), and Spherical Direction Regularization (SDR) to improve spherical positional modeling and directional prediction. The authors construct a dataset of 21,884 reference trajectories and train the model through progressive supervised fine-tuning and GRPO. Experiments evaluate localization accuracy and active perception on a direction-balanced benchmark and H* Bench.

Critical strengths

1.The research problem is practically relevant. The paper addresses active target localization in panoramic environments and explores the integration of global spatial context with local visual observations, which is meaningful for embodied spatial perception.

2.The framework is reasonably comprehensive and empirically evaluated. The method combines geometric modeling, data augmentation, and multi-turn interaction training, with component ablations and cross-benchmark experiments providing useful empirical insights.

Critical weaknesses

1.The independent contributions of the core methodological components are insufficiently established.

Table 3 shows that AtlasRoPE alone decreases overall accuracy from 76.30% to 75.88%, with a slight decrease in polar accuracy. In contrast, SSR alone improves overall accuracy to 83.03%, suggesting that directional replay accounts for most of the observed improvement. Although AtlasRoPE reduces angular error and provides additional gains when combined with SSR, its independent contribution remains limited.

Furthermore, the complete SFT configuration achieves 84.43% accuracy in Table 3, whereas the final model with additional interaction training and GRPO reports 84.01% in Table 1. Since these configurations follow different training sequences, this does not establish that GRPO is detrimental. However, the absence of a fully matched stage-wise comparison leaves its incremental contribution unclear. These results weaken the evidence for the necessity of the proposed components.

2.The evaluation design limits confidence in independent generalization.

Appendix D.1 states that 235 of the 713 test questions are evaluated on reoriented panoramas, including targets moved to high-latitude regions. While this produces balanced directional coverage, spherical reorientation can alter gravity alignment and the natural visual structure of a scene. Improvements on artificially reoriented panoramas therefore do not necessarily establish equivalent performance on naturally occurring polar targets.

Moreover, the paper explicitly acknowledges test-set reuse during development, and the main experiments rely on single training runs. Although the authors use source-panorama-disjoint splits and separate validation data, the robustness of the results to independent test scenes, evaluation design, and training randomness remains insufficiently established.

3.The contribution of active visual exploration to localization accuracy remains unclear.

Table 4 demonstrates that real tool observations improve success rates and outperform mismatched observations, providing useful evidence for the value of relevant visual feedback. However, only 121 tool calls are executed across 713 questions under standard-resolution evaluation, indicating that most questions are answered without additional inspection. With real tool views, correct answers increase from 559 to 600, while valid final answers increase from 642 to 707.

These results support the role of visual feedback in confirmation and answer completion, but do not establish how much of the improvement comes from actual coordinate refinement rather than fewer invalid responses. The no-tool condition also changes the instruction, limiting direct attribution. Without per-turn localization errors and correction statistics, the central claim of effective global-to-local active localization remains incompletely validated.

4.Cross-benchmark evaluation raises concerns about scoring reliability and protocol comparability.

Appendix D.2 acknowledges that the H* Bench scorer used in Table 2 omits the elevation check in its seam-handling branch for object search. However, the paper does not report complete results using the corrected scoring condition. This leaves the absolute object-search success rates and their comparison with prior methods insufficiently verified.

Additionally, the methods in Table 2 differ in initial observations, training conditions, and interaction budgets. While EAGLE-360 achieves 75.00% object-search success, its path-search success is only 34.75%, below PanoWorld's 47.5%. The results therefore support an advantage in object search under the reported protocol, but do not establish consistently superior performance across broader active spatial-search tasks.

Questions for authors

1.Component contributions: Could the authors report matched performance after geometry-aware SFT, interaction-focused SFT, and GRPO under identical initialization and training budgets? What are the independent contributions of AtlasRoPE, SSR, and SDR in the final system?

2.Generalization: How does EAGLE-360 perform on naturally oriented panoramas without artificial spherical reorientation, particularly on naturally occurring polar and seam-adjacent targets? Could the authors provide results on independent scenes and across multiple training seeds?

3.Active exploration: Could the authors report the proportion of questions requiring tool calls, localization errors before and after inspection, first-turn accuracy, and the fraction of incorrect initial predictions corrected by subsequent observations? These measurements would help distinguish coordinate refinement from improvements in valid answer submission.

4.Evaluation protocol: How do the H* Bench results change when the omitted elevation check is restored? Could EAGLE-360 and PanoWorld be compared under matched panoramic inputs, interaction budgets, and scoring rules?

Reviewer_LQhYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EAGLE-360, an embodied active global-to-local framework for target localization in panoramic environments. It combines optional perspective-view exploration with AtlasRoPE, direction-stratified replay, and continuous spherical supervision to address projection distortion, seam discontinuity, and directional imbalance. Evaluations show EAGLE-360-8B achieves 84.01% localization accuracy on a balanced panoramic benchmark and 75.00% object-search success on H*Bench, with the largest gains on polar targets.

Critical strengths
  • The paper addresses a practical problem of active target localization in 360° panoramas.
  • AtlasRoPE, SSR, and SDR are well motivated for handling spherical geometry and directional imbalance.
  • The experiments include strong ablations, feedback controls, and transfer evaluation.
Critical weaknesses
  • Concern about confounded gains in the main comparison. The EAGLE-360 model in Table 1 includes interaction-focused SFT and GRPO, whereas the matched component ablations in Table 3 stop before these stages. Consequently, the reported ablations cannot precisely separate the contributions of AtlasRoPE, SSR, and SDR from those of additional interaction training and reinforcement learning. A controlled comparison using identical training stages would better attribute the main performance gains.

  • Concern about the standalone effectiveness of AtlasRoPE. AtlasRoPE alone slightly reduces overall accuracy from 76.30% to 75.88% and polar accuracy from 37.13% to 36.71%, with improvements appearing only when combined with SSR. This makes it unclear whether AtlasRoPE provides an independently useful geometric representation or primarily benefits from increased exposure to polar examples. Additional multi-seed experiments and analyses of chart configurations would strengthen this claim.

  • Concern about the evaluation of camera control. The feedback ablation shows that real projected views improve final localization, but it does not directly measure the accuracy of the camera actions themselves. The paper does not report camera-to-target angular error, target-in-FoV rate, invalid tool-call rate, or improvement across successive actions. Thus, the evidence that the policy learns precise and reliable camera control remains indirect.

Questions for authors

Please kindly refer to the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.