acceptodds
Under review as a conference paper at ICLR 2027

NavGround3D: UAV-Centric Grounding of Semantic–Geometric Cues for Aerial Vision-Language Navigation

Abstract

Aerial Vision-and-Language Navigation (UAV-VLN) requires UAVs to follow natural-language instructions, understand scene semantics and spatial relation- ships in complex 3D environments, and navigate autonomously. Recent studies have begun to leverage geometric visual models in UAV-VLN to enhance 3D spa- tial understanding. However, existing approaches mainly focus on aligning or fus- ing semantic and geometric information, while paying limited attention to effec- tive visual information distillation and unified UAV-centric directional modeling. To address these issues, we propose NavGround3D, a semantic-geometric rep- resentation framework that progressively distills visual information and grounds it in a unified UAV-centric directional space. Specifically, Semantic-Geometric Visual Consolidation (SGVC) selects representative anchors based on seman- tic saliency and aggregates redundant representations according to semantic- geometric consistency, yielding more informative visual tokens. UAV-Centric Di- rectional Grounding (UCDG) then transforms view-dependent visual cues into the UAV body frame and introduces continuous directional encoding to estab- lish consistent cross-view spatial relationships. The resulting representations are integrated with navigation context for motion decision-making. Extensive exper- iments on TravelUAV and UAV-ON demonstrate consistent improvements over strong baselines, validating the effectiveness of semantic-geometric distillation and UAV-centric directional grounding for aerial navigation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Aerial Vision-Language Navigation, Semantic-Geometric Consolidation, UAV-Centric Directional Grounding, Multi-View Spatial Reasoning, 3D Spatial Understanding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HCcsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes NavGround3D for aerial vision-and-language navigation. Frozen CLIP and VGGT encoders extract semantic and geometric features from five RGB views. Semantic-Geometric Visual Consolidation (SGVC) selects attention-based anchors and merges remaining tokens using joint semantic–geometric similarity. UAV-Centric Directional Grounding (UCDG) transforms patch viewing rays into the UAV body frame, encodes their directions, and models cross-view interactions with a Transformer. A LoRA-tuned Qwen3 backbone predicts local motion direction and distance. Validation comprises simulated navigation on TravelUAV and UAV-ON, two backbone sizes, and module/design ablations.

Critical strengths
  • The architecture addresses a meaningful aerial-navigation problem by combining semantic and geometric features with explicit patch-level directions in a common body frame, while using RGB observations rather than requiring measured depth as an input.

  • The experimental design includes two navigation benchmarks, two backbone sizes, and module comparisons with a fixed visual-token budget. These are useful controls, although the reporting issues currently limit their evidential value.

Critical weaknesses
  • Limited novelty of SGVC. Its attention-based anchor selection and semantic–geometric token assignment closely follow Merge3D’s SemGeo Merger. The identified differences—cosine similarity, modality projections, and per-view budgets—are not sufficiently justified as substantive methodological advances. The feature aggregation rule is also unspecified.

  • Internally inconsistent results. In Table 7, the baseline’s Full OSR (53.78%) exceeds both Easy (50.91%) and Hard (42.78%), which is impossible when aggregating the same episode outcomes. Similar inconsistencies occur in SPL and Table 8, undermining confidence in the main comparisons and ablation conclusions.

  • Potential test-set-driven model selection. All ablations use Test-Seen, while Table 6 explicitly motivates selecting K=16 from these results. This appears to reuse the test set for hyperparameter selection, weakening the independence of the reported evaluation.

  • No real-world UAV validation. All experiments are conducted in simulation. No physical-UAV experiment establishes whether the method remains effective under real camera calibration errors, pose uncertainty, sensing noise, and execution constraints.

Questions for authors
  • Can you reconcile the Full/Easy/Hard inconsistencies in Tables 7–8 and provide the episode counts and aggregation procedure? Were Full and subset results computed from identical rollouts or separate runs? Which corrected values should replace the corresponding entries in Tables 1 and 3?
2 comments

W1. We thank the reviewer for the valuable comments. We acknowledge that SGVC shares similarities with Merge3D's SemGeo Merger in anchor selection and token assignment mechanisms. However, the two methods differ in their design objectives. SGVC is tailored to multi-view UAV navigation, performing independent aggregation within each view to reduce redundant information while preserving spatial coverage across different viewing directions. Furthermore, the token assignment relationships established by SGVC are reused for viewing-ray aggregation in UCDG, enabling continuous directional modeling within a unified UAV body coordinate frame.Regarding the feature aggregation rule, we compute the mean of the non-anchor token features assigned to each anchor and add it to the anchor's own feature to obtain the aggregated representation. For anchors without any assigned non-anchor tokens, the original anchor features are retained. We have supplemented the revised manuscript with these aggregation details. We sincerely thank the reviewer again for the careful evaluation and constructive suggestions.

W2 & Q1. We sincerely thank the reviewer for pointing out this issue. This discrepancy resulted from a formatting error introduced during multiple rounds of table revisions throughout the manuscript preparation process. To improve evaluation efficiency, some results were obtained by independently evaluating separate subsets. In the revised manuscript, we have updated the tables with the complete results, which are consistent with those reported in the current version. We assure the reviewer that the submitted code can reproduce these experimental results. We sincerely appreciate the reviewer's careful examination and valuable suggestions.

W3. We thank the reviewer for the valuable comments. We acknowledge that selecting hyperparameters based on test-set performance may compromise the independence of the evaluation. In this study, we conducted only a coarse-grained exploration of the anchor budget (K) ((K=8,12,16,24,32)), rather than performing fine-grained hyperparameter tuning on Test-Seen. Nevertheless, we recognize that this approach may still introduce model selection bias. In future work, we will conduct a more systematic hyperparameter sensitivity analysis on an independent validation set to further assess the appropriateness of the selected configuration and ensure the independence and reliability of the evaluation.

W4. We thank the reviewer for this valuable comment. We agree that real-world UAV flight validation is important for assessing the robustness and applicability of the proposed method under practical conditions. Our current study primarily focuses on algorithmic evaluation in standardized simulation environments, and physical UAV flight experiments have not yet been conducted. Due to local low-altitude airspace regulations and geographical constraints, conducting field experiments remains challenging at this stage. To address this limitation, we have added a corresponding discussion in the revised manuscript. In future work, we plan to conduct experiments on a physical UAV platform, with particular emphasis on evaluating the effects of camera calibration errors, pose uncertainty, sensing noise, and flight execution constraints on navigation performance, thereby further assessing the robustness and applicability of our method in complex real-world environments. We sincerely thank the reviewer again for the constructive suggestion.

Thank you for the clarification. The aggregation rule is now clear, but I still find the differences from Merge3D’s SemGeo Merger relatively minor and am not convinced that they establish sufficient novelty for SGVC.

I also remain concerned about the lack of real-world UAV experiments. I understand the practical difficulties, but acknowledging this limitation and planning future experiments do not show how well the method handles calibration errors, pose uncertainty, sensor noise, or actual flight constraints.

While I appreciate the authors’ responses, these concerns remain, so I am keeping my score.

Reviewer_VqE8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NavGround3D, a vision-language navigation framework designed for unmanned aerial vehicles (UAVs). The framework integrates Semantic-Geometric Visual-token Consolidation (SGVC) with UAV-Centric Directional Grounding (UCDG), aiming to reduce visual redundancy and enhance spatial reasoning capabilities across different viewpoints.

Critical strengths

This paper addresses the challenges of visual redundancy and inconsistent directional references in multi-view drone navigation.

SGVC integrates semantic and geometric similarities for feature aggregation, while UCDG explicitly encodes 3D observation directions within a unified "drone-centric" coordinate system.

Experiments conducted on the TravelUAV and UAV-ON datasets, complemented by comprehensive ablation studies, thoroughly validate the effectiveness of the proposed modules.

On the "Test-Unseen" subset of the TravelUAV dataset, the 4B variant outperforms SpatialFly across all four metrics, demonstrating strong generalization capability.

Critical weaknesses

The methodological innovation is limited. Its core components are all based on existing, mature technologies—including attention-based token selection, similarity-based aggregation, coordinate transformation, and Transformer-based cross-view interaction. The primary contribution of this research lies in the integration and application of these techniques to unmanned aerial vehicle (UAV) navigation.

Performance gains are inconsistent when using smaller backbone networks. The 0.6B version does not outperform SpatialFly across all metrics, suggesting that performance improvements may be dependent on the capacity of the backbone network.

Efficiency evaluation is insufficient. Although the method reduces the number of visual tokens by 75%, the paper provides no comparative analysis regarding inference latency, memory footprint, or computational cost.

There is a lack of validation in real-world scenarios. Experiments were conducted exclusively in simulated environments; consequently, the method’s robustness against factors such as camera calibration errors, pose noise, and data distribution shifts—typical of real-world conditions—remains unverified.

Questions for authors

see weaknesses

2 comments

W1. We thank the reviewer for the comments. We acknowledge that the fundamental techniques adopted in this work, including attention-based selection, similarity-based aggregation, coordinate transformation, and Transformer-based interaction, have been established in prior research. We do not claim these basic operations as novel contributions. Instead, our focus is on developing a task-driven structural design tailored to the characteristics of five-view UAV navigation. Specifically, SGVC performs view-wise semantic–geometric consolidation to reduce visual redundancy while preserving the representational capacity of each viewing direction. UCDG explicitly encodes viewing directions into a unified UAV body coordinate frame using known camera geometry. More importantly, the token assignments retained by SGVC are further utilized for directional aggregation in UCDG, enabling the compressed visual features to preserve both semantic–geometric information and directional attributes, thereby providing a unified representation for cross-view spatial interaction. We sincerely thank the reviewer again for this constructive suggestion.

W2. We thank the reviewer for the valuable suggestion and agree that comparisons using the same language model backbone are important for a fair evaluation. Our method aims to improve multi-view navigation representations through the collaborative design of SGVC and UCDG. On the Test-Seen Full split, NavGround3D-0.6B improves SR by 50.62% over its corresponding baseline (23.41% to 35.26%), while SpatialFly achieves a 71.29% improvement over its baseline (22.50% to 38.54%). We acknowledge that the performance gain of our 0.6B variant is relatively smaller. However, its language model backbone has only approximately one-fifth of the parameters of the backbone used by SpatialFly. Furthermore, on the Test-Unseen Full split, NavGround3D-0.6B achieves an OSR of 51.10%, exceeding SpatialFly's 44.41%, which provides some evidence of the potential applicability of our representation design with lightweight backbones.

W3. We thank the reviewer for the valuable suggestion. We have added a comparison of inference time and navigation performance across different token budgets in Figure 4 of the supplementary material. The default setting, K=16K=16, requires 305 ms, reducing inference time by 27.2% compared with 419 ms at K=32K=32, while increasing SR from 32.23% to 35.26%. These results indicate that the selected budget provides a favorable balance between navigation performance and inference efficiency. We again thank the reviewer for the constructive suggestion to strengthen our efficiency evaluation.

W4. We thank the reviewer for raising this important concern. We acknowledge that our method has so far been evaluated only in simulation, without real-world UAV flight validation. Due to local low-altitude airspace regulations and geographical constraints, conducting field experiments remains challenging. To address this limitation, we have added a discussion in the revised manuscript and emphasized the importance of real-world validation. In future work, we plan to conduct experiments on a physical UAV platform to systematically investigate the effects of camera calibration errors, pose noise, and environmental distribution shifts, thereby further assessing the robustness and applicability of our method in real-world navigation scenarios.

The authors’ rebuttal addressed some of my concerns, I maintain my positive score.

Reviewer_mbV1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NavGround3D, a framework for UAV-VLN. The paper focuses on two problems. The first is the visual redundancy of aerial multi-view observations. The second is the inconsistency of the direction reference frame among fixed cameras. To address these, NavGround3D compresses the dense observations from five views into a small number of tokens that fuse semantic and geometric features, and it adds direction information in the UAV body frame. The authors report that the proposed method outperforms existing methods on TravelUAV and UAV-ON.

Critical strengths
  • On TravelUAV, the Qwen3-4B version of the proposed method shows slightly better performance than the previous best method (SpatialFly).
  • The authors conduct ablation studies on each module and also on the internal design of SGVC and UCDG. These results show the effectiveness of the proposed modules to some extent.
Critical weaknesses
  • The technical difference from the closest prior work, SpatialFly, is small, the performance gain is marginal, and the comparison is not fair. Both methods share the same encoder configuration (frozen CLIP ViT-L/14 + VGGT) and the TravelUAV training data. The main differences are the fusion scheme and the addition of token compression (SGVC) and direction embedding (UCDG). Moreover, whereas SpatialFly uses Qwen2.5-3B, only the Qwen3-4B variant of the proposed method outperforms it, and the 0.6B variant falls below SpatialFly in SR on TravelUAV.
  • The problem that UCDG is designed to address, namely the inconsistency of directional reference frames across fixed cameras, is not shown to be a real problem. Since the cameras are rigidly mounted on the UAV with known mounting poses, the body-frame direction of each token is constant, so there is no missing information or ambiguity to resolve.
Questions for authors

Can the proposed method still show a clear advantage over SpatialFly when both are evaluated with the same backbone (i.e., SpatialFly with Qwen3-4B, or the proposed method with Qwen2.5-3B)?

1 comment

W1&Q1. We thank the reviewer for the valuable suggestion and agree that comparisons using the same language model backbone are important for ensuring a fair evaluation. Our work primarily aims to improve multi-view UAV navigation representations through the complementary design of SGVC and UCDG. On the Test-Seen Full split, NavGround3D-0.6B achieves a 50.62% relative improvement in success rate (SR) over its corresponding baseline (from 23.41% to 35.26%), while SpatialFly achieves a 71.29% relative improvement over its baseline (from 22.50% to 38.54%). We acknowledge that the relative performance gain of NavGround3D-0.6B is smaller. However, its language model backbone has only approximately one-fifth of the parameters of the backbone used by SpatialFly. Furthermore, on the Test-Unseen Full split, NavGround3D-0.6B achieves an oracle success rate (OSR) of 51.10%, exceeding SpatialFly's 44.41%. These results suggest the potential of the proposed representation design when applied to lightweight backbone.

W2. We thank the reviewer for this valuable suggestion. We agree that, for a UAV equipped with five fixed-view cameras with known mounting poses and calibration parameters, the viewing direction associated with each image location can be determined in the UAV body frame. However, the availability of such geometric information does not necessarily imply that it is explicitly encoded in the extracted visual features. Conventional visual encoders primarily capture scene content and semantics, without necessarily incorporating the known directional relationships among camera views. Therefore, UCDG aims to explicitly incorporate these known directional cues into visual tokens, providing a unified UAV-centric spatial reference for cross-view interaction, rather than recovering missing geometric information or resolving directional ambiguity.

We also acknowledge that our original descriptions of "directional ambiguity" and "missing directional information" were imprecise. To address this concern, we have revised the relevant descriptions in the manuscript. In the Introduction, we have replaced the original statement:

"Second, due to payload constraints, UAVs typically rely on a small set of fixed-orientation cameras to obtain discrete view observations. Since different views are associated with different local directional references, unified spatial relation modeling and directional representation become more challenging."

with the following revised description: "Second, due to payload constraints, UAVs typically rely on a limited number of fixed-orientation cameras to obtain discrete multi-view observations. Although the relative poses of the cameras are known, visual features extracted from different views do not explicitly encode their directional information in a unified UAV body frame. Consequently, constructing a multi-view spatial representation with a unified directional reference is essential."

In Figure 1, we have replaced the notion of directional ambiguity with the underutilization of directional information.

In Section 3.3, we have revised the opening statement to emphasize the explicit use of known camera-direction information to construct a unified directional representation in the UAV body frame.

These revisions more accurately reflect the motivation and role of UCDG. We sincerely thank the reviewer again for the constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.