acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Volumetric Reasoning for Aerial Vision-Language Navigation

Abstract

Aerial vision-language navigation (AVLN) requires unmanned aerial vehicles (UAVs) to follow natural-language instructions and reach specified destinations in large-scale, continuous 3D environments. Navigation-relevant cues are often distributed over long distances and observed at varying spatial scales, placing substantial demands on an agent's capacity for 3D spatial understanding and reasoning. This work introduces Hierarchical Volumetric Reasoning for AVLN, enabling the agent to reason over 3D scene structure from global layouts to fine-grained local regions. The reasoning is grounded in a Multi-Scale Voxel Map that organizes the observed environment into a coarse-to-fine 3D volume hierarchy and encodes geometric and semantic cues at each level. At each level, an instruction-conditioned decision token aggregates relevant voxel features, while its attention scores route subsequent reasoning to finer spatial regions. The final decision token is combined with navigation history to predict the next action. Experiments on AerialVLN-s show that our agent achieves 16.2% SR and 12.5% SDTW on seen environments, and 8.8% SR and 3.9% SDTW on unseen environments, with an inference latency of 0.09s per navigation step. Extensive ablation studies further validate the effectiveness of our core designs. The code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-and-langugae navigation, aerial robots, spatial reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7fGiOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Hierarchical Volumetric Reasoning for aerial vision-language navigation. The method first lifts RGB-D observations into a Multi-Scale Voxel Map that represents geometric and semantic information at coarse-to-fine resolutions. It then uses instruction-conditioned decision and proxy tokens to retrieve relevant voxel features, while attention scores route reasoning toward selected regions at finer scales. The final decision token is combined with a recurrent state to predict navigation actions. Experiments on AerialVLN-s show improvements over learning-based baselines and competitive performance relative to LLM-based methods, together with a reported inference latency of 0.09 seconds per step.

Critical strengths
  1. A clear and efficient way to bring 3D structure into aerial VLN. Most aerial agents reason over 2D views, grid maps or topological graphs. This paper lifts each observation into a coarse-to-fine voxel volume and lets the instruction decide which regions to refine. The design makes sense for UAVs, where relevant cues appear at very different heights and scales. It also stays cheap: 0.09 s and 1.71 GB per step, compared with seconds and tens of GB for LLM-based agents. Among learning-based agents, the gain on val seen is large.
  2. The evaluation is more careful than usual for this benchmark. Main results are averaged over five runs with standard deviations. The ablations cover the representation, the reasoning levels, the voxel selection ratio and the routing budget, and they show that routing is what keeps the finer levels within memory. The paper also measures occupancy quality next to navigation performance, which helps connect the quality of the 3D representation to downstream decisions.
Critical weaknesses
  1. It is unclear where the improvement comes from. Most of the gain in the ablation comes from adding MSVM, and the instruction-driven reasoning on top adds a smaller amount. No baseline receives the same depth input and auxiliary supervision without the volumetric structure, and the starting baseline is weaker than existing learning-based agents. It is hard to tell whether the gain comes from the voxel representation or simply from the extra depth input and occupancy labels.
  2. The claims go beyond what the representation and evaluation support. The volume is rebuilt from a single frame at every step, covers a limited range around the agent, and carries no information across steps. It is closer to a 3D view of the current observation than to a scene map, so the claim of reasoning from global scene layout is overstated. The evaluation is also limited. Only AerialVLN-s is used. The comparison with LLM-based agents is up to date, but the most recent learning-based baseline is LAG from 2023, even though the agent is itself learning-based. This makes it hard to tell how much of the gain over learning-based methods would remain against recent non-LLM aerial agents.
Questions for authors
  1. Is DtD_t at test time the ground-truth simulator depth, and are the extrinsics in Eq. (2) ground-truth poses? Do LAG, Seq2Seq and CMA in Table 1 receive the same depth input?
  2. How does the agent perform with MSVM trained without occupancy supervision, and with a 2D or BEV representation that receives the same depth input and occupancy losses? This would show whether the gain comes from volumetric reasoning or from the extra input and supervision.
  3. HVR is trained only through Lnav\mathcal{L}_{\mathrm{nav}}, and TopK in Eq. (6) is non-differentiable. How does a parent voxel that should have been selected but was not receive a learning signal? How often does the routed region at the middle and fine levels contain the next expert waypoint or the goal, and how does this routing compare with occupancy-based or random routing under the same budget?
  4. The volume covers only [−8 m, 8 m] vertically and 51.2 m forward. How does the agent handle steps where the next landmark or the goal lies outside this range?
  5. The volume is rebuilt from the current frame at every step. Once a landmark leaves the field of view, is the GRU state the only memory that carries it? Did you try fusing voxels across steps using poses?
  6. The most recent learning-based baseline in Table 1 is LAG (2023). Can you compare with at least one recent non-LLM aerial agent under the same protocol?
7 comments
AuthorsOpenReview ↗

Responses to Reviewer 7fGi (1/5)

We sincerely thank Reviewer 7fGi for the constructive feedback. We provide point-to-point responses below, and all corresponding revisions have been incorporated into the revised paper and highlighted in red. All references (e.g., Lines, Tables, Sections) correspond to the updated version.


W1-1: "...and the instruction-driven reasoning on top adds a smaller amount."

A1-1: Thanks for your thoughtful comments. We would like to clarify that the two modules play different roles: MSVM builds the multi-scale representation that encodes evidence across scales, while HVR performs cross-modal reasoning at the appropriate granularity. The +1.3% from HVR is obtained on top of an already strong MSVM baseline (SR 7.5%), and our ablation in Table 4 shows that reasoning over single-scale inputs is substantially worse, indicating that HVR is not a marginal add-on but the mechanism that makes multi-scale evidence usable.


W1-2: "No baseline receives the same depth input and auxiliary supervision ... the starting baseline is weaker. It is hard to tell whether the gain comes from the voxel representation or simply from the extra depth input and occupancy labels. "

A1-2: We appreciate your careful review. We address your concern by adding two new models: i) 2D w/ pretrain: 2D baseline model which is pretrained under the same depth and semantic supervision, and ii) 3D w/o pretrain: MSVM is optimized end-to-end through navigation training. As shown in table below, two observations can be drawn:

i) Volumetric representation is superior. Without geometry and semantic supervision, volumetric representation outperforms 2D representation (2.7% -> 6.5% in SR, 0.8% -> 2.4% in SDTW). With such supervision, volumetric representation improves SR from 4.1% -> 7.5%, and SDTW from 1.2% -> 3.1%. Both demonstrate the effectiveness of multi-scale voxel map.

ii) Auxiliary supervision further helps. Under the same representation, adding geometry and semantic supervision is itself beneficial: it improves the 2D model from 2.7% -> 4.1% in SR, 0.8% -> 1.2% in SDTW, and the 3D model from 6.5% -> 7.5% in SR, 2.4% -> 3.1% in SDTW. The supervision effect is thus consistently positive but smaller than the representation effect in both settings.

Taken together, of the total SR gain from 2.7% to 7.5% (+4.8%), roughly +1.4% is attributable to supervision and +3.4% to the volumetric representation, confirming that the two factors are both beneficial and that the voxel representation is the primary source of the improvement.

Finally, we recognize the original weak 2D baseline (2D w/o pretrain, Table 3 row 1) and its potential unfairness due to the unmatched supervision between our proposed method. In the revised version, we include 2D w/ pretrain as the matched counterpart under identical supervision, so that the representation effect is assessed fairly (Table 3).

#NE ↓\downarrow{}SR ↑\uparrow{}OSR ↑\uparrow{}SDTW ↑\uparrow{}
12D w/o pretrain154.82.712.60.8
22D w/ pretrain145.24.115.21.2
33D w/o pretrain125.36.521.82.4
43D w/ pretrain (Ours)118.27.524.63.1

W2-1: "... so the claim of reasoning from global scene layout is overstated."

A2-1: We apologize for this confusion. Our intended meaning was that coarse-scale reasoning captures the overall spatial structure within this observed volume (L216-217), before finer-scale reasoning focuses on selected regions. This does not imply access to a complete scene map.

In the revised version, we replace "global scene layout" with "coarse spatial layout" to avoid any ambiguity.

AuthorsOpenReview ↗

Responses to Reviewer 7fGi (2/5)

W2-2: "The evaluation is also limited. Only AerialVLN-s is used"

A2-2: Thanks for your comment. We evaluate our agent on OpenFly [1]. We retrain our agent using its official training split, using the same architecture, two-stage training recipe, and other key hyperparameters (e.g., volume resolution, perception range, number of selected voxels in HVR, etc.). As shown in the table below, our agent outperforms all baselines on test seen split (e.g., +14.5% SR over NaVila, +0.5% SR over OpenFly-Agent). On test unseen split, our agent remains competitive, achieving 20.7% SR and 17.2% SPL. The results demonstrate the proposed method remains effective across benchmarks.

AgentLLMTest-Seen NE ↓\downarrowTest-Seen SR ↑\uparrowTest-Seen OSR ↑\uparrowTest-Seen SPL ↑\uparrowTest-Unseen NE ↓\downarrowTest-Unseen SR ↑\uparrowTest-Unseen OSR ↑\uparrowTest-Unseen SPL ↑\uparrow
Seq2Seq--2052.924.32.62292.120.61.1
CMA--1615.428.14.82174.624.42.1
AerialVLN--1397.530.06.82147.328.14.4
NaVid✓\checkmark15313.038.211.621010.827.25.0
NaVila✓\checkmark13220.353.517.820214.742.19.6
OpenFly-Agent✓\checkmark9334.364.324.915422.656.219.1
Ours--9434.867.125.016420.756.817.2

We have added this quantitative comparison on OpenFly in the revised version (Appendix B.5, Table 11, L954-977).

[1] OpenFly: A Comprehensive Platform for Aerial Vision-Language Navigation. ICLR 2026.

AuthorsOpenReview ↗

Responses to Reviewer 7fGi (3/5)

W2-3: "but the most recent learning-based baseline is LAG from 2023, even though the agent is itself learning-based. "

A2-3: Thank you for highlighting this gap in our baseline coverage. With the flourishing of LLM/MLLM, most recent aerial agents developed on AerialVLN-s are LLM/MLLM-based. To address this, we adapt two recent learning-based, non-LLM-based aerial agents MGP (ICCV 2025) [1], HETT (AAAI 2026) [2] to the AerialVLN-s setting using the same training and evaluation protocol.

The results are shown in the table below, our agent achieves consistent performance gains, e.g., on val seen +5.1% SR / +6.1% SDTW over MGP, +3.9% SR / +5.3% SDTW over HETT; on val unseen +3.6% SR / +2.8% SDTW over MGP, +3.0% SR / +2.3% SDTW over HETT.

AgentPano.DepthLLMNE ↓\downarrow{}(val seen)SR ↑\uparrow{}(val seen)OSR ↑\uparrow{}(val seen)SDTW ↑\uparrow{}(val seen)NE ↓\downarrow{}(val unseen)SR ↑\uparrow{}(val unseen)OSR ↑\uparrow{}(val unseen)SDTW ↑\uparrow{}(val unseen)
MGP [1]--✓\checkmark--95.111.124.96.4130.15.214.81.1
HETT [2]--✓\checkmark--93.212.328.57.2127.05.816.21.6
Ours--✓\checkmark--84.116.235.412.5109.48.826.53.9

In the revised version, we incorporate MGP [1] and HETT [2] into Table 1, update the experiment section (Secs. 4.1, 4.2).

[1] CityNav: A large-scale dataset for real-world aerial navigation. ICCV 2025.

[2] History-Enhanced Two-Stage Transformer for Aerial Vision-and-Language Navigation. AAAI 2026.


Q1: "is DtD_t at test time the ground-truth depth ... are extrinsics ground-truth poses? Do LAG, ... receive the same depth input?"

A1: i) Yes, both DtD_t and camera poses are ground-truth from simulator. ii) Yes, the baseline models LAG, Seq2Seq and CMA all receive the same ground-truth depth as in the official implementations.

In the revised version, we clarify the source of depth map and camera pose (Sec. 3.5, L292-293), and update Table 1 by adding a depth column.


Q2: "How does the agent perform with MSVM trained without occupancy supervision, and with a 2D or BEV representation that receives the same depth input and occupancy losses?"

A2: Please see W1-2.


Q3-1: "HVR is trained only through LnavL_{nav} , and TopK in Eq. (6) is non-differentiable. How does a parent voxel that should have been selected but was not receive a learning signal? "

A3-1: Thanks for your comments. Recall that HVR employs a multi-layer transformer (MLT, with parameters θ\theta) to perform cross-modal reasoning between an instruction-conditioned learnable decision token (along with proxy tokens ), and multi-scale (candidate) voxel features. Before Top-K selection, all current-scale candidate voxels participate in attention aggregation (Eq. 5). The updated tokens are passed to subsequent scales and ultimately used for action prediction. Consequently, Lnav\mathcal{L}_{\mathrm{nav}} optimizes θ\theta, including the attention projections, to aggregate voxel evidence useful for navigation. Spatial routing (Top-k) reuses these task-trained attention scores to select parent regions for finer-scale reasoning.

Reviewer_EGbNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies aerial vision-language navigation using an explicit 3D representation. A Multi-Scale Voxel Map (MSVM) is pretrained with geometric and semantic occupancy supervision and then frozen during policy learning. Hierarchical Volumetric Reasoning (HVR) uses instruction-conditioned tokens to read the map at three scales, expanding regions selected by decision-token attention. A recurrent policy combines this representation with image features and navigation history. On AerialVLN-s, the method reports 16.2% SR and 12.5% SDTW on val_seen, and 8.8% SR and 3.9% SDTW on val_unseen.

Critical strengths

The paper brings multi-scale voxel representation and coarse-to-fine candidate selection into AVLN, and this is reasonably motivated. Reasoning over a flat set of fine voxels is infeasible (Table 7 shows OOM), and routing keeps the cost low. The efficiency evidence is solid: Table 2 shows 0.09 s / 1.71 GB per step, compared with 12.51 s / 32.76 GB for CityNavAgent, while unseen SR is clearly above LAG (8.8 vs 5.1). This matters for practical UAV deployment.

Table 5 gives reasonable evidence for the local design choices of MSVM (top-k ratio, 3D vs 2D attention in refinement, 3D conv). The limitations section (App. C.1) is also candid about the missing spatial memory, the static scenes, and the lack of real-world tests.

Critical weaknesses

Table 4 rows 2–4 (single-scale reasoning, SR 4.8 / 6.4 / 6.6) appear to include MSVM, yet all are below Table 3 row 2 (MSVM without HVR, SR 7.5). If so, reasoning at a single scale is worse than no reasoning, and the paper does not explain this. It is also never stated how the voxel features are used in Table 3 row 2. Table 4 therefore mixes the effect of MSVM with that of HVR, and the +2.1% “3D vs 2D” gain cannot be attributed to HVR.

The seen/unseen gap is large (SR 16.2 → 8.8, SDTW 12.5 → 3.9) compared with LAG (7.2 → 5.1, 2.4 → 1.4), which suggests overfitting to training scenes. The text highlights seen-split gains over LLM-based agents, but on unseen the method is below CityNavAgent and LookasideVLN on SR and clearly below on OSR (26.5 vs 35.2 / 36.0). That gap is not discussed.

Spatial routing, a core claim, is never isolated. Table 7 only varies the budget G, with attention-based selection in every row. There is no comparison with random, uniform, or occupancy-score selection at the same budget, and no visualization showing that routed voxels fall on instruction-referred landmarks.

The intro says +8.1% SR on val seen but Sec. 4.2 and Table 1 give +9.0%.

Questions for authors

What exactly are the models in Table 3 row 2 and Table 4 row 1, and do Table 4 rows 2–4 include MSVM? Why are the single-scale variants worse than MSVM without HVR (7.5 SR)? Can you add a 2D baseline trained with the same depth/semantic supervision (no voxel lifting) to separate the representation effect from the supervision effect?

Can you compare attention-based routing with random, uniform, and occupancy-score routing at the same budget, and show routed regions against the referred landmarks? Also, since h_{t-1} is not an input to HVR, how does the reasoning know the current sub-instruction in multi-stage instructions like Fig. 3?

Can you add at least one more recent learning-based baseline and report std for the ablations? Please also state explicitly whether val unseen was used for hyperparameter selection.

5 comments
AuthorsOpenReview ↗

Responses to Reviewer EGbN (1/4)

We sincerely thank Reviewer EGbN for the constructive feedback. We provide point-to-point responses below, and all corresponding revisions have been incorporated into the revised paper and highlighted in red. All references (e.g., Lines, Tables, Sections) correspond to the updated version.


W1-1: " Table 4 rows 2–4 (single-scale reasoning, SR 4.8 / 6.4 / 6.6) appear to include MSVM, yet all are below Table 3 row 2 (MSVM without HVR, SR 7.5). If so, reasoning at a single scale is worse than no reasoning, and the paper does not explain this. "

A1-1: Thank you for careful review. First, these configurations differ in their available input information. Table 3, row 2, uses all multi-scale voxel features (as clarified in W1-2), whereas the original Table 4, rows 2–4, use features from only one scale (as clarified in Q1-1). Consequently, this comparison does not isolate the effect of adding reasoning: it changes both the input scales and the feature-aggregation mechanism. The lower scores therefore cannot be attributed to reasoning alone. Second, the reason may stem from the perception gap between these two settings. Task-relevant evidence in aerial navigation is distributed across spatial scales (L033-034, Fig. 3), a single-scale voxel input may therefore expose the reasoning module to only part of these evidences, so its cross-modal associations are formed on an incomplete basis and may mislead action prediction. This is further supported by the monotonic trend within rows 2–4 (SR 4.8 →\rightarrow 6.4 →\rightarrow 6.6 from coarse to fine): performance depends strongly on which scale is used, indicating that no single fixed granularity can match the scale at which evidence appears in all scenes.


W1-2: "It is also never stated how the voxel features are used in Table 3 row 2."

A1-2: We apologize for the confusion. We clarify how voxel features are used in Table 3 row 2 below.

To investigate the effectiveness of perception side along (MSVM), we pool the multi-scale voxel features and fuse them into a single vector, which is concatenated with instruction feature and the pooled visual feature, to form the multimodal input for the GRU policy. So, there is no explicit cross-modal reasoning process before action prediction in this model.

Here we also clarify the 2D baseline in Table 3 row 1, it pools the 2D visual, depth feature, and concatenate them with language feature to form the multimodal input for a GRU policy network.

In the revised version, we provide more details of each model variant (Appendix. A.2).


W1-3: "Table 4 therefore mixes the effect of MSVM with that of HVR, ... gain cannot be attributed to HVR."

A1-3: We apologize for the confusion and appreciate your careful review. Table 4 was intended to investigate the effectiveness of hierarchical reasoning approach in HVR (i.e., hierarchical reasoning vs single-scale reasoning), and the consistent improvement over single-scale volumetric reasoning (+4.0%/2.4%/2.2% SR over coarse/middle/fine reasoning) highlights the importance of exploiting the multi-scale spatial and semantic cues in aerial navigation (L423–434). We agree that the original Row #1 vs Row #2/3/4 comparison conflates the effects of representation and reasoning. However, the two effects are separately isolated in Table 3: MSVM (representation) brings +3.4% SR over the 2D baseline under identical depth/semantic supervision, and HVR (reasoning) further improves SR by +1.3% on top of MSVM under the same multi-scale inputs.

In the revised version, we remove Table 4 row 1, update the caption of Table 4 to "Ablation study of hierarchical reasoning in HVR" and the corresponding analysis to more clearly position its role (Sec. 4.4, Table 4).

AuthorsOpenReview ↗

Responses to Reviewer EGbN (2/4)

W2-1: "The seen/unseen gap ... ,which suggests overfitting to training scenes."

A2-1: Thank you for the comment. We acknowledge that our method exhibits a larger seen-to-unseen performance gap than LAG. However, this does not imply that its gains are confined to seen environments.

Theoretically, seen-to-unseen degradation is a widely documented challenge in learning-based VLN. Its magnitude can depend on training-environment diversity and visual distribution shift [1,2], as well as the mismatch between expert demonstration states and policy-induced states, which can cause compounding errors in behavior cloning [3].

Empirically, on val unseen, our method achieves 8.8% SR and 3.9% SDTW, outperforming LAG by 3.7% and 2.5%, respectively. It also exceeds the LLM-based OpenFly agent by 1.2% in SR. These results demonstrate that our method retains competitive performance in unseen environments.

[1] EnvEdit: Environment Editing for Vision-and-Language Navigation. CVPR 2022.

[2] Vision-and-Language Navigation via Causal Learning. CVPR 2024.

[3] Vision-Language Navigation with Energy-Based Policy. NeurIPS 2024.


W2-2: "The text highlights seen-split gains over LLM-based agents, but on unseen the method is below CityNavAgent and LookasideVLN ... The gap is not discussed"

A2-2: Thanks for pointing this out. We acknowledge the performance gap between LLM-based agents, which is mainly attributed to two differences.

First, pretrained priors. The LLM/MLLM-based agents (i.e., CityNavAgent (GPT-4V), LookasideVLN (Qwen2.5-VL-72B)) draw on extensive semantic and commonsense knowledge through large-scale pretraining, which provides a strong inductive bias for unseen scenes. While our agent is learned via behavior cloning on limited demonstrations.

Second, spatial memory. CityNavAgent and LookasideVLN explicitly store a global topologic map or landmark knowledge base through pre-exploration, which provides persistent scene-level memory of the evaluated scenes. In contrast, our agent starts each episode without a prebuilt scene map or landmark database, voxel representation is reconstructed at each step and history is carried through a compact GRU state (Eq. 7). These differences in available scene information may partly contribute to the performance gap and should be considered when interpreting unseen-split results. We also note this as a trade-off between lightweight, map-free inference and cross-scene robustness, and regard explicit memory as a promising future direction (Appendix C.1).

In the revised version, we discuss the unseen performance gap (Sec. 4.2, L363-366).

AuthorsOpenReview ↗

Responses to Reviewer EGbN (3/4)

W3-1: "Spatial routing, a core claim, is never isolated. "

A3-1: Thanks for this valuable suggestion. We add comparison between attention-based routing with random, uniform, occupancy-based routing. We replace only the parent-selection criterion in Eq. (6), keeping child-expansion rule, other model and training settings, and candidate budgets unchanged (i.e., G1=1280G_1=1280, G2=G3=256G_2=G_3=256).

On val unseen split, attention-based routing consistently outperforms alternatives, with +4.6% SR/+2.6% SDTW over random routing, +4.1% SR/+2.5% SDTW over uniform routing, and +2.6% SR/+1.7% SDTW over occupancy-based routing. These results demonstrate the effectiveness of using instruction-conditioned attention to guide finer reasoning.

Spatial Routing StrategyNE ↓\downarrow{}SR ↑\uparrow{}OSR ↑\uparrow{}SDTW ↑\uparrow{}
Random146.84.215.71.3
Uniform140.54.716.01.4
Occupancy127.66.220.32.2
Attention (Ours )109.48.826.53.9

In the revised version, we add the quantitative comparison and analysis of different spatial routing method (Sec. 4.4, Table 5).


W3-2: "... and no visualization showing that routed voxels fall on instruction-referred landmarks."

A3-2: Thanks for your comment. We add spatial routing attention visualizations overlaid on the multi-scale voxel maps (Appendix B.4, Fig. 4). Across four examples, higher attention scores and selected finer-scale voxels appear around instruction-referred landmarks, including the pillar stone, lamp post, shop entrance, and terrace. The visualizations illustrate how coarse-scale attention guides finer-scale reasoning toward selected landmark regions, offering qualitative indications of a correspondence between routed regions and instruction-referred landmarks.


Q1-1: "What exactly are the models in Table 3 row 2 and Table 4 row 1, and do Table 4 rows 2–4 include MSVM ?"

A1-1: Sorry for the confusion.

i) For model in Table 3 row 2, please see W1-2.

ii) The model in the original Table 4 row 1 is the same model in Table 3 row 1. It serves as the 2D baseline, which pools the visual, depth feature, and concatenate them with language feature to form the multimodal input for a GRU policy network.

iii) Yes, the original Table 4 rows 2-4 include MSVM, and we further clarify that each variant reasons over one specific voxel feature in MSVM.

In the revised version, we provide more details of each model variant (Appendix. A.2).


Q1-2: "Why are the single-scale variants worse than MSVM without HVR ?"

A1-2: Please see W1-1.


Q1-3: "Can you add a 2D baseline trained with the same depth/semantic supervision (no voxel lifting) to separate the representation effect from the supervision effect?"

A1-3: Thanks for the valuable suggestion. We pretrain the original 2D baseline's image backbone with depth and semantic prediction tasks, and then finetune the whole policy network end-to-end. Results are reported below.

i) Row #1 vs Row #2: 2D w/ pretrain obtains additional gains over the 2D baseline without extra supervision (+1.4% in SR, +0.4% in SDTW), indicating that supervision itself is beneficial.

ii) Row #2 vs Row #4: under identical supervision, the 3D model substantially outperforms the 2D model (+3.4% in SR, +1.9% in SDTW), which demonstrate that our MSVM provide a better perceptional representation for the subsequent decision-making.

#NE ↓\downarrow{}SR ↑\uparrow{}OSR ↑\uparrow{}SDTW ↑\uparrow{}
12D w/o pretrain154.82.712.60.8
22D w/ pretrain145.24.115.21.2
33D w/o pretrain125.36.521.82.4
43D w/ pretrain (Ours)118.27.524.63.1

In the revised version, we update Table 3 row 1 to the 2D model with depth/semantic pretraining for setting a more controllable comparison (Sec. 4.4, Table 3).

Reviewer_ndAoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses Aerial Vision-Language Navigation (AerialVLN): a UAV agent must follow natural-language instructions over long-horizon (avg. 321 m) trajectories in large-scale continuous 3D scenes. The core claim is that existing agents reason either in 2D or over fixed-granularity spatial abstractions (grids, topological graphs, global scene context), diluting scale-varying spatial cues.

Results on AerialVLN-s: 16.2 SR / 12.5 SDTW (val seen), 8.8 SR / 3.9 SDTW (val unseen), at 0.09 s/step and 1.71 GB on a single RTX 4090, beating all learning-based baselines by large margins and LLM-based agents in the same egocentric setting, while using orders of magnitude less compute than MLLM agents. Ablations cover MSVM/HVR components, reasoning granularity, top-k ratios, 3D attention, proxy token count, and routing budgets.

Critical strengths
  1. Well-motivated, coherent design point. The paper identifies a real gap — aerial VLN cues span scales (global layout → local landmarks), while prior maps use fixed granularity — and addresses it with a representation (multi-scale voxels) and a reasoning procedure (coarse-to-fine routing) that are mutually reinforcing rather than bolted together. The hierarchical structure follows naturally from the observation that fine-scale reasoning over all voxels is OOM-intractable (Table 7).

  2. Favorable performance-efficiency trade-off. The efficiency story (Table 2) is the paper's most decision-relevant strength: 16.2 SR at 0.09 s/step / 1.71 GB vs. CityNavAgent's 13.9 SR at 12.51 s / 32.76 GB. For UAV deployment, an LLM-free, real-time agent that beats MLLM-based agents on the seen split without panoramic sensing is a meaningful result. On unseen, it still beats OpenFly (8.8 vs 7.6 SR) in the identical egocentric setting.

  3. Systematic ablation coverage. The paper ablates nearly every design axis: overall components (Table 3), reasoning granularity including each scale in isolation (Table 4), top-k ratios jointly reporting occupancy quality and downstream navigation (Table 5a — nice touch, showing the perception-reasoning transfer), 3D vs 2D attention and 3D conv (Table 5b), proxy token count (Table 6), and routing budgets (Table 7). Five-run means with std are reported for the main results.

  4. Honest limitations section. Appendix C.1 candidly lists static environments, lack of spatial memory across steps, and sim-to-real gaps, including the right citations (world models, global 3D map) for future directions. This increases trust in the rest of the paper.

Critical weaknesses
  1. The headline mechanism, attention-based spatial routing, is never isolated. Table 7 varies the budget (G2, G3) but never compares attention-based parent selection against a control (e.g., random TopK parent selection, occupancy-probability-based selection, or center-biased selection). Since top-k occupancy gating already prunes the volume, it is entirely possible that random routing at the same budget achieves similar SR. The +1.3% SR of routing (1280→256) could be a compute/regularization effect rather than evidence-grounded routing. This matters because "spatial routing guided by instruction-relevant attention" is the paper's central conceptual claim.

  2. All results are on AerialVLN-s. No evaluation on CityNav, OpenFly, AVDN, or the object-goal variants the related-work section itself surveys. Given that the method's components (depth discretization into 100 bins, 51.2 m range, occupancy supervision from back-projected depth) are dataset-engineering-heavy, generalization of the claims, not just of the agent, is unverified.

  3. Unseen SR of 8.8% means >91% of episodes fail; even the seen split is at 16.2%. More concerning, the generalization drop (16.2 → 8.8 SR) is much steeper than the strongest panoramic LLM agents (CityNavAgent 13.9 → 11.7; LookasideVLN 14.7 → 12.6), whose unseen NE (60.2) is far lower than ours (109.4). The paper frames this as a sensing-modality difference, but no analysis (e.g., per-scene breakdown, failure taxonomy) is offered to support that the gap comes from sensing rather than from the representation or BC training. The related CityNavAgent failure-case analysis (instruction ambiguity 45.4% / perception 34.1% / reasoning 20.5% on AerialVLN-s) suggests the bottleneck may not be where this paper invests.

Questions for authors
  1. How does attention-based parent selection compare to random TopK parent selection at identical budgets (G2=G3=256)? If random routing matches it, the spatial-routing claim needs revision.

  2. What is navigation performance when the MSVM is replaced by ground-truth occupancy/semantics? This separates the perception bottleneck from the reasoning bottleneck.

  3. Is the seen-split gain over LAG 8.1% or 9.0% SR? The introduction and section 4.2 disagree.

  4. Is DtD_t ground-truth simulator depth? What happens to occupancy IoU and SR when depth is corrupted with realistic noise (e.g., stereo/MiDaS-style error)?

5 comments
AuthorsOpenReview ↗

Responses to Reviewer ndAo (1/4)

We sincerely thank Reviewer ndAo for the constructive feedback. We provide point-to-point responses below, and all corresponding revisions have been incorporated into the revised paper and highlighted in red.


W1-1: "The headline mechanism, attention-based spatial routing, is never isolated."

A1-1: Thanks for this valuable suggestion. First, we compare the attention-based routing with random, uniform, occupancy-based routing. We replace only the parent-selection criterion in Eq. (6), keeping child-expansion rule, model and training settings, and candidate budgets unchanged (i.e., G1=1280G_1=1280, G2=G3=256G_2=G_3=256).

The results are reported below. On val unseen split of AerialVLN-s, attention-based routing outperforms random routing by 4.6% in SR, 2.6% in SDTW, uniform routing by 4.1% in SR, 2.5% in SDTW, and occupancy-based routing by 2.6% in SR, 1.7% in SDTW, respectively. These results demonstrate the effectiveness of the instruction-conditioned attention score as guidance for determining subsequent reasoning regions.

Second, we would like to clarify that top-k occupancy gating is used for voxel refinement only during MSVM pretraining (Eq.1, L295-296). During navigation, the agent takes the full MSVM (L243-244), and uses attention map to select which regions enter finer-scale reasoning (Eq.6, L243-247). Thus, the reasoning does not operate on a candidate set already filtered by occupancy gating, and as can be seen, random routing at the same budget decreases the navigation performance by 4.6% in SR and 2.6% in SDTW.

Spatial Routing StrategyNE ↓\downarrow{}SR ↑\uparrow{}OSR ↑\uparrow{}SDTW ↑\uparrow{}
Random146.84.215.71.3
Uniform140.54.716.01.4
Occupancy127.66.220.32.2
Attention (Ours)109.48.826.53.9

In the revised version, we add the quantitative comparison and analysis of different spatial routing strategy (Sec. 4.4, Table 5, L436–446).


W1-2: "The +1.3% SR of routing (1280→256) could be a compute/regularization effect rather than evidence-grounded routing."

A1-2: Thank you for this insightful observation. We investigate whether this improvement arises from a computational regularization or the instruction-guided spatial routing. Specifically, we compare attention-guided spatial routing with occupancy-based spatial routing under two different voxel budgets (i.e., G1=1280,G2=1280,G3=1280G_1=1280,G_2=1280, G_3=1280 and G1=1280,G2=256,G3=256G_1=1280,G_2=256, G_3=256). From the table below, several observations can be drawn:

i) Row #1 vs #3: Under occupancy-based routing, reducing G2G_2 and G3G_3 from 1280 to 256 degrades navigation performance (-0.2% in SR, -0.1% in SDTW). This indicates that reducing voxel budget alone does not introduce a beneficial regularization effect.

ii) Row #2 vs. #4:. Under attention-based routing, reducing G2G_2 and G3G_3 from 1280 to 256 improves all navigation performance (+1.3% in SR, +0.9% in SDTW). Since the two routings differ only in how spatial regions are selected, the opposite trends indicate that attention-guided spatial routing brings additional gains by focusing on more informative evidence.

Taken together, these results support that the +1.3% SR stems from evidence-grounded, instruction-guided spatial routing rather than just from a compute/regularization effect.

#Spatial Routing StrategyNE ↓\downarrow{}SR ↑\uparrow{}OSR ↑\uparrow{}SDTW ↑\uparrow{}
1Occupancy (G1=1280,G2=1280,G3=1280G_1=1280,G_2=1280, G_3=1280)124.36.420.72.3
2Attention (G1=1280,G2=1280,G3=1280G_1=1280,G_2=1280, G_3=1280)112.07.524.63.0
3Occupancy (G1=1280,G2=256,G3=256G_1=1280,G_2=256, G_3=256)127.66.220.32.2
4Attention (G1=1280,G2=256,G3=256G_1=1280,G_2=256, G_3=256) (Ours)109.48.826.53.9

In the revised version, we add this experiment and analysis (Appendix B.3, Table 9).

AuthorsOpenReview ↗

Responses to Reviewer ndAo (2/4)

W2: "No evaluation on CityNav, OpenFly, AVDN, or other object-goal variants. Given that the method's components are dataset-engineering-heavy (depth discretization into 100 bins, 51.2 m range, occupancy supervision from back-projected depth), generalization of the claims, not just of the agent, is unverified"

A2: Thanks for the thoughtful comment. We add experiments on OpenFly [1], retraining our agent on its training split and evaluating on both test-seen and test-unseen splits. We retain the same two-stage training procedure: MSVM pretraining with semantic occupancy supervision, followed by navigation policy training with MSVM frozen. We also reuse the key MSVM and HVR hyperparameters from AerialVLN-s, including voxel resolution, perception range, and number of candidate voxels in HVR, etc.

On test-seen, our method achieves 34.8% SR, 67.1% OSR, and 25.0% SPL, exceeding OpenFly-Agent by 0.5%, 2.8%, 0.1%, respectively. On test-unseen, our method achieves 20.7% SR, outperforming AerialVLN by 13.4% and the LLM-based NaVid and NaVila baselines by 9.9% and 6.0%, and is close to OpenFly-Agent. These results provide additional evidence that the training recipe and core design choices remain effective beyond AerialVLN-s.

AgentLLMTest-Seen NE ↓\downarrowTest-Seen SR ↑\uparrowTest-Seen OSR ↑\uparrowTest-Seen SPL ↑\uparrowTest-Unseen NE ↓\downarrowTest-Unseen SR ↑\uparrowTest-Unseen OSR ↑\uparrowTest-Unseen SPL ↑\uparrow
Seq2Seq--2052.924.32.62292.120.61.1
CMA--1615.428.14.82174.624.42.1
AerialVLN--1397.530.06.82147.328.14.4
NaVid✓\checkmark15313.038.211.621010.827.25.0
NaVila✓\checkmark13220.353.517.820214.742.19.6
OpenFly-Agent✓\checkmark9334.364.324.915422.656.219.1
Ours--9434.867.125.016420.756.817.2

We include the quantitative results on OpenFly in the updated manuscript (Appendix B.5, Table 11, L954-977).

[1] OpenFly: A Comprehensive Platform for Aerial Vision-Language Navigation. ICLR 2026.

AuthorsOpenReview ↗

Responses to Reviewer ndAo (3/4)

W3: "the generalization drop is much steeper than the strongest panoramic LLM agents. ... The paper frames this as a sensing-modality difference, but no analysis (e.g., per-scene breakdown, failure taxonomy) is offered to support that the gap comes from sensing rather than from the representation or BC training."

A3: Thanks for the comments. First, we acknowledge the larger seen-to-unseen SR drop (-7.4% SR) compared with the two panoramic LLM-based agents. We hypothesize that differences in training paradigm and pretraining resources contribute to this gap. Concretely, our agent is trained through behavior cloning on limited demonstration data, and environment-specific bias is a documented challenge for task-specific VLN policies. For example, HETT [1] reports a -7.68% SR drop on CityNav, while VLN$\circlearrowright$BERT [2] reports a -21.12% SR drop on REVERIE.

In contrast, LLM/MLLM-based agents may benefit from semantic and commonsense priors acquired through large-scale pretraining, which provides a strong inductive bias for unseen scenes. Relatively small SR drops have been reported by STMR [3] ( -1.8% SR drop on AerialVLN-S) and NavGPT-2 [4] (-1.92% SR drop on R2R).

Second, our discussion of panoramic observations (L361–363) was intended to contextualize the comparison under different observation settings, rather than establish sensing modality as the cause of the performance gap. We recognize that the wording could suggest a stronger interpretation and will clarify it.

In the revised version, we add explanation of performance gap on val unseen split (Sec. 4.2), and discuss integrating volumetric reasoning with LLM/MLLM-based agents as a future direction (Appendix C.1).

[1] History-Enhanced Two-Stage Transformer for Aerial Vision-and-Language Navigation. AAAI 2026.

[2] VLN$\circlearrowright$BERT: A Recurrent Vision-and-Language BERT for Navigation. CVPR 2021.

[3] Aerial Vision-and-Language Navigation via Semantic-Topo-Metric Representation Guided LLM Reasoning. arXiv 2024.

[4] NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models. ECCV, 2024.


Q1: "How does attention-based parent selection compare to random Top-K parent selection at identical budgets ?"

A1: Please see W1.


Q2: "What is navigation performance when the MSVM is replaced by ground-truth occupancy/semantics?"

A2: Thanks for the suggestion. We replace MSVM with ground-truth semantic occupancy as suggested. Specifically, the GT semantic occupancy labels are encoded into multi-scale voxel features with a hidden dimension of 512 to match the policy-network interface.

Results on val unseen are shown below. GT semantic occupancy yields lower NE (105.9 vs 109.4) and slightly higher OSR (27.2% vs. 26.5%). However, it does not translate into notable SR (8.8% vs. 8.8%) or SDTW (4.0% vs. 3.9%) gains. This indicates that the remaining errors are not mainly caused by the perception bottleneck. MSVM already supplies information sufficient for navigation, and further refining perception does not improve task success. Instead, the performance bottleneck lies on the reasoning side, i.e., whether the agent can select task-relevant regions and produce correct actions conditioned on the instruction and its state. This observation is also consistent with our finding in A1-2 that the gains of attention-based routing come from evidence-grounded region selection rather than from perception or compute.

PerceptionNE ↓\downarrowSR ↑\uparrowOSR ↑\uparrowSDTW ↑\uparrow
GT semantic occupancy105.98.827.24.0
Multi-Scale Voxel Map (Ours)109.48.826.53.9

Q3: Is the seen-split gain over LAG 8.1% or 9.0% SR?

A3: Sorry for the confusion, this is a typo, the actual performance gain over LAG on val seen split is 9.0% in SR. We have corrected this value in introduction in the updated manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Our Responses & Paper Revisions

We sincerely thank the Area Chair and all reviewers for their time and constructive feedback. The reviews helped us clarify the experimental comparisons, strengthen the evaluation, and better communicate the scope and limitations of our approach.

We have made our best effort to address the concerns and revise the paper accordingly. The major modifications are summarized as follows:

Clarifications on MSVM and Hierarchical Volumetric Reasoning

  1. Clarified the architectures of each ablation model variant (Appendix A.2).
  2. Added a 2D baseline with geometry and semantic pretraining to set a matched comparison (Sec. 4.4, Table 3).
  3. Revised Table 4 and its analysis to more clearly position its role (Sec. 4.4).
  4. Refined the description of coarse spatial reasoning to emphasize its scope within the currently observed volume (Sec. 3.2).

Additional Experiments and Analyses

  1. Added comparisons against random, uniform, and occupancy-based routing. (Sec. 4.4, Table 5).
  2. Added two recent learning-based baselines for comparison (Secs. 4.1–4.2, Table 1).
  3. Compared routing strategies across voxel budgets to examine whether budget reduction alone explains the improvement (Appendix B.3, Table 9).
  4. Provided quantitative and qualitative analysis of spatial routing (Appendix B.4, Table 10 and Fig. 4).
  5. Added quantitative results on OpenFly benchmark. (Appendix B.5, Table 11).
  6. Reported standard deviations for the main-text ablation studies.

Expanded Discussions

  1. Added discussion of the unseen performance gap and differences in model capabilities and pretraining resources (Sec. 4.2).
  2. Clarified the temporal memory, and discussed persistent spatial memory as future work (Sec. 3.3; Appendix C.1).
  3. Expanded discussion of integrating volumetric reasoning with LLM/MLLM-based agents to improve generalization (Appendix C.1).

Major manuscript revisions are highlighted in red. Detailed responses and supporting results are provided in the reviewer-specific responses. We welcome any further discussions and will address any remaining concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.