acceptodds
Under review as a conference paper at ICLR 2027

Continuous Semantic Navigation Fields for Language-Conditioned Aerial Exploration

Abstract

Language-conditioned aerial navigation is commonly formulated as next-action prediction over a sparse set of discrete waypoints. This representation discards the feasible intermediate states that determine collision risk, dynamic smoothness, and semantic progress. We introduce CSNF, a Continuous Semantic Navigation Field that treats navigation as conditional probability-flow generation over a continuous trajectory manifold. A multimodal encoder maps language, RGB-D/LiDAR observations, and vehicle state to a semantic-geometric field; a flow-matching transport model then evolves a noise trajectory into a dynamically feasible trajectory while differentiably enforcing occupancy, instruction alignment, and uncertainty constraints. We provide a consistency objective that couples intermediate states to future observations and prove that, under a local regularity assumption, field-based transport has lower discretization bias than vertex-only action policies for the same number of policy evaluations. We evaluate on VLN-CE, Habitat-Matterport3D, and AirSim-style UAV scenes. Extensive real-world UAV flight experiments further validate its robustness, generalization, and practical deployability.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-and-Language Navigation, Aerial Robotics, Embodied AI, Vision-Language-Action Models, Flow Matching, Safe Robot Control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BywFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CSNF, a language-conditioned UAV navigation policy combining an interpolated semantic-geometric field, conditional flow matching over trajectory sequences, auxiliary objectives, and an inference-time safety projection. It reports simulator comparisons, component ablations, cross-environment transfer, and qualitative demonstrations on a physical UAV. The theoretical analysis covers local quadrature error and numerical integration error.

Critical strengths
  1. Relevant problem and coherent system design. Combining language grounding, local geometry, trajectory generation, and safety-filtered execution addresses practical requirements of aerial navigation. The explicit nonlinear post-check and fallback behavior are useful design choices.

  2. Promising reported performance. CSNF achieves the highest SR/SPL and lowest collision rate among the listed baselines. The ablations and physical-platform examples provide preliminary support for the approach, although the evidence has important limitations below.

Critical weaknesses
  1. The experimental evidence is insufficient to establish the claimed improvements. Table 1 improves over 3D Diffuser Actor by only 1.3 percentage points in SR and 0.4 points in SPL, but episode counts, seed variability, and confidence intervals are absent. Appendix D prescribes uncertainty reporting without supplying it. Exact dataset splits, baseline adaptation details, and timing hardware are also insufficiently specified. The physical evaluation provides four qualitative task examples without aggregate success, collision, or intervention statistics, falling short of the abstract’s claim of extensive real-world validation.

  2. The central learning mechanism and theoretical justification remain incomplete. Eqs. (6)–(9) do not clearly identify the trajectory on which auxiliary losses are evaluated or how these losses update the generator. Appendix A.2 describes queries at the sampled interpolation XτX_\tau; if the dynamics loss is evaluated on that sample, it has no gradient with respect to the generator parameters. Furthermore, Proposition 1 establishes a standard quadrature improvement using additional interior evaluations, not superiority at an equal policy-evaluation budget as claimed in the abstract. These gaps undermine the attribution of gains to the proposed continuous-field formulation.

Questions for authors
  1. Training clarification: Which trajectory enters each auxiliary loss: XτX_\tau, a predicted endpoint, or an integrated generated trajectory? Please specify the gradient paths, the consistency weight used in Table 2, and how Eq. (8), whose weight decreases with uncertainty, produces the intended uncertainty-aware behavior.

  2. Experimental clarification: Please provide evaluation counts, exact splits, confidence intervals, baseline adaptation details, and latency hardware. Do the reported SR/SPL gains remain reliable across runs? What are the aggregate outcomes and intervention counts for the physical trials?

  3. Contribution clarification: What supports the equal-budget claim beyond composite quadrature? Do the gains persist against a trajectory generator using the same intermediate-state checks and safety projection?

1 comment

Thank you for clarifying how the auxiliary losses update the generator. The matched safety-filter comparisons and the results from 48 indoor flight trials address my main concerns. I will raise my rating to 3.

Reviewer_4afzOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CSNF (Continuous Semantic Navigation Field), which formulates language-conditioned UAV navigation as conditional flow-matching generation over continuous trajectories rather than as discrete waypoint or action prediction.

Critical strengths
  1. The motivation of modeling navigation as continuous trajectory generation rather than discrete waypoint or action selection is reasonable for UAVs, since intermediate states matter for collision avoidance and dynamic smoothness.

  2. The paper includes physical UAV flight demonstrations on a customized platform, which shows that the pipeline can run in closed loop on real hardware to some extent.

Critical weaknesses
  1. The experimental setup is seriously unclear. The "Habitat → Habitat-unseen" row in Table 3 (76.5/61.8/69.8, 51 ms) is identical to the CSNF results in Table 1. This suggests that the so-called UAV-VLN main results are in fact obtained on Habitat, which contradicts the paper's positioning as an aerial navigation method.

  2. More details are needed on how the baselines, such as RT-2, were retrained under the proposed protocol.

  3. I would encourage the authors to provide more quantitative metrics for the real-world experiments to support their claims. The current real-world experiments are mostly qualitative, so describing them as "extensive" experiments that "validate robustness, generalization, and practical deployability" is somewhat overclaimed.

  4. The theoretical section feels somewhat disconnected from the rest of the paper and does not closely correspond to the motivation.

Questions for authors

Please see the weaknesses above.

Reviewer_E6JWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates language-conditioned UAV navigation as generating an entire continuous trajectory rather than selecting the next discrete action. It fuses language, RGB-D/LiDAR, and the agent state with cross-attention, and builds a semantic-geometric field that stores occupancy, semantic features, and variance per voxel and is read at arbitrary positions by interpolation. A 32-state (3.2 s) trajectory is generated with conditional flow matching, with auxiliary losses for semantics, collision, dynamics, uncertainty, and consistency. At inference, only violating states are projected with a linearized QP, with a fallback to hover if the projection fails. The paper reports first place among 15 baselines with SR 76.5, SPL 61.8, and CR 5.6, along with ablations, cross-environment transfer, and qualitative results of real flights.

Critical strengths
  • Formulating language-conditioned aerial navigation as continuous trajectory generation instead of waypoint prediction, and querying occupancy and instruction relevance directly from the semantic-geometric field at each intermediate state, is a clean design.

  • The method includes a safety mechanism (QP projection at inference with a hover fallback on failure), and the authors explicitly narrow the scope of their theoretical guarantee.

  • Fifteen baselines from four families are rerun under the same protocol, and metrics on which the method does not win (NE, SC, latency) are reported as well.

Critical weaknesses
  • The paper claims UAV navigation, but I am not sure the main results come from a UAV environment. The values in Table 1 ("UAV-VLN protocol", 76.5/61.8/69.8/51) are identical to the Habitat → Habitat-unseen row of Table 3, and training uses ground-level indoor data such as VLN-CE and HM3D (p. 6). VLN-CE trajectories lack UAV dynamics such as altitude, attitude, and angular rate, so I would like to know how the 9-dimensional UAV state and L_dyn were learned. The AirSim results consist of only two cross-embodiment transfer rows without baseline comparisons, and the source and size of that data are not given. Comparisons on public UAV-VLN benchmarks such as AerialVLN (ICCV 2023), OpenUAV (ICLR 2025), and OpenFly (ICLR 2026) appear necessary. The real-world evidence is also limited to qualitative results on four tasks in an indoor Vicon arena.

  • Navigation policies that generate continuous trajectories with flow matching already exist (FlowNav, IROS 2025; NaviBridger, CVPR 2025), and π0 is a language-conditioned flow-based VLA. Applying flow matching with dynamics supervision to UAV VLN closely follows DynFly, which is included as a baseline. The remaining differences are a few auxiliary losses and the inference-time QP, and I am not convinced that they constitute a distinct contribution. Moreover, the paper frames prior methods as "sparse discrete waypoint prediction," but the strong baselines in Table 1 (Diffusion Policy, 3D Diffuser Actor, DynFly) already output dense trajectories over the same horizon, so the "discrete vs. continuous" axis does not separate CSNF from these baselines.

  • CSNF takes LiDAR and RGB-D voxel inputs, whereas the VLN baselines are said to keep their original encoders and likely lack 3D or LiDAR input. It is unclear to me whether the state-of-the-art result comes from flow matching or from the added LiDAR and 3D input. The advantage in collision rate may also come from the inference-time QP and the hover fallback, but there is no ablation without the projection, and it is not stated whether the baselines were given the same projection. A comparison with Diffusion Policy or 3D Diffuser Actor using the same encoder and the same QP would help.

  • The numbers differ substantially from previously reported results. Seq2Seq reaches SR 61.0 here, whereas the original VLN-CE paper reports around 20% on val-unseen, and DynFly reports 27.6 on OpenUAV in its own paper versus 74.4 here. FlowPilot, which takes no language input, reaching SR 73.6 also suggests that the task may not require much instruction grounding. I would like to know how the episodes were constructed (source of instructions, subgoal annotation, obstacle density).

  • On real UAVs, a common design is modular: a VLM identifies the goal and a local planner such as EGO-Planner (RA-L 2021) generates the trajectory. Its cost terms (smoothness, collision, feasibility, progress) closely match L_dyn + L_col + L_sem, and planning takes about 1 ms. A comparison of accuracy and cost with such a modular alternative would be helpful. CSNF uses LiDAR, RGB-D, a voxel map, 20 ODE steps, and a QP, yet the hardware on which the 51 ms latency was measured is not stated, and the parameter count is left blank.

Questions for authors

Please see the questions raised in the Weaknesses. Answers to Weaknesses 1, 3, and 4 (clarification) would most likely change my assessment; Weaknesses 2 and 5 are suggestions that would strengthen the paper but are not required for the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.