acceptodds
Withdrawn from ICLR 2027

Think Like a Pilot: Fine-Grained Long-Horizon UAV Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language-guided UAV agents must execute long-horizon semantic instructions while producing smooth, physically feasible continuous flight commands, yet existing Vision-Language Navigation (VLN) benchmarks typically use discrete or coarse actions and existing UAV Vision-Language-Action (VLA) tasks focus on short, atomic maneuvers. To address this gap in UAV task settings, we introduce , a ine-grained ong-horizon nstruction-uided benchmark for ybrid UAV navigation and reasoning asks, which combines multi-stage instructions with dense 6-DoF trajectory annotations across two dataset splits: Fine-grained VLN and Long-horizon Flow. To endow the UAV agent with the capability of real-time in-flight reasoning over task execution status and mission planning, while simultaneously accommodating high-frequency, real-time precise control, we further propose , an asynchronous architecture that decouples a low-frequency Streaming Pilot Vision-Language Model (VLM) for task-state reasoning from a high-frequency diffusion action model for continuous control, supervised by explicit texts that summarize the current flight state and anticipate the next subgoal. In closed-loop evaluation, FLIGHT VLA consistently surpasses representative VLN and VLA baselines on our FLIGHT benchmarks, achieving stronger multi-stage completion, subgoal adherence, and terminal control. Its trained Streaming Pilot Reasoning VLM further improves UAV video reasoning, validating the effectiveness of our design.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vison Language Action, Vison Language Navigation, UAV

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AaLLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FLIGHT, a benchmark for instruction-guided UAV navigation and reasoning tasks, combining multi-stage textual instructions with dense 6-DoF trajectory annotations across two splits: Fine-grained VLN and Long-horizon Flow. To address the temporal mismatch between high-frequency control and low-frequency semantic understanding, the authors propose FLIGHT VLA, an asynchronous architecture that decouples a streaming VLM for task-state reasoning from a high-frequency diffusion action model for continuous control, supervised by explicit "Pilot Reasoning" texts. While the paper tackles an important and challenging problem in embodied aerial agents, it suffers from significant limitations regarding real-world validation and the clarification of its technical novelty over existing video-based VLN/VLA frameworks with chain-of-thought reasoning.

Critical strengths
  1. Comprehensive Benchmark Design: The FLIGHT benchmark addresses gaps in prior datasets by scaling up to multi-stage compositions, dense local referential constraints, and continuous action supervision rather than sparse waypoints or discrete choices.
  2. Asynchronous Architecture Design: The proposed slow-fast asynchronous architecture (decoupling the streaming Pilot VLM from the high-frequency Diffusion Action Model) is logically sound and practically well-engineered to handle inference latency during deployment.
Critical weaknesses

[! I personally think the authors should submit to the Primary Area: datasets and benchmarks rather than Primary Area: applications to robotics, autonomy, planning. The review is based on the principle in applications to robotics, autonomy, planning]

  1. Absence of Real-World Physical Experiments Given that the paper emphasizes real-world deployment challenges—such as latency-aware alignment, continuous control feasibility, and sensor-based telemetry—the complete lack of real-world physical flight experiments severely undermines the claims of practical applicability. To truly substantiate the paper's claims on long-horizon capabilities, the authors must provide real-world hardware experiments demonstrating stable, continuous autonomous flight exceeding 1 minute in duration under complex, multi-stage semantic instructions. Simulation-only results are insufficient to prove that the proposed diffusion action model and streaming VLM are robust against real-world aerodynamic disturbances, visual domain gaps, and sensor noise.

  2. Unclear Novelty and Outdated Baseline Selection The paper emphasizes "Pilot Reasoning" (explicitly generating text on current state and next-step planning) to bridge high-level reasoning and low-level control. However, Chain-of-Thought (CoT) reasoning and intermediate task decomposition have already been extensively explored in recent VLN and VLA literature (e.g.,ABot-N0). I think the contribution in method is merely a straightforward domain adaptation of streaming VLM and diffusion action models to UAVs. The chosen baselines (such as CMA+LAG and NaVid) are relatively outdated given the rapid progress in VLN. Recent open-source SOTA frameworks and benchmarks—such as LightNav, NaviDA—should be included as baselines. Comparing only against older models makes it difficult to ascertain whether FLIGHT VLA's performance gains stem from genuine architectural superiority or simply from using a stronger base model and task design.

While the FLIGHT benchmark could potentially serve as a valuable resource for the embodied aerial navigation community, there remains a substantial gap on the methodological side. The paper lacks sufficient novelty over existing CoT/VLN paradigms, relies on outdated baseline comparisons, and lacks real-world physical flight validation to back its claims on long-horizon continuous control.

Questions for authors

See Weakness

2 comments

Thank you for recognizing the contributions of our dataset. Our work is the first to couple long-horizon semantic navigation with fine-grained UAV control, helping unlock the potential of embodied intelligence models, such as vision-language-action (VLA) models, for UAV navigation. We respectfully encourage the reviewer to further consider the potential value of this dataset and benchmark and their contributions to the community.

Although we do not claim to be the first to apply chain-of-thought (CoT) reasoning to navigation, we are the first to systematically incorporate Pilot Reasoning, a navigation CoT modality, into a UAV dataset, together with a dedicated CoT design and annotation pipeline. To the best of our knowledge, existing UAV datasets do not provide a text modality covering fine-grained flight descriptions and task planning during flight. Our method demonstrates the effectiveness of Pilot Reasoning text for task supervision and historical memory management, providing a foundation for future work to incorporate CoT text into training and inference. We hope the reviewer will recognize this contribution to dataset design. Furthermore, our framework couples long-horizon, multi-stage semantic understanding with real-time UAV control, enabling end-to-end semantic understanding and fine-grained control, including obstacle avoidance. To the best of our knowledge, no prior work has achieved this coupling end to end.

Our architecture also differs substantially from ABot-N0. Approaches such as ABot-N0 fully separate navigation CoT reasoning from action generation through explicit if–else branching, effectively using text to pretrain the “Brain” module without supporting in-flight reasoning and memory management. In our architecture, textual reasoning and action generation proceed concurrently during inference. We further integrate latency-aware training with asynchronous inference to fully exploit explicit textual analysis and historical information management during flight, more closely aligning with the purpose of CoT reasoning.

Compared with conventional VLN methods, our design also incorporates several considerations for real-world deployment. Practical constraints prevented us from conducting real-world flight tests in urban environments. Nevertheless, our simulation environment supports continuous control and advances continuously, streaming image frames as the UAV moves. Compared with the stepwise decision-making of conventional baselines, concurrent inference and UAV operation offer an advantage for deployment. The following table provides additional measurements of inference latency.

Latency was measured on an RTX 4090:

Request typeMeanMedianP90
VLM + action-head request1,881 ms1,875 ms2,085 ms
Action-only request124 ms112 ms143 ms

Thanks for the detailed response. However, after carefully reading the rebuttal, my primary concerns regarding the paper's suitability for the Robotics, Autonomy, and Planning track, its lack of real-world experimental validation, and its methodological novelty and missing sota baselines remain unaddressed. I decide to keep my score.

Reviewer_hnYNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new benchmark for long-horizon UAV navigation with much more fine-grained annotation on the flight instructions. Besides, a new method named FLIGHT VLA is proposed to enabling long-context understanding with a fast-slow hybrid design.

Critical strengths
  1. A new dataset with fine-grained annotation and long-horizon design.
  2. A framework dedicated for the above task.
Critical weaknesses
  1. The motivation of the proposed benchmark is not very clear. A comparison with existing works that focus on UAV navigation should be conducted in detail.
  2. More baseline models should be added to the proposed benchmark to provide a thorough report on the capabilities of existing methods. Currently, on 2+2 methods are compared for long-horizon and fine-grained tasks.
  3. The novelty of the design of the proposed flight VLA should be strengthened. The fast-slow design is a very common design for navigation tasks; thus, it can hardly be regarded as a valuable contribution.
  4. Ablation studies are missing for the proposed method.
Questions for authors
  1. How well does the flight vla transfer to the real world?
Reviewer_QwujOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces the FLIGHT benchmark and FLIGHT VLA for multi-stage language-guided UAV navigation with continuous control. The method combines a low-frequency semantic reasoning module with a high-frequency diffusion action head and uses historical feature offsets during training to account for inference latency. The full system substantially outperforms the compared baselines on Long-horizon Flow and achieves competitive results on Fine-grained VLN.

Critical strengths

S1. The task setting has clear research value. FLIGHT combines multi-stage language instructions with continuous flight control and provides fine-grained language and action supervision (Table 3). This makes the connection between semantic decisions and low-level control directly testable.

S2. The system-level gains on Long-horizon Flow are substantial. FLIGHT VLA achieves 59.00%/78.97% SR/SSR, compared with 18.50%/38.69% for Memory VLA (Table 1). The size of this improvement supports further investigation of the proposed system design.

Critical weaknesses

W1. The experiments do not establish which design choices drive the gains. The paper emphasizes Pilot Reasoning, reasoning memory, and latency-aware training, but neither the main paper nor the appendix isolates these components. The Table 1 baselines also differ in backbone, action representation, and supervision, while Sec. 5.1 leaves the comparability of training data and budgets unclear. The results therefore support the full system’s effectiveness but do not establish the contribution of the highlighted mechanisms. The most useful addition would be targeted component controls with the same backbone, data, and budget, particularly a comparison with and without latency-aware training under matched inference delays. Baseline training conditions also need to be specified.

W2. Missing evaluation details limit the interpretability of the benchmark results. The paper does not specify test episode counts, the separation of training and test scenes, or subtask-completion rules for SSR/IASR. Annotation models and human-review coverage are also unclear. On Fine-grained VLN, the SR advantage over NaVid is only 0.5 percentage points (Table 1); without sample counts and uncertainty estimates, its stability is difficult to assess. Complete splits, decision rules, and annotation-quality checks are necessary to evaluate the benchmark, and close results should be described with appropriate restraint.

Questions for authors
  1. Could you provide the test splits, episode counts, SSR/IASR rules, annotation models, and human-review procedure? Does Table 1 report a single evaluation or an average across runs?
  2. What training data, model sizes, and budgets were used for the baselines? Are results available under comparable conditions without Pilot Reasoning, reasoning memory, or latency-aware training to help identify component contributions?
  3. Does the simulator continue advancing during VLM inference? Please report measured VLM latency, action frequency, and hardware, and clarify whether comparisons with and without latency-aware training use the same delay.
4 comments
AuthorsOpenReview ↗

**1. Evaluation protocol, SSR/IASR criteria, and annotation quality (W2, Q1).**

We thank the reviewer for identifying the need for component ablations and clearer evaluation details. Below, we clarify the evaluation protocol, baseline adaptations, ablation results, and inference settings.

Our evaluation includes 200 Long-horizon Flow (LHF) tasks across 7 scenes and 200 Fine-grained VLN tasks across 11 scenes. Each task is evaluated twice, yielding 400 trials per task family; success rates are computed over the pooled trials. For FLIGHT VLA, the two rollouts use independently sampled initial DiT noise. These repetitions assess rollout variability rather than variation across independently trained models. Training uses the FLIGHT benchmark training set, with evaluation tasks and their instructions held out. This task/instruction separation alone does not establish that training and evaluation scenes are disjoint.

For LHF, SSR measures the fraction of subtasks correctly completed within a trajectory. A subtask passes only when the specified action, target instance, direction or relative position, magnitude or repetition count, and applicable ordering constraints are satisfied. For example, partially circling an object does not satisfy an instruction to circle it clockwise three times. Correctly completed subtasks receive SSR credit even if the overall task fails. LHF SR requires reaching the destination and completing all specified subtasks.

For Fine-grained VLN, SR requires the final position to be within 15 m of the goal; OSR checks whether the UAV enters this region at any point during the rollout. IASR additionally requires compliance with every mandatory instruction constraint, including landmark identity, turn location and direction, action order, and any specified landing requirement. A violation of any mandatory constraint makes the trial unsuccessful under IASR. Its denominator is all evaluation trials, rather than only SR-successful trials. Thus, reaching the goal after turning at the wrong landmark may satisfy SR but fails IASR.

Annotation follows a staged process with human review of subtask descriptions and refinement of temporal boundaries. Reviewers check whether described actions, such as takeoff, landing, and turning, match the observed motion; whether referenced landmarks are clearly visible; and whether the annotated intervals cover the corresponding maneuver.

AuthorsOpenReview ↗

**2. Baseline training, UAV data adaptation, and component contributions (W1, Q2).**

We clarify how UAV demonstrations are converted into the inputs and supervision required by each baseline. The methods retain different backbones, action representations, and supervision objectives.

MethodBackbone and adaptationRecorded training configuration
FLIGHT VLAQwen2.5-VL-3B-Instruct with LoRA; DiT-B action headGlobal batch 128; LHF: learning rate 5×10⁻⁵, 8×A800-80GB; VLN: learning rate 1×10⁻⁴, 8×A100-80GB
NaVidVicuna-7B-v1.5 and EVA-ViT-G; LoRA rank 64; UAV action-text supervisionGlobal batch 256; learning rate 2×10⁻⁵; 8×A100-80GB
Memory VLAOpenVLA/Llama-2-7B with DINOv2 and SigLIP; DiT-L head adapted to four-dimensional UAV actionsGlobal batch 256; learning rate 2×10⁻⁵; 8×A100-80GB
OpenVLA-UAVOpenVLA, approximately 7.5B parameters; LoRA rank 32; tokenized six-degree-of-freedom UAV pose predictionGlobal batch 32; learning rate 5×10⁻⁴; 8×A100
CMA+LAGCMA architecture: bidirectional LSTM instruction encoder, frozen ResNet50 visual encoder, two GRUs, cross-modal attention, and a nine-class action headAdam, Global batch 16; learning rate 2.5×10⁻⁴,

NaVid. Each UAV trajectory from the FLIGHT benchmark is converted into per-step prefix samples containing historical observations, the current FPV image, and the task instruction, with the next action as the target text. The model uses at most the 32 most recent history frames. We retain NaVid's visual compression: four visual tokens per history frame and 64 for the current frame. The prompt specifies a UAV navigation assistant and the available flight actions, and the target follows The next action is [action] [value] [unit]. Labels include ascent/descent by 40 cm, clockwise/counterclockwise rotation by 15°, forward motion by 60/120/180 cm, and stop; up to three consecutive forward steps are merged. The visual encoder and multimodal projector are frozen, and only LoRA parameters are trained. Turning and stopping samples are oversampled by a factor of two to mitigate action imbalance.

Memory VLA. We initialize from pretrained OpenVLA-7B weights and retain the DINOv2/SigLIP visual encoders and Llama-2-7B backbone. UAV demonstrations are loaded through a streaming JSONL dataloader with an associated action-statistics file. The DiT-L head uses four-dimensional actions and a future-action-window setting of three; it learns expert action sequences through diffusion denoising. Training uses image augmentation and a shuffle buffer of 128k. Action chunks containing the zero action [0,0,0,0] are oversampled by a factor of four to increase the weight of stationary-control examples.

OpenVLA-UAV. We retain single-frame visual input and combine the current UAV position and orientation with the instruction in the prompt: In: Current State: {state}, What action should the uav take to {instruction}? The model retains OpenVLA's discretization scheme, encoding action values into 256 bins to predict a six-degree-of-freedom UAV pose (x, y, z, roll, pitch, yaw). Fine-tuning uses action-token prediction as supervision.

CMA. Expert actions, positions, and orientations from flight logs are aligned with FLIGHT task instructions and scene information, then converted into AirVLN-format episodes containing the instruction, scene and episode identifiers, reference path, and action sequence. Instructions are tokenized using the dataset vocabulary, with a maximum length of 500 tokens. FPV images are converted to RGB and resized to 224×224. Since reliable depth is unavailable, the depth branch is ablated by setting its features to zero. The offline pipeline replays expert trajectories with teacher forcing, caches frozen visual features in LMDB, and trains the recurrent policy by behavior cloning. CMA combines instruction features, visual observations, and the previous action through its recurrent and cross-modal attention modules. Under the documented discrete training pipeline, supervision is weighted cross-entropy over nine actions: stop, forward/backward motion, left/right turns, ascent/descent, and left/right translation. Steps at which the expert action changes receive weight 1.9; other steps receive weight 1.

AuthorsOpenReview ↗

3. Ablation Study

Component ablations. The available Fine-grained VLN results are:

ConfigurationSR (%)OSR (%)NE (m)
Full FLIGHT VLA13.537.045.52
Action-only supervision6.07.554.79
Without latency-aware training6.517.051.08

The action-only variant removes Pilot Reasoning text supervision while retaining action supervision. The variant without latency-aware training conditions the action head on VLM features associated with the latest video frame, without the training-time temporal offsets used by the full model. These results support the benefit of the two training mechanisms in the tested system.

AuthorsOpenReview ↗

Simulator advancement, measured latency, and control timing (Q3)

For FLIGHT VLA, the simulator continues advancing during VLM inference. The action model continues operating with the most recently available semantic features while the VLM computes its next update. The evaluation records describe continuous advancement for the two VLA baselines as well. Discrete VLN baselines pause UAV motion between action steps while waiting for the next decision. We disclose this execution difference because the system-level comparison does not use identical timing conditions across all methods.

Latency was measured on an RTX 4090:

Request typeMeanMedianP90
VLM + action-head request1,881 ms1,875 ms2,085 ms
Action-only request124 ms112 ms143 ms

The mean times of the profiled VLM stages sum to approximately 1,765 ms, while the separately profiled action pipeline takes approximately 112 ms. VLM inference latency is affected by context length, fluctuations in hardware performance, and communication delays, and therefore is not a fixed value. This variability motivates adaptive training across different latency conditions, enabling the policy to accommodate changing inference delays during deployment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.