acceptodds
Under review as a conference paper at ICLR 2027

AheadFly: Think and Act Ahead for Multi-Stage Aerial Vision-and-Language Navigation

Abstract

Aerial vision-and-language navigation (AVLN) requires unmanned aerial vehicles (UAVs) to follow natural-language instructions in open 3D environments. Multi-stage navigation requires sequentially grounding multiple landmarks. Existing zero-shot approaches treat each stage as an independent grounding problem and primarily address what to look at in each stage. However, we argue that the dynamic nature of UAVs introduces additional challenges: aerial navigation must also resolve "where to look" and "where to look from" while maintaining grounding continuity across stages. In this paper, we propose AheadFly, a training-free modular framework that thinks and acts ahead for multi-stage aerial navigation. First, it "thinks ahead" by parsing instructions into explicit landmark queries and inter-stage transition directions. It then "acts ahead" by jointly adjusting the UAV's orientation and flight path under the guidance of these directions. Specifically, a pre-grounding view adjustment decides where to look by reorienting the UAV so the current landmark enters its field of view, and a lookahead path adjustment decides where to look from through multiple guiding forces, to mitigate cross-stage landmark occlusion and avoid obstacles along the path. To evaluate intermediate grounding and continuity, we introduce LandmarkFly, a multi-stage, long-range AVLN benchmark generated through a landmark-centric pipeline with explicit instruction-trajectory-landmark alignment, together with stage-level evaluation metrics. Experiments on both OpenFly-AirSim and LandmarkFly show consistent gains over the strongest baselines, improving Success Rate by 5.5 and 32.4 percentage points on the two benchmarks, respectively, and Grounding Point Accuracy by 7.6 percentage points on LandmarkFly.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Aerial vision-and-language navigation, Zero-shot navigation, AVLN Benchmark, Multi-stage navigation, Stage-level evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nWnnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

In this paper, the authors introduce AheadFly, a method for multi-stage aerial vision-and-language navigation, along with LandmarkFly, a new benchmark for evaluating intermediate landmark grounding during navigation. The core idea is to think and act ahead by parsing instructions into explicit landmark queries and transition directions. This adjusts the UAV view and flight path in advance. The benchmark contains 1,000 multi-stage trajectories with explicit alignment among instructions, trajectories, and landmarks. In the experiments, AheadFly achieves the best performance among the evaluated baselines on OpenFly and LandmarkFly.

Critical strengths

The proposed AheadFly framework presents an intuitive idea for multi-stage aerial navigation. In particular, the novel idea of adjusting the UAV view and flight path in advance is interesting. This can help the UAV more reliably find the next landmark.

Critical weaknesses

1. Degraded performance of the Fly0 baseline.

This paper reports baseline performance using Fly0. However, its performance is significantly degraded compared to the original paper. Fly0 reports 64.67% SR on OpenFly in its original paper, whereas it achieves only 23.22% SR in this paper, while AheadFly achieves 30.17%. It is unclear whether the reported improvement reflects a genuine advantage over Fly0 or a weak reproduction of the baseline.

2. Lack of comparison under the original Fly0 setting.

The authors modify the Fly0 setup by replacing its grounding model and adding Qwen3-Max-based TIS parsing rather than evaluating the original system under its original experimental setting. It is also important to report performance on the AerialVLN benchmark.

3. The effectiveness of AheadFly on stronger baselines remains unclear.

There are several existing AVLN methods with significantly higher SR on OpenFly, including SPF (42.94%), OpenFly (33.61%), and UAV-Flow (32.14%), compared with 30.17% for AheadFly in this paper. It would be important to verify whether the proposed ideas still provide consistent gains when applied to these stronger methods.

4. Potential benchmark bias.

LandmarkFly appears to be closely aligned with the design assumptions of AheadFly, since both are structured around explicit landmark stages and discrete transition directions. Moreover, the reported improvement is much larger on the authors' own LandmarkFly benchmark than on the public OpenFly. This raises concern that the proposed benchmark may disproportionately favor the proposed method.

Questions for authors
  1. Why does Fly0 drop from 64.67% SR in its original OpenFly evaluation to only 23.22% in this paper?
  2. Does AheadFly still provide consistent improvements when its proposed mechanisms are applied to stronger baselines?
5 comments
AuthorsOpenReview ↗

Response to Reviewer nWnn (1)

We thank the reviewer for the careful assessment and address all concerns below.

Metric groups used in the tables: Standard: SR, OSR, NE, SPL. Soft: SSR, SOSR. Stage-level: LVR, GPA, SRS, SRC.

W1 & Q1 — Degraded performance of the Fly0 baseline

The difference between Fly0's performance in our experiments and the performance reported in its original paper stems mainly from the different type and scale of the VLM backbone used, and from the different data subset used for evaluation. The original Fly0 paper uses Qwen2.5-VL-32B as the landmark grounding backbone and may have used the full OpenFly dataset (the paper does not state this). In our experimental setup we use LocateAnything-3B as the VLM backbone and use only the AirSim subset of OpenFly (1,210 samples in total).

Concretely, neither the original Fly0 paper nor its open-source implementation states how its subset of OpenFly was selected, so one must assume it used the full dataset. When running the experiments we found that several of the scenes provided by OpenFly are in fact unusable (most notably the 3DGS-based dataset that has not been released). Given the constraints of our hardware, we finally conducted the main experiments only on its six AirSim subsets. Even so, the 1,210 instruction–trajectory pairs across the six scenes still constitute a substantial scale.

Since the subset selection in the experimental setup may introduce bias, directly taking the numbers claimed in the Fly0 paper as its performance under our configuration would be unfair. On the other hand, our AheadFly uses the 3B-parameter LocateAnything as the VLM backbone for landmark grounding; unless a unified grounding model is used, the evaluation could be affected by the VLM's own performance and thus become unfair.

For these two reasons, we had to align both the dataset selection and the configuration of the landmark-grounding VLM across baselines before running the experiments. Together with the fluctuation that incidental factors may introduce, the resulting performance naturally differs from the numbers reported in the original papers. That said, given that OpenFly-Agent still comes close to its claimed performance under the same configuration (except that its end-to-end 7B backbone cannot be replaced) — 22.64% versus the 27.78% aggregate over four scenes claimed in its repository — we consider the actual performance of each baseline under our unified experimental setup to be objective and reasonable.

W2 — Lack of comparison under the original Fly0 setting

Given what Fly0 has released so far, a comparison under its original setting is difficult to carry out. Considering both the fairness of the evaluation and our emphasis on the gain attributable to the mechanisms in the architecture, we consider the comparison and analysis under the present experimental configuration to be reasonable and valid. As for the AerialVLN dataset, given that OpenFly already supersedes it in several respects (also a simulation scene built on UE4 + AirSim, a larger dataset scale, a more sensible construction method, and higher instruction–trajectory quality), and given the limited time available in the rebuttal period, we have not scheduled comparative experiments on that dataset.

Two changes were made to Fly0's original architecture in the main comparison of this paper. First, to avoid an unfair evaluation dominated by the strength of the VLM itself, we replaced its landmark grounding backbone (Qwen2.5-VL-32B in the original Fly0 paper). As for the design and choice of the task-parsing module: on the one hand, it keeps the parsing backbone identical to AheadFly's so that only one variable changes; on the other hand, Fly0 has not open-sourced the scripts related to its experimental setup and instruction parsing, so our choice of TIS is a reasonable inference from the input types its released implementation actually supports — it accepts only a single-sentence language instruction, supplied interactively. Restoring Fly0's architecture exactly to its current open-source implementation would therefore make it impossible to use the present datasets as a testbed for comparison.

In summary, our adjustments to Fly0's architecture are to some extent constrained by its actual implementation. Considering both the fairness of the baseline comparison and our emphasis on evaluating the mechanisms within the architecture rather than the strength of the module backbones, we consider the present experimental setup reasonable.

AuthorsOpenReview ↗

Response to Reviewer nWnn (2)

W3 & Q2 — Effectiveness of AheadFly on stronger baselines

Because the experimental configurations differ, these numbers are not suitable for direct comparison. Under our unified choice of VLM backbone and data subset, replacing the task-parsing module of the strongest existing zero-shot AVLN baseline, Fly0, with our TPG and adding the pre-grounding view adjustment improves its performance on both OpenFly and LandmarkFly to a certain degree, demonstrating the benefit of our ideas to methods of the same family.

First, we consider a direct horizontal comparison of results across different experimental configurations to be insufficient: the numbers are not commensurable. Those numbers come from the Fly0 paper and may involve OpenFly scenes and data that we do not cover, or other experimental settings that differ from ours. Our present setup follows the released code of SPF and Fly0 as closely as possible, and we applied to them the minimal modification needed to adapt to the OpenFly and LandmarkFly datasets, strictly in line with the relevant discussion in their papers. Under the present configuration, we therefore consider that the differences in actual performance among the methods revealed by our experiments still corroborate the effectiveness of AheadFly itself.

Second, fully incorporating our ideas into modular zero-shot methods of the same family (with instruction parsing, landmark grounding and motion control modules) would leave them differing from AheadFly essentially only in the VLM backbone. We therefore applied only two of our ideas — the task-parsing module and the pre-grounding view adjustment — to the best-performing baseline, Fly0, obtaining the new baseline Fly0+TPG, and observed its performance on OpenFly-AirSim and LandmarkFly, in order to analyse the gain that our main ideas bring to the strongest baseline. The results are:

OpenFly-AirSim:

MethodSR (%) ↑OSR (%) ↑NE (m) ↓SPL (%) ↑SSR (%) ↑SOSR (%) ↑
OpenFly-Agent24.67 ± 1.0953.58 ± 1.11130.5 ± 10.922.79 ± 1.8727.37 ± 2.1658.49 ± 2.08
DSF+LDE26.54 ± 1.8049.58 ± 0.84124.5 ± 15.324.96 ± 1.4329.01 ± 1.7353.99 ± 0.26
DSF+TIS26.54 ± 1.6053.03 ± 1.39138.2 ± 19.824.93 ± 1.4428.89 ± 1.8857.24 ± 1.67
Fly0+TPG26.84 ± 0.9054.51 ± 1.93119.1 ± 5.525.10 ± 0.7229.65 ± 0.7959.54 ± 2.64
Fly0+TIS24.31 ± 1.5052.67 ± 0.55144.2 ± 10.522.24 ± 1.1626.49 ± 1.5258.60 ± 0.31
SPF+TIS26.70 ± 1.7551.04 ± 1.69135.8 ± 16.825.08 ± 1.4728.93 ± 1.8555.80 ± 2.07
AheadFly31.26 ± 1.2157.27 ± 1.32105.5 ± 12.027.69 ± 2.0333.32 ± 0.3262.73 ± 1.51

LandmarkFly:

MethodSR (%) ↑OSR (%) ↑NE (m) ↓SPL (%) ↑SSR (%) ↑SOSR (%) ↑LVR (%) ↑GPA (%) ↑SRS (%) ↑SRC (%) ↑
OpenFly-Agent1.93 ± 1.875.98 ± 1.47325.3 ± 10.91.76 ± 0.673.03 ± 1.988.36 ± 2.1552.87 ± 2.76–42.95 ± 1.2742.05 ± 0.92
DSF+LDE11.03 ± 0.9316.40 ± 0.35273.9 ± 9.110.86 ± 0.9512.14 ± 1.0017.52 ± 0.6645.12 ± 0.1156.79 ± 0.7657.31 ± 0.1757.01 ± 0.19
DSF+TIS11.67 ± 0.4216.93 ± 0.87265.0 ± 9.411.53 ± 0.3312.78 ± 0.3018.27 ± 1.3445.12 ± 0.6757.44 ± 0.7058.18 ± 1.2857.89 ± 1.18
Fly0+TPG42.63 ± 1.3552.63 ± 1.35113.4 ± 8.737.75 ± 1.8746.20 ± 2.3154.81 ± 1.7264.49 ± 0.4781.19 ± 0.5782.28 ± 0.3281.94 ± 0.32
Fly0+TIS38.28 ± 2.1350.72 ± 1.73128.1 ± 7.333.13 ± 2.2742.05 ± 1.2554.57 ± 1.3063.84 ± 1.0381.31 ± 1.0979.59 ± 0.9378.86 ± 0.71
SPF+TIS10.93 ± 0.1515.90 ± 0.98264.8 ± 9.810.83 ± 0.2412.01 ± 0.2117.29 ± 1.1545.10 ± 0.2857.41 ± 0.5157.75 ± 0.8157.44 ± 0.32
AheadFly69.64 ± 0.8276.62 ± 0.6860.1 ± 6.161.92 ± 0.6272.17 ± 0.8178.58 ± 0.7170.49 ± 0.1087.70 ± 0.1690.72 ± 0.3890.46 ± 0.37

As can be seen, after switching to our task-parsing module (TPG) and adding the pre-grounding view adjustment, Fly0 improves on every metric to a certain degree, even without any direct change to the VLM backbone. This shows that our main ideas do bring a gain to the strongest existing zero-shot AVLN baseline, and confirms the effectiveness of our approach of improving AVLN performance through dedicated mechanism design without switching to a stronger VLM.

AuthorsOpenReview ↗

Response to Reviewer nWnn (3)

W4 — Potential benchmark bias

AheadFly surpassing its own OpenFly result on LandmarkFly is not because our benchmark deliberately adopts design assumptions consistent with AheadFly. It is because AheadFly's objective is to optimise multi-stage AVLN that places high demands on grounding continuity, whereas LandmarkFly is proposed to raise the quality of multi-stage task generation in existing benchmarks and to fill their gap in stage-level evaluation. The latter effectively releases the evaluation ceiling of various AVLN baselines and separates them, which allows baselines with stronger direction awareness to stand out.

First, the data format and instruction style of the LandmarkFly benchmark were not designed to be consistent with AheadFly. We adopt the stage-wise AVLN task design and five discrete directions (left turn, right turn, ascend, descend, straight flight) to remain consistent with the established setting of the mainstream OpenFly benchmark; on that basis we further raise the task complexity (mainly the complexity and length of the instruction descriptions, and the tortuosity and length of the trajectories) as well as the instruction–trajectory–landmark alignment, while providing an explicit stage count and explicit landmark annotations, so as to support a more comprehensive and finer-grained AVLN evaluation. AheadFly's architecture, by contrast, starts from the goal of enabling the UAV to perform better on such stage-wise tasks. The two therefore merely share a common optimization objective grounded in OpenFly and the classical AVLN task definition, and there is no direct dependency between them.

Second, as shown in Section 3.5 (Method) and Section 4.2 (Experiments), the existing OpenFly dataset has numerous problems in the quality of landmark descriptions, the instruction–trajectory alignment, the annotation of trajectory endpoints, and the tortuosity of trajectories. On its relatively short and straight trajectories, the baselines evaluated on it often need only a single straight flight to reach or overshoot the intended endpoint (as the SR and OSR behaviour of the two minimal baselines DSF+LDE and DSF+TIS, and of SPF, which cannot actively turn, indicates); and ambiguous or repeated multi-segment landmark descriptions make it hard for the agent to determine the specific stage at which it should stop flying, so that baselines lacking an effective task-parsing strategy are difficult to separate (the six metrics are nearly tied across the baselines). These factors may push some simpler baseline designs into a spurious "advantage zone", and may conversely cap the measured performance of baselines that are otherwise reasonably designed and can follow the instruction well. Hence, when faced with tasks whose descriptions are more complex, whose trajectories are more tortuous, and which require the UAV to act strictly according to the language instruction (specifications in Appendix D), baselines that perform well on OpenFly may either degrade (the minimal baselines and SPF) or have their performance ceiling released (Fly0 and AheadFly). The larger improvement of our method stems precisely from what we claim as its novelty — the "think ahead" and "act ahead" architecture — and this is strongly supported by the stage-level metrics.

Reviewer_znhJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AheadFly is a training-free modular system for multi-stage aerial vision-and-language navigation. A task-planning graph separates landmark queries from transition directions; pre-grounding pose adjustment and a next-stage directional path bias prepare subsequent observations. The paper also introduces LandmarkFly with stage-aligned landmark annotations and reports improvements on both LandmarkFly and OpenFly-AirSim.

Critical strengths

Coordinating the approach to one landmark with the viewpoint needed for the next addresses a concrete navigation problem. LandmarkFly provides useful intermediate annotations beyond endpoint evaluation.

Table 1 reports SR of 30.17% versus the best baseline value of 24.71% on OpenFly-AirSim, and 70.30% versus Fly0+TIS at 37.90% on LandmarkFly. Modular baselines share parsing and grounding backbones. Table 3 supplies substantial component evidence: full-system SR is 66.50%, versus 40.00% without lookahead and 18.50% without pre-grounding adjustment; MFG also exceeds DSF with TPG and pre-grounding adjustment fixed.

Critical weaknesses

The stage-level metric definitions leave the grounding-continuity interpretation uncertain. Appendix E describes LVR and GPA using normalized distances, while the result tables mark both as higher-is-better; the transformation is not specified there. The definition does not explain how projected centers establish unoccluded visibility, or how SRS assigns trajectory times to ordered landmarks and handles omitted stages. This is a documentation and construct-validity concern, not a demonstrated scoring bug, and it does not negate the endpoint gains or controlled ablations.

The comparable-obstacle-avoidance claim exceeds what the reported RMC comparison alone establishes. RMC takes a minimum over stage-averaged central-view depth quantiles, so it is neither collision frequency nor the minimum vehicle clearance along a trajectory. The similar proxy values and faster reported runtime are useful evidence, but the safety interpretation should stay within that scope.

Questions for authors

Which result establishes visible, ordered landmark progress under the intended evaluation? Please clarify the LVR/GPA distance-to-score conversion and visibility handling, and the SRS temporal assignment and skipped-stage rule. This would distinguish missing documentation from a substantive limitation of the metrics.

15 comments
AuthorsOpenReview ↗

Response to Reviewer znhJ (1)

We thank the reviewer for the careful assessment and address all concerns below.

Metric groups used in the tables: Standard: SR, OSR, NE, SPL. Soft: SSR, SOSR. Stage-level: LVR, GPA, SRS, SRC. Obstacle avoidance: RMC, TMC.

W1-1 & Q1-1 — Computation of the stage-level metrics and treatment of invisible landmarks

The LVR and GPA metrics in Appendix E in fact use the difference between the normalized Euclidean distance and 1 as the result, so as to match the intuitive "higher is better" design and to make the statistics convenient. For LVR, besides the Euclidean distance from the projected geometric center of the landmark to the image center, the area share of its bounding box is an equally important factor in how visible it is. For invisible objects, which cannot be projected, we simply set the corresponding metric to 0.

First, to specify how the three stage-level metrics are computed, we give their formal mathematical definitions here, and further explain why LVR and GPA are "higher-is-better" metrics:

{\sqrt{(W/2)^2 + (H/2)^2}}) A_{i,t}$$ $$\mathrm{GPA}_{i,t} = 1 - \frac{\lVert b_{i,t} - q_{i,t} \rVert_2}{\sqrt{W^2 + H^2}}$$ $$\mathrm{SRS} = \frac{1}{N} \sum_{i=1}^{N} \max_{t}\bigl(\mathrm{SSR}_i(t)\bigr)$$ LVR and GPA do not take the normalized Euclidean distance directly as the result; we further take the difference between that quantity and 1. Intuitively, the value before the subtraction does describe the normalized distance from the GT landmark center to the image center (LVR), or from the center of the grounding box to the GT landmark center (GPA), so the complement naturally describes how close the actual point is to the expected point, and "higher is better" is the appropriate choice. In addition, LVR does not assess landmark visibility solely by "how close the target is to the image center": it simultaneously accounts for the area share $A_{i,t}$ of the landmark's bounding box. This design makes the evaluation consistent with the intuition that "an object that is closer to the view center and larger is more salient", and is thus more reasonable. As for GT landmarks that leave the camera's field of view or fall behind the camera during flight, the invisible 3-D bounding box cannot be projected onto the image plane at all, so both LVR and GPA are set to 0 directly, which is a reasonable design.
AuthorsOpenReview ↗

Response to Reviewer znhJ (2)

W1-2 & Q1-2 — A stage-continuity-aware refinement of SRS

Among existing benchmarks, the SRS metric in LandmarkFly is the first to answer the question "which subtasks did the actual flight cover", and it can be used to assess the spatial reachability of each stage landmark for the UAV; it retains a certain reference value in finer-grained stage-level evaluation.

However, as the reviewer notes, the current SRS lacks an explicit temporal matching rule for stage skipping, so a UAV that takes a roundabout route may still obtain a decent score on a multi-stage task. We have therefore refined SRS so that, while retaining the overall task-completion progress or subtask-coverage statistics, it incorporates the continuity and ordering between stages. The resulting metric is the Success Rate by Continuity (SRC), computed as:

SRC=1N∑i=1Nmax⁡t(SSRi(t))⋅di\mathrm{SRC} = \frac{1}{N} \sum_{i=1}^{N} \max_{t}\bigl(\mathrm{SSR}_i(t)\bigr) \cdot d_i

The core idea of SRC is to attach a discount factor did_i to every term max⁡t(SSRi(t))\max_{t}(\mathrm{SSR}_i(t)) that originally entered the SRS computation, so that the score for the current subtask is adjusted according to how well the preceding subtasks were completed. Concretely, if the preceding subtasks were poorly completed when this subtask was finished (i.e. the coverage of the preceding subtasks is low, which can also be read as a low SRS over the preceding task set), the soft success rate of this stage (i.e. how close the UAV came to the stage landmark) is further discounted, which lowers the original SRS result.

This design inherits the advantage of SRS in requiring only the UAV's own pose and the GT landmark positions, and is therefore immune to perception noise. We therefore consider SRC an efficient and stable complement to SRS that answers the question of whether the subtasks are being completed in order. Moreover, since only our LandmarkFly benchmark provides explicit landmark annotations, SRC can only be computed on that benchmark. The results are:

MethodSR (%) ↑OSR (%) ↑NE (m) ↓SPL (%) ↑SSR (%) ↑SOSR (%) ↑LVR (%) ↑GPA (%) ↑SRS (%) ↑SRC (%) ↑
OpenFly-Agent1.93 ± 1.875.98 ± 1.47325.3 ± 10.91.76 ± 0.673.03 ± 1.988.36 ± 2.1552.87 ± 2.76–42.95 ± 1.2742.05 ± 0.92
DSF+LDE11.03 ± 0.9316.40 ± 0.35273.9 ± 9.110.86 ± 0.9512.14 ± 1.0017.52 ± 0.6645.12 ± 0.1156.79 ± 0.7657.31 ± 0.1757.01 ± 0.19
DSF+TIS11.67 ± 0.4216.93 ± 0.87265.0 ± 9.411.53 ± 0.3312.78 ± 0.3018.27 ± 1.3445.12 ± 0.6757.44 ± 0.7058.18 ± 1.2857.89 ± 1.18
Fly0+TPG42.63 ± 1.3552.63 ± 1.35113.4 ± 8.737.75 ± 1.8746.20 ± 2.3154.81 ± 1.7264.49 ± 0.4781.19 ± 0.5782.28 ± 0.3281.94 ± 0.32
Fly0+TIS38.28 ± 2.1350.72 ± 1.73128.1 ± 7.333.13 ± 2.2742.05 ± 1.2554.57 ± 1.3063.84 ± 1.0381.31 ± 1.0979.59 ± 0.9378.86 ± 0.71
SPF+TIS10.93 ± 0.1515.90 ± 0.98264.8 ± 9.810.83 ± 0.2412.01 ± 0.2117.29 ± 1.1545.10 ± 0.2857.41 ± 0.5157.75 ± 0.8157.44 ± 0.32
AheadFly69.64 ± 0.8276.62 ± 0.6860.1 ± 6.161.92 ± 0.6272.17 ± 0.8178.58 ± 0.7170.49 ± 0.1087.70 ± 0.1690.72 ± 0.3890.46 ± 0.37

As expected from the definition, SRS is the theoretical upper bound of SRC, but the gap between the two is in fact small. We attribute this to the fact that, in low-altitude scenes that are themselves open and large, multi-stage tasks with generally long trajectories make the "roundabout route" described above hard to realise in practice. Once a preceding subtask is missed, the UAV can hardly continue to complete the subsequent ones, so stage skipping rarely occurs in practice.

AuthorsOpenReview ↗

Response to Reviewer znhJ (3)

W1-3 & Q3-1 — LandmarkFly construction and the source of the performance gain

LandmarkFly is constructed with reference to the existing design of OpenFly but provides data of higher quality. The performance gain of AheadFly on it comes from the higher instruction–trajectory–landmark consistency in LandmarkFly together with the view-adjustment-related mechanism design in AheadFly.

First, the data format and instruction style of the LandmarkFly benchmark were not designed to be consistent with AheadFly. We adopt the stage-wise AVLN task design and five discrete directions (left turn, right turn, ascend, descend, straight flight) to remain consistent with the established setting of the mainstream OpenFly benchmark; on that basis we further enrich the diversity of the tasks and improve the instruction–trajectory–landmark alignment, while providing an explicit stage count and explicit landmark annotations, so as to support a finer-grained AVLN evaluation.

As for why we choose landmarks with unoccluded centers that support an unambiguous feature description, this follows from the requirement of the task itself: an ambiguous or homogeneous object description would make the intent unclear even to a human pilot, and embodied autonomous-exploration tasks that require searching for a currently invisible, occluded object in an open scene are not the research direction we are discussing here. Our focus is on improving the instruction-following capability of a low-altitude UAV under relatively explicit language guidance. AheadFly's architecture, by contrast, starts from the goal of enabling the UAV to perform better on such stage-wise tasks. The two therefore merely share a common optimization objective grounded in OpenFly and the classical AVLN task definition, and there is no direct dependency between them.

Second, as shown in Section 3.5 (Method) and Section 4.2 (Experiments), the existing OpenFly dataset has numerous problems in the quality of landmark descriptions, the instruction–trajectory alignment, the annotation of trajectory endpoints, and the tortuosity of trajectories. On its relatively short and straight trajectories, the baselines evaluated on it often need only a single straight flight to reach or overshoot the intended endpoint (as the SR and OSR behaviour of the two minimal baselines DSF+LDE and DSF+TIS, and of SPF, which cannot actively turn, indicates); and ambiguous or repeated multi-segment landmark descriptions make it hard for the agent to determine the specific stage at which it should stop flying, so that baselines lacking an effective task-parsing strategy are difficult to separate (the six metrics are nearly tied across the baselines).

These factors may push some simpler baseline designs into a spurious "advantage zone", and may conversely cap the measured performance of baselines that are otherwise reasonably designed and can follow the instruction well. Hence, when faced with tasks whose descriptions are more complex, whose trajectories are more tortuous, and which require the UAV to act strictly according to the language instruction (specifications in Appendix D), baselines that perform well on OpenFly may either degrade (the minimal baselines and SPF) or have their performance ceiling released (Fly0 and AheadFly). The larger improvement of our method stems precisely from what we claim as its novelty — the "think ahead" and "act ahead" architecture.

Reviewer_4uaGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work addresses multi-stage aerial VLN, where each landmark must be grounded from the terminal pose of the previous stage. AheadFly is a modular, training-free framework. An LLM decomposes the instructions into a Task Planning Graph with landmark queries and transition directions, a viewpoint adjustment reorients the drone before grounding, and a multi-force controller approaches each landmark while correcting the trajectory toward the next step. The authors additionally introduce LandmarkFly, a benchmark with 1000 episodes across 4 AirSim scenes, with per-stage landmark annotations and stage-level metrics (LVR/GPA/SRS). AheadFly achieves an SR of 30.17 on OpenFly-AirSim, about 5.5 points above the best baseline, and 70.30 on LandmarkFly.

Critical strengths

1.The modular baselines use the same grounding VLM and the same analysis LLM as AheadFly and even receive the multi-stage analysis, so these comparisons are matched at the component level. The minimal baseline DSF+TIS answers the question "wouldn't a simpler controller be enough?" The paper honestly reports, contrary to its own framing, that on OpenFly-AirSim it is on par with SPF+TIS in SR and slightly exceeds it on LandmarkFly. The 11 ablations in Table 3 and the substitution of the grounding VLM backbone in Table 4, where no single backbone dominates alone more directly support that the improvement comes from the mechanism rather than the choice of models.

2.The stage-level metrics (LVR/GPA/SRS) are a useful contribution: they reveal when and where grounding continuity breaks, which an end-to-end SR cannot show.

Critical weaknesses

1.The largest gain (+32.4) is achieved on the authors' own benchmark. LandmarkFly covers only 4 AirSim scenes, the instructions and landmark annotations are generated automatically, and no quantitative human review of their correctness is available. The stage-level metrics likewise reuse the automatically generated landmark annotations, so annotation errors feed directly into the evaluation. Both the method and the benchmark emphasize sequential approach to landmarks; evidence from a broader instruction distribution would strengthen the evaluation of generalization. Partial external validation on a public benchmark was achieved (+5.46 SR), but no variance is given, making it difficult to assess the robustness of the gain.

2.Many behaviors depend on manual mappings. Section 3.3 gives illustrative mappings, for example "ahead and to the left" as a 45-degree yaw turn, or "ahead at a higher elevation" as a 10 m climb, but the rules' coverage of the instruction vocabulary is not analyzed. No sensitivity analysis is carried out for the controller's weights (η, δ); when an erroneous grounding leads the trajectory away from the following steps, this is neither detected nor corrected — a point the paper itself acknowledges in Section 5.

Questions for authors

1.Please report the variance across repeated runs for Table 1, especially the +5.46 difference on OpenFly-AirSim.

2.Please report the success rate of the human review of the correctness of instructions and annotations, the error rate of the landmark-identifier mapping, and how occluded or out-of-sight landmarks are handled in the stage-level metrics. Also confirm that the public release includes all scripts for generation, landmark-identifier mapping, and evaluation implementation.

3.To what degree does performance depend on the fixed mappings in the viewpoint adjustment and on the weights η and δ?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 4uaG (1)

We thank the reviewer for the careful assessment and address all concerns below.

Metric groups used in the tables: Standard: SR, OSR, NE, SPL. Soft: SSR, SOSR. Stage-level: LVR, GPA, SRS.

W1-1 — Quantitative human review of the dataset

Compared with the existing OpenFly dataset, LandmarkFly provides, while preserving high simulation fidelity and reliability, an explicit stage segmentation and landmark description, a more consistent instruction–trajectory–landmark triple, and more realistic and more diverse within-trajectory direction changes. The human evaluation on the sampled data supports the advantage of our data-generation method.

Because we carry out AVLN data generation in scenes built on Unreal Engine 4 and obtain the ground-truth bounding boxes of target entities through the AirSim plugin, the fidelity of the explicit landmark annotations in LandmarkFly is guaranteed to a certain degree. The detailed task instructions are then generated and integrated by an MLLM (we use a Qwen-series model) conditioned on the camera input and the landmark annotations, and its reasoning capability is in itself sufficient to ensure the reliability of the generated instructions. To supplement this with a more objective and quantifiable assessment of dataset quality provided by human evaluators, we designed the following evaluation protocol for OpenFly and LandmarkFly.

Concretely, we randomly sampled 100 tasks from each of LandmarkFly and OpenFly-AirSim, covering different stage counts. After random mixing, the 200 samples were handed to external experts for human evaluation. Each sample was rated along four aspects:

  1. Is the stage segmentation of the original instruction unambiguous, i.e. is it clear what to do at each step?
  2. Do the landmark descriptions in the instruction map unambiguously onto landmarks in the scene, i.e. how ambiguous is the landmark description?
  3. Judged against the GT trajectory, is the instruction prone to misinterpretation, i.e. how consistent is the instruction's guidance with the trajectory?
  4. Does the GT trajectory exhibit a rich variety of direction changes, i.e. how tortuous is the trajectory?

Each aspect is scored on a 1–5 scale. The detailed evaluation protocol will be added to Appendix K of the camera-ready version as supplementary material. The final scores are:

Dimension (1–5)LandmarkFlyOpenFlyDifference (former − latter)
Clarity of stage segmentation4.544.14+0.40
Identifiability of landmark descriptions4.013.25+0.76
Instruction–trajectory consistency3.642.98+0.66
Diversity of direction changes3.382.45+0.93

As can be seen, LandmarkFly's advantages are largest in "diversity of direction changes" and "identifiability of landmark descriptions", followed by "instruction–trajectory consistency". This shows that our landmark-centric AVLN data-generation approach reliably produces test samples of higher quality than OpenFly and whose motion more closely matches realistic tasks.

W1-2 — Instruction distribution and generalization

Compared with OpenFly, LandmarkFly exhibits a broader and more balanced instruction distribution.

Concretely, as for the distribution of the discrete actions contained in the language instructions, we adopt the same five transition directions as OpenFly (forward, ascend, descend, left, right), which is also the most commonly used discrete action space in existing AVLN tasks, and we control the frequency of each direction during data generation so that it stays as close as possible to the distribution in OpenFly-AirSim or is more balanced. As for the overall instruction statistics, the longest instruction in LandmarkFly contains 190 words, the average over all tasks is 94.2 words (67.1 words in OpenFly-AirSim), and the total vocabulary size is 1,271. The maximum number of stages is 6 and the mean is 3.62 (2.86 in OpenFly-AirSim), as described in Appendix D.

The distribution of the discrete directions mentioned in the GT instructions of the two benchmarks is summarized below:

DirectionLandmarkFly countShare of all directionsTasks containing itShare of all tasksOpenFly-AirSim countShare of all directionsTasks containing itShare of all tasks
Forward132236.5%77677.6%159243.8%119999.1%
Left102628.4%62262.2%88724.4%72960.2%
Right86623.9%57957.9%87924.2%71158.8%
Ascend3399.4%27827.8%1684.6%16113.3%
Descend661.8%555.5%1072.9%1008.3%
Total3619100%––3633100%––
AuthorsOpenReview ↗

Response to Reviewer 4uaG (2)

W1-3 & Q1 — Variance and stability of the results

After thorough validation and comparison, we consider the results of the main table (Table 1) to be robust and reliable, and the possibility of any metric being inflated or deflated by incidental factors to be excluded.

Concretely, to verify the stability of the baseline comparisons, we repeated the experiments of every compared baseline four times under an identical configuration on both OpenFly-AirSim and LandmarkFly, and combined these repeats with the original results to compute the mean and standard deviation of each metric, updating and supplementing the existing results. The results on the two datasets are:

OpenFly-AirSim:

MethodSR (%) ↑OSR (%) ↑NE (m) ↓SPL (%) ↑SSR (%) ↑SOSR (%) ↑
OpenFly-Agent24.67 ± 1.0953.58 ± 1.11130.5 ± 10.922.79 ± 1.8727.37 ± 2.1658.49 ± 2.08
DSF+LDE26.54 ± 1.8049.58 ± 0.84124.5 ± 15.324.96 ± 1.4329.01 ± 1.7353.99 ± 0.26
DSF+TIS26.54 ± 1.6053.03 ± 1.39138.2 ± 19.824.93 ± 1.4428.89 ± 1.8857.24 ± 1.67
Fly0+TPG26.84 ± 0.9054.51 ± 1.93119.1 ± 5.525.10 ± 0.7229.65 ± 0.7959.54 ± 2.64
Fly0+TIS24.31 ± 1.5052.67 ± 0.55144.2 ± 10.522.24 ± 1.1626.49 ± 1.5258.60 ± 0.31
SPF+TIS26.70 ± 1.7551.04 ± 1.69135.8 ± 16.825.08 ± 1.4728.93 ± 1.8555.80 ± 2.07
AheadFly31.26 ± 1.2157.27 ± 1.32105.5 ± 12.027.69 ± 2.0333.32 ± 0.3262.73 ± 1.51

LandmarkFly:

MethodSR (%) ↑OSR (%) ↑NE (m) ↓SPL (%) ↑SSR (%) ↑SOSR (%) ↑LVR (%) ↑GPA (%) ↑SRS (%) ↑
OpenFly-Agent1.93 ± 1.875.98 ± 1.47325.3 ± 10.91.76 ± 0.673.03 ± 1.988.36 ± 2.1552.87 ± 2.76–42.95 ± 1.27
DSF+LDE11.03 ± 0.9316.40 ± 0.35273.9 ± 9.110.86 ± 0.9512.14 ± 1.0017.52 ± 0.6645.12 ± 0.1156.79 ± 0.7657.31 ± 0.17
DSF+TIS11.67 ± 0.4216.93 ± 0.87265.0 ± 9.411.53 ± 0.3312.78 ± 0.3018.27 ± 1.3445.12 ± 0.6757.44 ± 0.7058.18 ± 1.28
Fly0+TPG42.63 ± 1.3552.63 ± 1.35113.4 ± 8.737.75 ± 1.8746.20 ± 2.3154.81 ± 1.7264.49 ± 0.4781.19 ± 0.5782.28 ± 0.32
Fly0+TIS38.28 ± 2.1350.72 ± 1.73128.1 ± 7.333.13 ± 2.2742.05 ± 1.2554.57 ± 1.3063.84 ± 1.0381.31 ± 1.0979.59 ± 0.93
SPF+TIS10.93 ± 0.1515.90 ± 0.98264.8 ± 9.810.83 ± 0.2412.01 ± 0.2117.29 ± 1.1545.10 ± 0.2857.41 ± 0.5157.75 ± 0.81
AheadFly69.64 ± 0.8276.62 ± 0.6860.1 ± 6.161.92 ± 0.6272.17 ± 0.8178.58 ± 0.7170.49 ± 0.1087.70 ± 0.1690.72 ± 0.38

Under the same configuration, no baseline shows a marked fluctuation in any metric, which supports the robustness of the evaluation reported in this paper.

W2-1 — Coverage of the discrete action mapping over instruction intent

Through an analysis of the task scenes, we consider the existing mapping from a discrete direction in the instruction to a numerically parameterised control signal to be sufficient to cover the requirements of the task.

This design follows OpenFly's practice of mapping a turning action to 30° and an altitude change to 2 m. What differs from OpenFly-Agent, which uses the mapped value directly for motion control, is that in AheadFly the numerically parameterised control signals obtained through the mapping (e.g. "turn left 45°", "ascend 10 m") serve to guarantee in advance that a landmark lying in roughly that direction enters the camera's field of view as far as possible. Specifically, for horizontal turning we choose 45°, because the camera in our fixed experimental setting has a 90° horizontal field of view, so a single left/right rotation already captures the front-left/front-right scene in full. For vertical motion we choose 10 m and 5 m, because the target objects in AVLN scenes are mostly buildings 20–30 m tall, so a single ascent/descent step already brings a farther/nearer landmark into the camera's field of view. Taken together, in terms of its intended purpose, this action-mapping design already covers the intent expressed in the language instruction.

AuthorsOpenReview ↗

Response to Reviewer 4uaG (3)

W2-2 & Q3 — Sensitivity analysis of the weights (η\eta, δ\delta)

Varying the two weights η\eta and δ\delta in AheadFly has little effect on performance, indicating a low sensitivity to these two parameters. Nevertheless, increasing or decreasing either of them slightly lowers performance, and our default setting is also the best combination in the overall results.

Concretely, the two weights are computed as follows (Appendix C):

\dfrac{|V_{\mathrm{att}}|}{|V_{\mathrm{att}}| + |V_{\mathrm{rep}}|}, & 0.3 \le p \le 0.7, \\\\[6pt] \max(p,\\, 1-p), & \text{otherwise.} \end{cases}$$ $$\delta = \frac{50}{50 + \mathrm{{Q}_{0.2}}(D_{blk}^5)}$$ To analyse the sensitivity of performance to their specific values and to justify our current choice, we varied $\eta$ and $\delta$ while holding all other settings fixed and re-ran the experiments. This mainly covers increasing/decreasing the boundary values of the parameter $p$ in $\eta$, and varying the quantile used in the denominator of $\delta$. The results are: **Analysis of $\eta$:** | Parameter | Setting | SR (%) ↑ | OSR (%) ↑ | NE (m) ↓ | SPL (%) ↑ | SSR (%) ↑ | SOSR (%) ↑ | LVR (%) ↑ | GPA (%) ↑ | SRS (%) ↑ | |:---|:---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | $\eta$ | $0.2 \le p \le 0.8$ | 69.40 | 74.30 | 61.92 | 60.33 | 71.35 | 76.75 | 70.73 | 87.61 | 90.11 | | $\eta$ | **$0.3 \le p \le 0.7$** | **70.30** | **76.80** | **50.48** | **62.60** | **72.77** | **78.79** | 70.32 | 87.80 | **91.07** | | $\eta$ | $0.4 \le p \le 0.6$ | 70.10 | 76.80 | 57.11 | 62.47 | 72.43 | 78.71 | **70.96** | **87.84** | 91.05 | **Analysis of $\delta$:** | Parameter | Setting | SR (%) ↑ | OSR (%) ↑ | NE (m) ↓ | SPL (%) ↑ | SSR (%) ↑ | SOSR (%) ↑ | LVR (%) ↑ | GPA (%) ↑ | SRS (%) ↑ | |:---|:---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | $\delta$ | $Q_{0.1}$ | 69.00 | 75.90 | 58.29 | 61.41 | 71.50 | 78.30 | 70.64 | **87.85** | 90.71 | | $\delta$ | $Q_{0.2}$ | **70.30** | **76.80** | **50.48** | **62.60** | **72.77** | **78.79** | 70.32 | 87.80 | **91.07** | | $\delta$ | $Q_{0.3}$ | 68.10 | 74.70 | 58.84 | 61.27 | 71.17 | 76.88 | **70.83** | 87.78 | 90.21 | As can be seen, the internal settings of $\eta$ and $\delta$ have little effect on performance, and the default combination is the best overall. As for why the two parameters matter at all, our analysis is as follows. $\eta$ directly controls the weight of the attractive force that guides the UAV toward the landmark; if the UAV is not promptly aligned with the target at the flight stage indicated by $p$, the current stage is likely to end at a position with a poor observation, which in turn affects the subsequent landmark grounding. $\delta$ attenuates the repulsive force according to the openness of the region directly in front of the UAV; a larger (or smaller) quantile makes the avoidance behaviour too lax (or too cautious), which likewise affects whether the UAV can reach the neighbourhood of the stage landmark. # W2-3 — Error recovery after grounding drift Our method indeed does not include the design of an error-recovery scheme for a failed or biased landmark grounding, because our focus remains on preventing grounding drift in advance rather than remedying it after the fact. As discussed in **Section 2.1** on modular AVLN, such a backtracking-style strategy has to be designed specifically, and none of the baselines we compare against — SPF, Fly0, or even the end-to-end OpenFly-Agent — incorporates such a strategy either. Therefore, to keep the focus of the studied problem from drifting, our current work does not make this part of the paper's contributions, but we regard it as an important direction for future extension. # Q2-1 — Manual evaluation results and treatment of out-of-view landmarks The quantitative results of the manual evaluation show that LandmarkFly outperforms the existing OpenFly benchmark in AVLN data quality. As for landmarks outside the field of view, the stage-level metrics are set directly to 0 when computed. As described in W1-1, after sampling we had external experts assess the validity and quality of the dataset along four aspects — clarity of stage segmentation, ambiguity of landmark descriptions, consistency between instruction and trajectory, and diversity of direction changes in the trajectory — and compared the results of LandmarkFly and OpenFly-AirSim, demonstrating the advantage of the dataset constructed by our pipeline. As for GT landmarks that leave the camera's field of view or fall behind the camera during flight, the invisible 3-D bounding box cannot be projected onto the image plane at all, so all stage-level metrics are set to 0 directly, which is a reasonable design. # Q2-2 — Release of the data-generation and evaluation scripts All scripts for task generation, landmark-label mapping, and evaluation will be released together with the method code once the paper is accepted.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.