- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
In this paper, the authors introduce AheadFly, a method for multi-stage aerial vision-and-language navigation, along with LandmarkFly, a new benchmark for evaluating intermediate landmark grounding during navigation. The core idea is to think and act ahead by parsing instructions into explicit landmark queries and transition directions. This adjusts the UAV view and flight path in advance. The benchmark contains 1,000 multi-stage trajectories with explicit alignment among instructions, trajectories, and landmarks. In the experiments, AheadFly achieves the best performance among the evaluated baselines on OpenFly and LandmarkFly.
Critical strengths
The proposed AheadFly framework presents an intuitive idea for multi-stage aerial navigation. In particular, the novel idea of adjusting the UAV view and flight path in advance is interesting. This can help the UAV more reliably find the next landmark.
Critical weaknesses
1. Degraded performance of the Fly0 baseline.
This paper reports baseline performance using Fly0. However, its performance is significantly degraded compared to the original paper. Fly0 reports 64.67% SR on OpenFly in its original paper, whereas it achieves only 23.22% SR in this paper, while AheadFly achieves 30.17%. It is unclear whether the reported improvement reflects a genuine advantage over Fly0 or a weak reproduction of the baseline.
2. Lack of comparison under the original Fly0 setting.
The authors modify the Fly0 setup by replacing its grounding model and adding Qwen3-Max-based TIS parsing rather than evaluating the original system under its original experimental setting. It is also important to report performance on the AerialVLN benchmark.
3. The effectiveness of AheadFly on stronger baselines remains unclear.
There are several existing AVLN methods with significantly higher SR on OpenFly, including SPF (42.94%), OpenFly (33.61%), and UAV-Flow (32.14%), compared with 30.17% for AheadFly in this paper. It would be important to verify whether the proposed ideas still provide consistent gains when applied to these stronger methods.
4. Potential benchmark bias.
LandmarkFly appears to be closely aligned with the design assumptions of AheadFly, since both are structured around explicit landmark stages and discrete transition directions. Moreover, the reported improvement is much larger on the authors' own LandmarkFly benchmark than on the public OpenFly. This raises concern that the proposed benchmark may disproportionately favor the proposed method.
Questions for authors
- Why does Fly0 drop from 64.67% SR in its original OpenFly evaluation to only 23.22% in this paper?
- Does AheadFly still provide consistent improvements when its proposed mechanisms are applied to stronger baselines?
5 comments
Response to Reviewer nWnn (2)
W3 & Q2 — Effectiveness of AheadFly on stronger baselines
Because the experimental configurations differ, these numbers are not suitable for direct comparison. Under our unified choice of VLM backbone and data subset, replacing the task-parsing module of the strongest existing zero-shot AVLN baseline, Fly0, with our TPG and adding the pre-grounding view adjustment improves its performance on both OpenFly and LandmarkFly to a certain degree, demonstrating the benefit of our ideas to methods of the same family.
First, we consider a direct horizontal comparison of results across different experimental configurations to be insufficient: the numbers are not commensurable. Those numbers come from the Fly0 paper and may involve OpenFly scenes and data that we do not cover, or other experimental settings that differ from ours. Our present setup follows the released code of SPF and Fly0 as closely as possible, and we applied to them the minimal modification needed to adapt to the OpenFly and LandmarkFly datasets, strictly in line with the relevant discussion in their papers. Under the present configuration, we therefore consider that the differences in actual performance among the methods revealed by our experiments still corroborate the effectiveness of AheadFly itself.
Second, fully incorporating our ideas into modular zero-shot methods of the same family (with instruction parsing, landmark grounding and motion control modules) would leave them differing from AheadFly essentially only in the VLM backbone. We therefore applied only two of our ideas — the task-parsing module and the pre-grounding view adjustment — to the best-performing baseline, Fly0, obtaining the new baseline Fly0+TPG, and observed its performance on OpenFly-AirSim and LandmarkFly, in order to analyse the gain that our main ideas bring to the strongest baseline. The results are:
OpenFly-AirSim:
| Method | SR (%) ↑ | OSR (%) ↑ | NE (m) ↓ | SPL (%) ↑ | SSR (%) ↑ | SOSR (%) ↑ |
|---|---|---|---|---|---|---|
| OpenFly-Agent | 24.67 ± 1.09 | 53.58 ± 1.11 | 130.5 ± 10.9 | 22.79 ± 1.87 | 27.37 ± 2.16 | 58.49 ± 2.08 |
| DSF+LDE | 26.54 ± 1.80 | 49.58 ± 0.84 | 124.5 ± 15.3 | 24.96 ± 1.43 | 29.01 ± 1.73 | 53.99 ± 0.26 |
| DSF+TIS | 26.54 ± 1.60 | 53.03 ± 1.39 | 138.2 ± 19.8 | 24.93 ± 1.44 | 28.89 ± 1.88 | 57.24 ± 1.67 |
| Fly0+TPG | 26.84 ± 0.90 | 54.51 ± 1.93 | 119.1 ± 5.5 | 25.10 ± 0.72 | 29.65 ± 0.79 | 59.54 ± 2.64 |
| Fly0+TIS | 24.31 ± 1.50 | 52.67 ± 0.55 | 144.2 ± 10.5 | 22.24 ± 1.16 | 26.49 ± 1.52 | 58.60 ± 0.31 |
| SPF+TIS | 26.70 ± 1.75 | 51.04 ± 1.69 | 135.8 ± 16.8 | 25.08 ± 1.47 | 28.93 ± 1.85 | 55.80 ± 2.07 |
| AheadFly | 31.26 ± 1.21 | 57.27 ± 1.32 | 105.5 ± 12.0 | 27.69 ± 2.03 | 33.32 ± 0.32 | 62.73 ± 1.51 |
LandmarkFly:
| Method | SR (%) ↑ | OSR (%) ↑ | NE (m) ↓ | SPL (%) ↑ | SSR (%) ↑ | SOSR (%) ↑ | LVR (%) ↑ | GPA (%) ↑ | SRS (%) ↑ | SRC (%) ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| OpenFly-Agent | 1.93 ± 1.87 | 5.98 ± 1.47 | 325.3 ± 10.9 | 1.76 ± 0.67 | 3.03 ± 1.98 | 8.36 ± 2.15 | 52.87 ± 2.76 | – | 42.95 ± 1.27 | 42.05 ± 0.92 |
| DSF+LDE | 11.03 ± 0.93 | 16.40 ± 0.35 | 273.9 ± 9.1 | 10.86 ± 0.95 | 12.14 ± 1.00 | 17.52 ± 0.66 | 45.12 ± 0.11 | 56.79 ± 0.76 | 57.31 ± 0.17 | 57.01 ± 0.19 |
| DSF+TIS | 11.67 ± 0.42 | 16.93 ± 0.87 | 265.0 ± 9.4 | 11.53 ± 0.33 | 12.78 ± 0.30 | 18.27 ± 1.34 | 45.12 ± 0.67 | 57.44 ± 0.70 | 58.18 ± 1.28 | 57.89 ± 1.18 |
| Fly0+TPG | 42.63 ± 1.35 | 52.63 ± 1.35 | 113.4 ± 8.7 | 37.75 ± 1.87 | 46.20 ± 2.31 | 54.81 ± 1.72 | 64.49 ± 0.47 | 81.19 ± 0.57 | 82.28 ± 0.32 | 81.94 ± 0.32 |
| Fly0+TIS | 38.28 ± 2.13 | 50.72 ± 1.73 | 128.1 ± 7.3 | 33.13 ± 2.27 | 42.05 ± 1.25 | 54.57 ± 1.30 | 63.84 ± 1.03 | 81.31 ± 1.09 | 79.59 ± 0.93 | 78.86 ± 0.71 |
| SPF+TIS | 10.93 ± 0.15 | 15.90 ± 0.98 | 264.8 ± 9.8 | 10.83 ± 0.24 | 12.01 ± 0.21 | 17.29 ± 1.15 | 45.10 ± 0.28 | 57.41 ± 0.51 | 57.75 ± 0.81 | 57.44 ± 0.32 |
| AheadFly | 69.64 ± 0.82 | 76.62 ± 0.68 | 60.1 ± 6.1 | 61.92 ± 0.62 | 72.17 ± 0.81 | 78.58 ± 0.71 | 70.49 ± 0.10 | 87.70 ± 0.16 | 90.72 ± 0.38 | 90.46 ± 0.37 |
As can be seen, after switching to our task-parsing module (TPG) and adding the pre-grounding view adjustment, Fly0 improves on every metric to a certain degree, even without any direct change to the VLM backbone. This shows that our main ideas do bring a gain to the strongest existing zero-shot AVLN baseline, and confirms the effectiveness of our approach of improving AVLN performance through dedicated mechanism design without switching to a stronger VLM.
Response to Reviewer nWnn (3)
W4 — Potential benchmark bias
AheadFly surpassing its own OpenFly result on LandmarkFly is not because our benchmark deliberately adopts design assumptions consistent with AheadFly. It is because AheadFly's objective is to optimise multi-stage AVLN that places high demands on grounding continuity, whereas LandmarkFly is proposed to raise the quality of multi-stage task generation in existing benchmarks and to fill their gap in stage-level evaluation. The latter effectively releases the evaluation ceiling of various AVLN baselines and separates them, which allows baselines with stronger direction awareness to stand out.
First, the data format and instruction style of the LandmarkFly benchmark were not designed to be consistent with AheadFly. We adopt the stage-wise AVLN task design and five discrete directions (left turn, right turn, ascend, descend, straight flight) to remain consistent with the established setting of the mainstream OpenFly benchmark; on that basis we further raise the task complexity (mainly the complexity and length of the instruction descriptions, and the tortuosity and length of the trajectories) as well as the instruction–trajectory–landmark alignment, while providing an explicit stage count and explicit landmark annotations, so as to support a more comprehensive and finer-grained AVLN evaluation. AheadFly's architecture, by contrast, starts from the goal of enabling the UAV to perform better on such stage-wise tasks. The two therefore merely share a common optimization objective grounded in OpenFly and the classical AVLN task definition, and there is no direct dependency between them.
Second, as shown in Section 3.5 (Method) and Section 4.2 (Experiments), the existing OpenFly dataset has numerous problems in the quality of landmark descriptions, the instruction–trajectory alignment, the annotation of trajectory endpoints, and the tortuosity of trajectories. On its relatively short and straight trajectories, the baselines evaluated on it often need only a single straight flight to reach or overshoot the intended endpoint (as the SR and OSR behaviour of the two minimal baselines DSF+LDE and DSF+TIS, and of SPF, which cannot actively turn, indicates); and ambiguous or repeated multi-segment landmark descriptions make it hard for the agent to determine the specific stage at which it should stop flying, so that baselines lacking an effective task-parsing strategy are difficult to separate (the six metrics are nearly tied across the baselines). These factors may push some simpler baseline designs into a spurious "advantage zone", and may conversely cap the measured performance of baselines that are otherwise reasonably designed and can follow the instruction well. Hence, when faced with tasks whose descriptions are more complex, whose trajectories are more tortuous, and which require the UAV to act strictly according to the language instruction (specifications in Appendix D), baselines that perform well on OpenFly may either degrade (the minimal baselines and SPF) or have their performance ceiling released (Fly0 and AheadFly). The larger improvement of our method stems precisely from what we claim as its novelty — the "think ahead" and "act ahead" architecture — and this is strongly supported by the stage-level metrics.
Response to Reviewer nWnn (1)
We thank the reviewer for the careful assessment and address all concerns below.
Metric groups used in the tables: Standard: SR, OSR, NE, SPL. Soft: SSR, SOSR. Stage-level: LVR, GPA, SRS, SRC.
W1 & Q1 — Degraded performance of the Fly0 baseline
The difference between Fly0's performance in our experiments and the performance reported in its original paper stems mainly from the different type and scale of the VLM backbone used, and from the different data subset used for evaluation. The original Fly0 paper uses Qwen2.5-VL-32B as the landmark grounding backbone and may have used the full OpenFly dataset (the paper does not state this). In our experimental setup we use LocateAnything-3B as the VLM backbone and use only the AirSim subset of OpenFly (1,210 samples in total).
Concretely, neither the original Fly0 paper nor its open-source implementation states how its subset of OpenFly was selected, so one must assume it used the full dataset. When running the experiments we found that several of the scenes provided by OpenFly are in fact unusable (most notably the 3DGS-based dataset that has not been released). Given the constraints of our hardware, we finally conducted the main experiments only on its six AirSim subsets. Even so, the 1,210 instruction–trajectory pairs across the six scenes still constitute a substantial scale.
Since the subset selection in the experimental setup may introduce bias, directly taking the numbers claimed in the Fly0 paper as its performance under our configuration would be unfair. On the other hand, our AheadFly uses the 3B-parameter LocateAnything as the VLM backbone for landmark grounding; unless a unified grounding model is used, the evaluation could be affected by the VLM's own performance and thus become unfair.
For these two reasons, we had to align both the dataset selection and the configuration of the landmark-grounding VLM across baselines before running the experiments. Together with the fluctuation that incidental factors may introduce, the resulting performance naturally differs from the numbers reported in the original papers. That said, given that OpenFly-Agent still comes close to its claimed performance under the same configuration (except that its end-to-end 7B backbone cannot be replaced) — 22.64% versus the 27.78% aggregate over four scenes claimed in its repository — we consider the actual performance of each baseline under our unified experimental setup to be objective and reasonable.
W2 — Lack of comparison under the original Fly0 setting
Given what Fly0 has released so far, a comparison under its original setting is difficult to carry out. Considering both the fairness of the evaluation and our emphasis on the gain attributable to the mechanisms in the architecture, we consider the comparison and analysis under the present experimental configuration to be reasonable and valid. As for the AerialVLN dataset, given that OpenFly already supersedes it in several respects (also a simulation scene built on UE4 + AirSim, a larger dataset scale, a more sensible construction method, and higher instruction–trajectory quality), and given the limited time available in the rebuttal period, we have not scheduled comparative experiments on that dataset.
Two changes were made to Fly0's original architecture in the main comparison of this paper. First, to avoid an unfair evaluation dominated by the strength of the VLM itself, we replaced its landmark grounding backbone (Qwen2.5-VL-32B in the original Fly0 paper). As for the design and choice of the task-parsing module: on the one hand, it keeps the parsing backbone identical to AheadFly's so that only one variable changes; on the other hand, Fly0 has not open-sourced the scripts related to its experimental setup and instruction parsing, so our choice of TIS is a reasonable inference from the input types its released implementation actually supports — it accepts only a single-sentence language instruction, supplied interactively. Restoring Fly0's architecture exactly to its current open-source implementation would therefore make it impossible to use the present datasets as a testbed for comparison.
In summary, our adjustments to Fly0's architecture are to some extent constrained by its actual implementation. Considering both the fairness of the baseline comparison and our emphasis on evaluating the mechanisms within the architecture rather than the strength of the module backbones, we consider the present experimental setup reasonable.