acceptodds
Under review as a conference paper at ICLR 2027

AlignFly: Landmark-Alignment Reinforcement Fine-Tuning for UAV Vision-Language Navigation

Abstract

Vision-language navigation for unmanned aerial vehicles (UAV-VLN) requires agents to follow natural language instructions using rapidly changing visual observations along three-dimensional trajectories. Such instructions usually describe routes via ordered visual landmarks, which provide stage-wise feedback for navigation. Existing UAV-VLN research rarely explores the correspondence between landmarks and navigation stages. In this paper, we propose AlignFly, a landmark-alignment reinforcement fine-tuning (RFT) method for optimizing navigation actions. Starting from a vision-language model (VLM) initialized through supervised fine-tuning (SFT) on expert demonstrations, AlignFly samples candidate action chunks from the same expert state and executes them independently in a simulator. A VLM-as-Judge is introduced to compare the visual evidence of the target landmark in post-action observations of candidates and construct rewards for group relative policy optimization (GRPO). To provide the required target-landmark information, we develop an automated annotation pipeline that extracts ordered landmark phrases from instructions, scores their visual evidence in expert trajectory observations, and monotonically aligns them with trajectory stages through dynamic programming. We apply this pipeline to a scene dataset of OpenFly and use the resulting annotations for RFT. Experiments on the dataset demonstrate consistent improvements on the seen split and a held-out scene, while ablations support the contribution of landmark-alignment feedback to UAV navigation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Unmanned Aerial Vehicles, Vision-Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iuDpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AlignFly, a landmark-alignment reinforcement fine-tuning method for UAV vision-language navigation. An automated pipeline extracts ordered landmark phrases from instructions, scores their visibility in expert post-action observations using a VLM judge, and assigns landmarks to trajectory stages through monotonic dynamic programming. Starting from a supervised navigation policy, AlignFly samples candidate action chunks from the same expert state, executes them independently in simulation, and applies GRPO using landmark-evidence rewards together with distance, format, and stopping rewards.

Experiments train on 9,500 trajectories from one OpenFly scene and evaluate on 231 seen-scene trajectories and 231 trajectories from another scene. AlignFly reports success rates of 44.64% and 32.85%, compared with 41.13% and 28.26% for AirVLN-R1. Reward ablations, an additional experiment applying the procedure to OpenFly-Agent, and a manual audit of 100 annotated trajectories provide supporting evidence.

Critical strengths
  1. I liked the improvements supported by the reward ablations. In Table 2, removing the landmark-alignment reward reduces SR from 44.64% to 39.94% on Test-Seen and from 32.85% to 24.94% on Test-Unseen, while keeping the other rewards and training settings unchanged. This supports the usefulness of landmark feedback. Applying the complete RFT procedure to OpenFly-Agent also improves SR by 5.96 and 7.32 percentage points, suggesting that its benefits are not limited to the authors’ own SFT initialization.

  2. The automated annotation pipeline is another useful contribution. It provides stage-level landmark supervision from existing instructions and expert trajectories. The manual check in Appendix D supports its quality: 85 of 100 sampled trajectories agree with the human target-landmark assignments at every decision state, and another nine differ at only one state. These annotations are used during training, so the navigation policy does not require annotated target landmarks or expert future observations at inference.

  3. I also liked that the landmark reward evaluates the visual outcomes of executed actions. Candidate action chunks are executed independently from the same expert state, allowing their resulting observations to be compared under the same initial conditions. This provides semantic feedback for viewpoint changes and landmark transitions that a reward based only on distance to the final goal may miss.

Critical weaknesses
  1. My main concern is that the experiments do not isolate the benefit of ordered stage alignment. Removing the landmark reward also removes the semantic feedback from the VLM judge. This supports the usefulness of landmark feedback, but does not show how much the proposed alignment procedure contributes. I would like to see comparisons with simpler alternatives, such as rewarding instruction landmarks without considering their order, or selecting a landmark independently at each state without the monotonic constraint. This is particularly relevant because landmark-focused reinforcement rewards were already explored in Landmark-RxR (He et al., 2021). The automated annotations and VLM-based scoring are useful differences, but their added value needs clearer experimental support. The LE analysis uses the same judge as training and, as the authors acknowledge, provides diagnostic evidence.

  2. I also could not reconcile some reported success rates with the evaluation setup. Section 4.1 states that each test split contains 231 trajectories and that results are averaged over three runs. If SR is computed as the equally weighted fraction of successful episodes, its three-run average should equal 100k/693100k/693 for an integer kk. The reported values of 44.64% and 32.85% cannot be obtained under this calculation, even after rounding to two decimal places. There may be an additional averaging step or a different denominator, but this needs clarification.

    The paper also reports no variability estimates and does not clarify whether the seeds vary training, inference, or both. Together with evaluation on only one training scene and one held-out scene, this makes it difficult to assess how reliable and general the improvements are.

Questions for authors
  1. Could you clarify how SR is calculated in Tables 1–3? Please provide the number of evaluated episodes and successful episodes in each run for AlignFly and AirVLN-R1, together with the averaging procedure. With 231 episodes per run and three equally weighted runs, the closest possible averages around 44.64% are 44.59% and 44.73%. Is there an additional averaging step or a different denominator?

  2. To better understand the contribution of ordered stage alignment, could you compare AlignFly with two simpler alternatives: a landmark reward that ignores stage order, and independent landmark selection at each state without the monotonic constraint? Keeping the initialization, judge, and other RFT settings unchanged would help show how much the proposed alignment procedure contributes.

  3. Do the three seeds vary training, inference, or both? Please report the variability across runs or confidence intervals for the main performance differences. Also, which scenes were used, how were they selected, and was a separate validation split used to select checkpoints and hyperparameters?

1 comment

We thank the reviewer for identifying the inconsistency in the reported success rates and for requesting clearer experimental controls and variability estimates. To address these concerns, we corrected the evaluation denominator to include all scheduled episodes and updated Tables 1–3. We now report means and sample standard deviations alongside individual run results and success counts (Appendix I, Tables 7–10). We have also added controlled comparisons of target-landmark assignment strategies (Section 4.5, Table 4). Our responses clarify the scope of evaluation seeds, scene selection, data splits, checkpoint selection, and hyperparameter tuning. We address each question below.

1.The original evaluation script discarded episodes terminated due to unparseable predicted action sequences, which reduced the denominator and introduced statistical bias. Under the revised evaluation protocol, all scheduled episodes are included in the denominator; episodes terminated early by parsing errors are counted as failures. Each split contains 231 episodes per run. The overall success rate across the three replicated runs is calculated as the total number of successes divided by 693 (231 × 3). The corrected success rates are 44.59% and 32.76% for AlignFly on Test-Seen and Test-Unseen, respectively, and 41.13% and 28.14% for AirVLN-R1. We have updated Tables 1–3 and report individual per-run results in Appendix I.

2.We added a controlled comparison in Section 4.5 and Table 4. Independent visual matching selects the highest-scoring instruction landmark at each expert state without a sequence constraint. Uniform segmentation divides the expert trajectory into approximately equal-length intervals by decision step and assigns landmarks in instruction order. Both variants and monotonic DP share the SFT checkpoint, VLM judge, reward formula, other reward terms, and GRPO settings. Monotonic DP achieves mean SRs of 44.59% and 32.76% on Test-Seen and Test-Unseen. Its gains over independent matching are 5.34 and 9.52 percentage points, and its gains over uniform segmentation are 3.61 and 8.95 points, computed from the unrounded rates. These comparisons support using both visual evidence and instruction order to assign stage targets.

3.The three random seeds only vary action sampling during evaluation for fixed checkpoints. In the revised manuscript, Tables 1–4 report the mean and sample standard deviation for all four metrics, and Appendix I lists the individual results for each run. For example, AlignFly achieves an SR of 44.59 ± 0.75% on Test-Seen and 32.76 ± 0.66% on Test-Unseen. These standard deviations quantify the variability originating from evaluation sampling. We use the OpenFly dataset [1], which has also been adopted for training and evaluation by Xu et al. [2] and Ma et al. [3].The OpenFly-S subset used by Xu et al. contains both airsim_18 and airsim_16. In our work, we train on airsim_18 and reserve 231 trajectories from this scene for Test-Seen. airsim_16 is held out from training, and 231 trajectories sampled from it are used as Test-Unseen. This scene comparison evaluates the model’s generalization to one held-out layout under the same simulator engine. The training data is split 85%/15% into training and validation partitions. The initial learning rates and LoRA ranks for both stages are selected based on the validation loss on this fixed validation split. We initialize RFT with the final SFT checkpoint and evaluate the final RFT checkpoint.

References

[1] Gao et al. OpenFly: A Comprehensive Platform for Aerial Vision-Language Navigation. ICLR, 2026.

[2] Xu et al. Aerial Vision-Language Navigation With a Unified Framework for Spatial, Temporal and Embodied Reasoning. IEEE Transactions on Circuits and Systems for Video Technology, 2026.

[3] Ma et al. From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation. ACM Multimedia, 2026.

Reviewer_JqYCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

AlignFly proposes a landmark-aligned reinforcement fine-tuning method for UAV vision-language navigation (VLN). The method first acquires basic navigation capabilities through supervised fine-tuning, then uses a VLM to assess evidence of target landmarks in images resulting from candidate actions. These assessments provide stage-specific semantic rewards for optimizing the navigation policy with GRPO. The main contributions are the landmark-alignment reward and an automated annotation pipeline that aligns ordered landmarks in instructions with stages of expert trajectories. Experiments in the OpenFly simulator demonstrate improved success rates over the strongest baseline in both seen and unseen environments. Ablation studies and transfer experiments on OpenFly-Agent further support the method’s effectiveness.

Critical strengths
  1. The method effectively exploits landmark semantics in VLN through reinforcement fine-tuning, enabling the model to better incorporate landmark information into navigation decisions and long-horizon task management.
  2. The introduction of landmark evidence (LE) offers a useful direction for designing semantic rewards to improve VLN performance.
Critical weaknesses
  1. The alignment mechanism relies on the visual distinguishability of textual landmark descriptions. When an instruction repeatedly refers to the same landmark or includes distinct landmark instances with identical descriptions—particularly at consecutive stages—single-frame visual evidence and a monotonic ordering constraint may be insufficient to uniquely determine stage correspondence. The paper provides neither a dedicated mechanism nor targeted experiments to address this ambiguity.
Questions for authors

Could the authors provide a more detailed analysis of failure cases, distinguishing among landmark recognition errors, incorrect stage transitions, action execution errors, and incorrect stopping decisions? An empirical breakdown of these failure modes, ideally comparing models with and without the Landmark-Alignment module, would help clarify the module’s contribution and remaining limitations.

1 comment

We thank the reviewer for the questions concerning navigation failures. In response, we added a failure-mode analysis comparing AlignFly with the variant without the landmark-alignment reward under matched evaluation episodes and the same inference seed (Appendix H, Table 6). We examined every failed rollout for each policy in the selected evaluation run and classified the first critical error into landmark instance selection, stage transition, action selection or execution, or stopping decision. The revised manuscript reports the counts and proportions of these errors and discusses the improvements and remaining failure modes. We provide the detailed response below.

We define the potential error categories as follows:

Landmark instance selection error: The instruction stage is clear and the intended landmark is present, yet the UAV keeps moving toward a visually similar distractor landmark.

Stage transition error: The policy switches stages at a time inconsistent with the instruction trigger; for example, turning before passing the specified building.

Action selection or execution error: The policy attempts the maneuver required by the current stage with incorrect direction, displacement, or altitude change. Cases where an unparseable action sequence constitutes the first critical error are also included in this category.

Stopping decision error: The UAV has reached the vicinity of the goal but fails to stop there, where the stopping failure is the first critical error. Cases for which an oracle path could reach the goal successfully but the UAV fails to stop at the end fall into this category. We determine whether the first error is a stopping decision error based on the rollout trace.

We have added a failure-mode analysis in Appendix H (Table 6) of the revised manuscript, comparing AlignFly with the variant without the landmark-alignment reward. For each split, both models are evaluated on the same 231 episodes using the same inference seed from the second evaluation run. We annotate all failed rollouts separately for each model. Using the instruction, visual observations, predicted actions, and executed trajectory, we assign a label to each failed rollout for the earliest observable error that accounts for the failure.

Percentages are computed within the failed rollouts of each model and split. Compared with the ablation variant without the landmark-alignment reward, AlignFly exhibits fewer first errors caused by landmark instance selection and a lower proportion of such errors among all failed trajectories on both Test-Seen and Test-Unseen. This indicates that our landmark-alignment reward mitigates landmark instance selection errors to some extent and helps the navigation policy better align with the correct landmark during action decision-making.

Reviewer_pehpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AlignFly, a landmark-alignment reinforcement fine-tuning framework for UAV vision-language navigation. It introduces an automated annotation pipeline that aligns instruction landmarks with expert trajectory stages using VLM-based visual evidence and dynamic programming. The results annotations are used to construct landmark-alignment rewards for GRPO, enabling navigation policies to learn from the visual outcomes of candidate actions. Experiments on OpenFly demonstrate improvements in both seen and unseen scenes.

Critical strengths
  1. The paper addresses a meaningful problem in UAV navigation by introducing stage-specific landmark feedback to complement conventional geometric navigation rewards.
  2. The proposed landmark-alignment reward improves navigation performance on both seen and unseen splits, with particularly notable gains in success rate over the SFT baseline and the RFT variant without landmark feedback.
  3. The human validation of the automatic landmark annotations, and additional experiments on OpenFly-Agent, supporting the effectiveness and applicability of the proposed training procedure.
Critical weaknesses
  1. I think the proposed landmark reward cannot precisely and sufficiently reflect actual navigation progress, because when the landmark remains visible from multiple positions or directions, the UAV may move toward an incorrect route.
  2. Although the landmark reward is validated through ablation, the paper does not compare monotonic dynamic programming with simpler landmark assignment strategies. It is therefore unclear how much of the improvement comes specifically from stage-aware alignment.
  3. Fine-grained landmark alignment and landmark-related rewards have been explored in Landmark-RxR, while AirVLN-R1 already employs SFT followed by GRPO with subgoal-alignment rewards.
Questions for authors
  1. Can the authors clarify how increased landmark visibility corresponds to actual navigation progress? Have they analyzed cases where the target landmark becomes more visible without the UAV moving along the correct route?
  2. How does the proposed monotonic alignment compare with simpler strategies, such as independent visual matching or uniform trajectory segmentation? This comparison would help establish the contribution of the alignment algorithm itself.
  3. What is the specific advantage of the proposed visual-evidence reward over the subgoal-alignment rewards used in AirVLN-R1? A controlled comparison under the same SFT initialization and GRPO settings would clarify the methodological contribution.
4 comments

We thank the reviewer for the questions and suggestions concerning the landmark reward and stage alignment. In response, we added quantitative statistics and qualitative case studies examining situations where landmark evidence increases while the UAV deviates from the intended route (Appendix G, Figure 6). We also added controlled comparisons with independent visual matching and uniform segmentation to evaluate the contribution of monotonic alignment (Section 4.5, Table 4). Furthermore, we clarified the methodological differences between our visual-evidence reward and the geometric subgoal-alignment rewards in AirVLN-R1. We address each question below.

AuthorsOpenReview ↗

Response to Question 1

We thank the reviewer for raising this question. The correspondence between landmark visibility and navigation progress depends on the current instruction stage. Instruction landmarks provide visual references for successive segments of the route. Within a stage, an action that brings the UAV to a viewpoint consistent with the intended maneuver makes the target landmark easier to recognize. Stronger visual evidence can therefore serve as a local semantic cue for reaching the stage’s intended visual context. In AlignFly, this evidence is quantified by the log-probability difference between affirmative and negative responses from the VLM judge. The correspondence also depends on viewing geometry, which motivates the following empirical analysis.

We added a decision-level analysis in Appendix G and Figure 6. For each of the 231 trajectories in each test split, we randomly selected one expert decision state. From the same initial state, we independently executed 100 candidate action chunks sampled from the AlignFly policy. Among candidates whose landmark evidence increased relative to the initial observation, we calculated the fraction that ended farther from the goal than the expert action. This fraction falls in the 0–10% interval for 187/231 states (80.95%) on Test-Seen and 189/231 states (81.82%) on Test-Unseen. Thus, at most selected states, evidence-increasing actions usually attain goal proximity comparable to or better than the expert outcome under this local diagnostic. These results support using landmark evidence as a stage-dependent progress cue.

We also specifically examined exceptions. Figure 6 presents one state with a mismatch rate above 90% from each split. In the Test-Seen example, the candidate increases landmark evidence from 1.65 to 2.74, exceeding the expert post-action score of 1.90. In the Test-Unseen example, it increases the score from 1.09 to 1.93, exceeding the expert score of 1.66. However, both candidates end farther from the goal than their expert counterparts and deviate from the intended route. These examples demonstrate that viewpoint changes can strengthen landmark evidence while producing poorer geometric progress, illustrating the ambiguity raised by the reviewer.

The reward design mitigates this issue through complementary components. The composite reward optimized by GRPO includes a goal-distance reward. For candidates starting from the same initial state, greater goal-distance reduction receives a larger distance reward within its unclipped range. This geometric signal reduces the relative reward of actions that obtain a favorable landmark view while making insufficient progress toward the goal.

AuthorsOpenReview ↗

Response to Question 2

We thank the reviewer for suggesting this comparison. Independent visual matching assigns each expert state the instruction landmark with the highest evidence score in its post-action observation, without considering assignments at other states. This preserves local visual relevance, but a landmark that remains visible across multiple stages can produce temporally inconsistent assignments. Uniform trajectory segmentation divides the expert states into approximately equal-length intervals by decision step and assigns landmarks in instruction order. This preserves order, but assumes similar stage durations and can place stage transitions at positions inconsistent with the visual evidence. Our monotonic alignment jointly optimizes target-landmark assignments over the entire expert trajectory using cumulative visual evidence and a nondecreasing landmark-index constraint. It prevents backward stage assignments while allowing stage durations to adapt to the evidence.

To quantify these differences, we added controlled experiments in Section 4.5 and Table 4. We used each of the three assignment strategies to generate target-landmark labels for RFT. All three variants share the same SFT checkpoint, VLM judge, landmark-reward formula, auxiliary reward terms, and GRPO hyperparameter settings. Only the target-landmark assignment rule changes.

Monotonic DP achieves the best mean performance across all four metrics—NE, SR, OSR, and SPL—on both test splits. Its SR is 44.59 ± 0.75% on Test-Seen and 32.76 ± 0.66% on Test-Unseen. Compared with independent visual matching, SR increases by 5.34 and 9.52 percentage points, respectively. Compared with uniform segmentation, the increases are 3.61 and 8.95 percentage points, respectively.

AuthorsOpenReview ↗

Response to Question 3

We thank the reviewer for raising this question. The core difference between our method and AirVLN-R1 lies in what they align against. AirVLN-R1 adopts an alignment reward based on geometric subgoal states, computed from the UAV’s distance to intermediate subgoals and the yaw angle between the UAV heading and the reference subgoal orientation. This type of reward guides the UAV toward expert-defined state points. In contrast, our approach evaluates the semantic content of the observation view after action execution. This constitutes a distinct supervision signal: instead of only measuring geometric distance and heading consistency, it judges whether the post-action observation visually matches the landmark specified by the current language instruction. In our model comparison experiments, we re-implemented AirVLN-R1 under identical SFT initialization and GRPO settings on the same OpenFly dataset.

Table 1 shows that our method achieves a higher success rate than AirVLN-R1 on both Test-Seen (44.59% vs. 41.13%) and Test-Unseen (32.76% vs. 28.14%).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.