acceptodds
Under review as a conference paper at ICLR 2027

VoLN: Vision-Only Long-Horizon Navigation—Paradigm, Benchmark, and Method

Abstract

Long-horizon navigation in open environments requires sustained reasoning over changing visual evidence. We propose Vision-Only Long-Horizon Navigation (VoLN), a navigation paradigm for multi-stage reasoning that couples visual goal grounding, context-dependent cue interpretation, and closed-loop planning across a route. Goal views specify the destination, while onboard observations and interaction history support online decisions. We instantiate VoLN for aerial navigation through VoLN-UAV, a 7,210-episode benchmark across 17 simulated environments, combining continuous 3D flight, substantial viewpoint variation, and held-out scene evaluation. Active and passive semantic beacons create repeated demands for cue selection, providing an intermediate setting between instruction-guided Vision-and-Language Navigation and exploration in unfamiliar environments. We further present VoLN-MLLM, a two-stage visual-semantic planning framework that aligns self-supervised visual features with a structured semantic space and predicts short-horizon waypoints and stopping decisions from observation history, goal views, retrieved concepts, and proprioception. On Test-Unseen, it achieves success rates of 7.4%, 4.5%, and 1.8% on Easy, Normal, and Hard episodes, respectively. These results highlight remaining challenges in temporal evidence integration and reliable long-horizon execution. Code is available at https://anonymous.4open.science/r/VoLN-UAV-DDC6/.

Primary area:
datasets and benchmarks
Keywords:
Vision-and-Language Navigation, Benchmark, Visual-Semantic Reasoning, Image-Goal Navigation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TzpWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Vision-Only Long-Horizon Navigation (VoLN), a navigation paradigm in which agents navigate toward visually specified goals without separate language instructions. The task emphasizes long-horizon visual-semantic reasoning through semantic beacons encountered during navigation. The authors construct VoLN-UAV, a benchmark containing 7,210 episodes across 17 simulated environments, with active beacons providing task-relevant guidance and passive beacons serving as distractors. They also propose VoLN-MLLM, a two-stage framework that aligns DINO visual features with CLIP's semantic space and uses a LoRA-adapted Vicuna planner to predict short-horizon waypoints and stopping decisions. The proposed method outperforms the adapted baselines across the reported metrics. However, the absolute success rates remain low, reaching only 7.4%, 4.5%, and 1.8% on the Easy, Normal, and Hard subsets of Test-Unseen, respectively.

Critical strengths
  1. The paper proposes a meaningful extension of image-goal navigation by introducing semantic guidance encountered during execution. This setting provides an opportunity to investigate how embodied agents use successive environmental cues to support long-horizon decisions without explicit route instructions.

  2. VoLN-UAV includes long-distance navigation, continuous 3D movement, viewpoint variation, and unseen environments. The distinction between active and passive beacons provides a potentially useful mechanism for studying task-relevant visual reasoning under distracting observations.

Critical weaknesses
  1. A central contribution of VoLN is the requirement to interpret successive task-relevant semantic beacons while ignoring distracting cues. However, the current experiments do not directly establish whether the proposed model learns this capability. In particular, the ablations focus on visual alignment, LoRA adaptation, and backbone selection, without isolating the effects of active beacons, passive beacons, semantic retrieval, or observation history. Consequently, it remains unclear whether the reported improvements arise from multi-stage semantic reasoning or primarily from stronger visual representations and model adaptation.

  2. Although VoLN-MLLM outperforms the evaluated baselines, the absolute success rates remain extremely low, particularly on long-horizon episodes. Moreover, the comparisons rely primarily on adapted versions of older navigation architectures, without evaluating stronger recent image-goal or aerial navigation approaches. The absence of stronger reference baselines or informative upper-bound evaluations makes it difficult to determine whether the low success rates reflect the inherent difficulty of the proposed benchmark, limitations of the selected models, or aspects of the task construction.

Questions for authors
  1. Could the authors report controlled experiments that remove active beacons, remove passive beacons, or vary the number of distracting beacons? How does the model distinguish task-relevant cues from visually similar distractors? Such experiments would directly test the central reasoning capability targeted by VoLN.

  2. Could the authors include more recent image-goal navigation or UAV navigation baselines, or provide a reference upper bound using privileged route information or reliable beacon interpretation? Given the very low absolute SR, these experiments would help establish the solvability and difficulty of the proposed benchmark.

  3. How does performance change when the semantic bank or observation history is removed? Since the proposed framework combines several established components, these ablations are important for determining which components actually contribute to long-horizon visual-semantic reasoning.

  4. Appendix B describes a nine-dimensional state vector containing linear velocity, angular velocity, and relative displacement, while the main text excludes GPS and world-frame position. Could the authors clarify how relative displacement is computed and confirm that it does not introduce privileged goal or reference-trajectory information?

I would currently lean toward a borderline reject, but I would be open to increasing the rating if the rebuttal provides clarifications on the questions and concerns.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer TzpW — Part 1

We thank the reviewer for the suggestions on beacon controls, stronger baselines, and benchmark difficulty.

Q1. Beacon controls and cue relevance

We added No-Passive, which removes passive beacons while preserving active guidance, together with the Left-only placement control.

The comparisons use the same trained checkpoint, paired Test-Unseen episodes, and runtime seeds, without retraining. Standard uses equal-probability left/right placement of active beacons. Left-only fixes them on the left while retaining their forward and vertical offsets and the passive-beacon layout. No-Passive preserves the Standard active-beacon layout. Section 6.3 and Table 3 report:

SettingActive-beacon placementNE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
StandardRandom side94.615.016.949.910.7
Left-onlyFixed left98.214.216.148.710.0
No-PassiveRandom side91.816.118.051.211.4

Removing passive beacons improves route adherence and goal-region entry under unchanged active guidance. This indicates that competing cues remain a source of navigation difficulty for the reference policy. The slight decrease with Left-only shows a modest influence of placement geometry on navigation decisions.

Active beacons provide the route guidance that VoLN is designed to study. Removing them changes the task to image-goal exploration and no longer evaluates how the agent interprets and follows the designated visual guidance. We therefore retain active beacons in the controls examining beacon placement and competing cues.

Regarding cue relevance, the planner receives no active/passive labels, reference route, or future reference waypoints. A global image representation is used to retrieve a semantic descriptor, and its similarity score measures image–descriptor correspondence. The planner combines this semantic input with the recent visual and constructed-state history to predict waypoints.

The planner has no separate per-beacon relevance classifier. Section 5 and Appendix B describe the input flow; Q3 reports the semantic and history ablations.


Q2. Stronger baselines and failure analysis

We added TravelUAV-VG, adapted from the OpenUAV implementation. It replaces language instructions with goal images and uses visual history and constructed motion states to predict eight body-frame 3D waypoints. TravelUAV-VG and VoLN-MLLM use the same Vicuna-7B language backbone, training split, waypoint supervision, controller, target-confirmation procedure, and evaluation budget.

Under the revised protocol, SR requires correct target confirmation and episode termination within 40 m of the target asset's 3D center. OSR records whether the executed trajectory enters the same region. All methods use the same target-confirmation procedure and evaluation criteria.

Section 6.1 describes the baseline adaptation, and Table 1 reports the following Test-Unseen results:

DifficultyMethodNE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
EasyTravelUAV-VG102.010.913.248.77.4
EasyVoLN-MLLM79.616.418.555.412.1
NormalTravelUAV-VG146.87.79.832.64.6
NormalVoLN-MLLM101.814.115.946.19.9
HardTravelUAV-VG202.03.14.615.91.6
HardVoLN-MLLM138.211.513.137.27.1

TravelUAV-VG achieves higher SR than the three lightweight baselines across all difficulty levels. VoLN-MLLM further improves on this stronger baseline, with its advantages in SR and nDTW increasing from Easy to Hard.

To examine the remaining failures, Section 6.2 reports the following breakdown across five Test-Unseen runs:

Failure eventCountShare of failures/%
Budget exhaustion without goal-region entry3,94385.81
Collision3277.12
Premature target-triggered termination1132.46
Goal-region entry without successful termination1032.24
Other execution errors1092.37
Total4,595100.00

Budget exhaustion before goal-region entry is the dominant recorded failure event. This indicates that reaching the destination within the evaluation budget remains the main challenge for the reference policy, while collision avoidance and reliable termination also affect performance.

AuthorsOpenReview ↗

Reply to Reviewer TzpW — Part 2

Q3. How does performance change when the semantic bank or observation history is removed?

We thank the reviewer for suggesting these input ablations. We added Current-only and No-Semantic, with the corresponding input settings applied consistently during both training and testing.

  • Current-only retains only the current visual, semantic, and constructed-state inputs, removing the preceding seven observations from the history.
  • No-Semantic retains the aligned visual features and eight-observation visual and constructed-state history, while removing the retrieved-descriptor tokens and their similarity embeddings.

Both comparisons retain the same target-confirmation procedure, data split, training budget, and runtime seed pool. Section 6.3 and Appendix B specify the input configurations. Table 2 reports the Test-Unseen results:

SettingCT/s ↓EER/% ↓NE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
Full1.250.594.615.016.949.910.7
Current-only1.050.9134.87.810.243.85.3
No-Semantic1.401.1128.98.712.735.55.0

Current-only reduces average cycle time but lowers navigation performance. Retaining recent history allows earlier observations, semantic cues, and motion states to remain available for subsequent waypoint predictions.

No-Semantic also reduces navigation performance while preserving the aligned visual representation. This supports an additional contribution from explicit semantic inputs beyond visual alignment.


Q4. Relative displacement and planner inputs

We thank the reviewer for requesting this clarification. The revision distinguishes platform measurements ptp_t from the nine-dimensional constructed state supplied to the planner:

zt=[(vtb)⊤,(ωtb)⊤,(Δrtb)⊤]⊤∈R9.\displaystyle z_t=[(\mathbf{v}_t^b)^\top,(\boldsymbol{\omega}_t^b)^\top,(\Delta\mathbf{r}_t^b)^\top]^\top\in\mathbb{R}^9.

The three components are body-frame linear velocity, angular velocity, and displacement between consecutive observation samples.

The displacement term records the UAV's 3D position change between two consecutive RGB observations, sampled 2 s apart, expressed in the body frame at the current observation. It is computed by subtracting the previous measured position from the current position and rotating the difference into the current body frame.

Let rt\mathbf{r}_t be the measured UAV position at observation time tt, and let RtR_t rotate the current body frame into the world frame:

Δrtb=Rt⊤(rt−rt−1).\displaystyle \Delta\mathbf{r}_t^b=R_t^\top(\mathbf{r}_t-\mathbf{r}_{t-1}).

The first displacement is set to zero.

The pose measurements come from AirSim in simulation and motion capture on the physical testbed. The planner receives this three-dimensional displacement together with body-frame linear and angular velocities as its nine-dimensional motion-state input.

Sections 3 and 5 and Appendix B now specify these distinctions between platform measurements, state construction, and planner inputs.

Thank you for the detailed rebuttal and additional experiments. I appreciate the efforts to improve the evaluation, particularly the inclusion of TravelUAV-VG, the semantic and observation-history ablations, and the clarification of relative displacement. These additions have addressed several of my original concerns.

Before raising my score, please discuss the issue further:

The No-Passive experiment is useful, but the improvement is relatively modest (15.0% to 16.1% SR). Moreover, removing visual distractors does not directly demonstrate that the model can identify task-relevant active beacons or distinguish their semantic guidance from competing cues. While I understand that removing all active beacons changes the task setting, controlled interventions such as masking selected active beacons or changing their directional semantics could directly assess whether the model relies on these cues for navigation. Such analysis would better support the central motivation of VoLN.

Reviewer_xeF2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces VoLN, a paradigm for achieving long-horizon navigation in open environments with visual goals and without separate language-instruction inputs. Embedded semantic beacons in the scene can be used to provide cues for sustained reasoning for the agent, in a way analogous to how humans consult road signs during navigation. After formulating the VoLN paradigm, the authors further propose VoLN-MLLM, which aligns self-supervised visual features with a structured semantic space, and predicts short-horizon waypoints and stopping decisions based on observation history, goal views, retrieved concepts, and proprioception. On the Test-Unseen split, the framework achieves success rates of 7.4%, 4.5%, and 1.8% on Easy, Normal, and Hard difficulty levels, respectively.

Critical strengths

This paper introduces VoLN, a navigation paradigm for long-range, multi-stage visual-semantic reasoning in open environments, and proposes VoLN-MLLM to combine visual-semantic alignment with closed-loop waypoint and stopping prediction based on goal views, recent observations, and proprioception. It removes separate language instructions and updates executable navigation plans solely through the agent's interpretation of semantic beacons along the route and the integration of observation history, providing a feasible approach for intelligent long-horizon UAV navigation.

Critical weaknesses
  1. The route analysis of the VoLN-MLLM method relies on a fixed semantic bank, so its generalization ability is tied to the scale of the semantic bank and it cannot effectively utilize unseen in-process scene cues. Moreover, no fine-grained spatial partitioning method is mentioned for the extracted Top-k matches, and it is unclear how to effectively distinguish between distractors and helpful beacons.

  2. The paper designs a vision-only long-horizon navigation paradigm and proposes the VoLN-MLLM method accordingly. However, the appendix mentions that the designed history window only contains the eight most recent frames, without mentioning whether a long-horizon memory window is designed. Relying solely on a short-term observation history window, the paper does not specifically point out how long-horizon navigation is achieved.

  3. The visual-goal variants of the three instruction-following architectures used for comparison do not receive separate language inputs for comparison. The experiments replace language inputs with the VoLN interface, which merely shows the standalone advantage of VoLN-MLLM. The authors need clearer comparative experiments to show what the gap is when VoLN-MLLM relies solely on in-process beacons for navigation after removing separate language inputs compared to not removing them, and why this is done. Although it achieves the best performance compared to the baselines set in the paper, the compared methods (Seq2Seq, CMA, and LAG) are only lightweight methods, whereas VoLN-MLLM introduces a multimodal large model with far more parameters than the three lightweight methods above. The difference in parameter scale is insufficient to support the superiority of VoLN-MLLM, and strong models from the past two years could be selected as comparative baselines.

  4. In the design of the beacons, the active beacons clearly have indicative meanings. The authors could conduct comparative experiments between using language inputs and visually observing active beacons, or remove the active beacons and retain only the passive beacons to verify the effectiveness of the method; otherwise, it looks as if the model is merely fitting a predetermined route.

Questions for authors
  1. Why do the comparative experiments select lightweight methods prior to 2023 such as Seq2Seq, CMA, and LAG as baselines, instead of selecting models from the past two years with comparable parameter scales for comparison?

  2. For the VoLN-MLLM method, why is there no dedicated introduction to the implementation mechanism for achieving long-horizon navigation?

  3. Why are there no experiments specifically evaluating the proposed beacons in the experimental section to verify the substitutive role of beacons for separate language instructions?

4 comments
AuthorsOpenReview ↗

Reply to Reviewer xeF2 — Part 1

We thank the reviewer for the suggestions on baseline strength, the navigation mechanism, and the role of semantic beacons.

Q1. Why do the comparative experiments select lightweight methods prior to 2023 such as Seq2Seq, CMA, and LAG as baselines, instead of selecting models from the past two years with comparable parameter scales for comparison?

The original Seq2Seq-VG, CMA-VG, and LAG-VG baselines provide classical sequence-modeling and attention architectures under the VoLN task interface. Following the reviewer's suggestion, we added TravelUAV-VG, adapted from the OpenUAV implementation, to include a recent UAV navigation method using a large language model.

TravelUAV-VG replaces language instructions with goal images and uses visual history and constructed motion states to predict eight body-frame 3D waypoints. It and VoLN-MLLM use the same Vicuna-7B language backbone, training split, waypoint supervision, controller, target-confirmation procedure, and evaluation budget. The adaptation is described in Section 6.1.

Table 1 reports the following Test-Unseen results:

DifficultyMethodNE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
EasyTravelUAV-VG102.010.913.248.77.4
EasyVoLN-MLLM79.616.418.555.412.1
NormalTravelUAV-VG146.87.79.832.64.6
NormalVoLN-MLLM101.814.115.946.19.9
HardTravelUAV-VG202.03.14.615.91.6
HardVoLN-MLLM138.211.513.137.27.1

VoLN-MLLM performs better across all three difficulty levels, and its advantages in SR and nDTW widen from Easy to Hard. The comparison therefore extends the evidence beyond lightweight baselines to a UAV planner using the same language backbone.

We also added Current-only and No-Semantic ablations within VoLN-MLLM. These keep the language backbone fixed while evaluating the contributions of recent history and explicit semantic inputs. The full baseline results and ablations are provided in Tables 1 and 2.

AuthorsOpenReview ↗

Reply to Reviewer xeF2 — Part 2

Q2. For the VoLN-MLLM method, why is there no dedicated introduction to the implementation mechanism for achieving long-horizon navigation?

We thank the reviewer for suggesting a clearer explanation of the long-horizon navigation mechanism. We have clarified this mechanism in Section 5, fig. 3, and Appendices B and C. VoLN-MLLM performs long-horizon navigation through successive beacon-guided local planning updates.

VoLN defines a long-horizon navigation task, instantiated by the VoLN-UAV benchmark, and allows policies to use the complete observation and action history collected during an episode. The eight-observation window is a design choice of the reference baseline, used to bound the input sequence length and support efficient online planning.

The reference method samples observations and synchronized states every 2 s and maintains eight observations spanning 14 s. Every 8 s, it incorporates four new observations and predicts the next eight spatial waypoints. Execution continues during inference, and the returned prediction replaces the remaining waypoints of the previous plan.

Long-range guidance is acquired progressively as successive beacons enter view. The overlapping history retains a recently observed cue together with the motion and viewpoint changes that follow it. For example, a turning cue can remain available in the history after leaving the current view, supporting waypoint predictions while the maneuver continues. Subsequent beacons then provide guidance for the following route segments. Repeatedly updating local plans from newly encountered cues connects these segments over the full navigation episode.

We also added the Current-only ablation to evaluate the contribution of recent history. Current-only retains only the current visual, semantic, and constructed-state inputs during both training and testing, with all other settings unchanged. Table 2 reports:

SettingCT/s ↓EER/% ↓NE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
Full1.250.594.615.016.949.910.7
Current-only1.050.9134.87.810.243.85.3

Retaining recent history improves route adherence, goal-region entry, and navigation success, with a higher average cycle time. This supports the use of recent multimodal context to connect successive planning decisions in the reference method.

AuthorsOpenReview ↗

Reply to Reviewer xeF2 — Part 3

Q3. Why are there no experiments specifically evaluating the proposed beacons in the experimental section to verify the substitutive role of beacons for separate language instructions?

We thank the reviewer for requesting experiments that examine the role of semantic beacons more directly.

Active beacons provide local route guidance acquired through visual observation during execution, while goal images specify the destination. A route description supplies guidance before navigation; scene-embedded beacons become available as the agent encounters them and must be interpreted from the current viewpoint and recent history. Sections 3 and 4 clarify this distinction.

We added beacon-configuration experiments in Section 6.3 and Table 3, using the same trained checkpoint, paired Test-Unseen episodes, and runtime seeds, without retraining. Standard uses equal-probability left/right placement of active beacons. Left-only fixes active beacons on the left; No-Passive removes passive beacons while preserving the Standard active-beacon layout.

SettingActive-beacon placementNE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
StandardRandom side94.615.016.949.910.7
Left-onlyFixed left98.214.216.148.710.0
No-PassiveRandom side91.816.118.051.211.4

The slight performance decrease with Left-only indicates a modest influence of beacon geometry on navigation decisions. Removing passive beacons improves route adherence and goal-region entry under unchanged active guidance, showing the additional navigation difficulty introduced by competing cues.

Active beacons provide the route guidance that VoLN is designed to study. Removing them changes the task to image-goal exploration and no longer evaluates how the agent interprets and follows the designated visual guidance. We therefore retain active beacons in the controls examining beacon placement and competing cues.

Regarding the concern about following a predetermined route, reference trajectories determine beacon placement and provide training supervision. During inference, the planner receives neither the reference route nor future reference waypoints. It predicts waypoints from onboard observations, recent history, constructed motion states, and retrieved concepts. Test-Unseen evaluates this policy in held-out environments.

AuthorsOpenReview ↗

Reply to Reviewer xeF2 — Part 4

W1. Fixed semantic bank, spatial representation, and cue relevance

We thank the reviewer for highlighting the need to clarify the scope of the semantic bank and how retrieved information is used for navigation.

(1) Scope of the semantic bank

The fixed bank contains 64 descriptors and supplies explicit semantic inputs alongside aligned visual features. Image content is also represented through the visual features, while the descriptors returned by retrieval are limited to the bank. Test-Unseen evaluates navigation in unseen environments; it does not separately test recognition of concepts outside this vocabulary.

(2) Spatial representation and cue relevance

Retrieval compares a global image embedding with the descriptor embeddings. It does not partition individual beacons into spatial regions or assign active/passive labels. The similarity score measures image–descriptor correspondence.

The retrieved descriptor and its similarity embedding are stored with the visual and constructed-state tokens from the same observation. The planner combines these synchronized inputs over the recent history to predict waypoints. Section 5 and Appendix B now describe this information flow.

(3) Contribution of explicit semantic inputs

No-Semantic retains the aligned visual features and removes only the retrieved-descriptor tokens and similarity embeddings during both training and testing. The remaining settings are unchanged. Table 2 reports:

SettingCT/s ↓EER/% ↓NE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
Full1.250.594.615.016.949.910.7
No-Semantic1.401.1128.98.712.735.55.0

Removing explicit semantic inputs reduces route adherence, goal-region entry, and navigation success while preserving the aligned visual representation. This supports an additional benefit from retrieved semantic information beyond visual alignment.

(4) Retrieval on observations containing active beacons

We also evaluated whether the retrieved description matches a visible active beacon:

FramesCorrect matchesHit rate/%
77,86873,68394.63

A match is counted when the retrieved description corresponds to the attributes or instruction content of a visible active beacon. Descriptions matching only background scenes, landmarks, or passive beacons are excluded.

This evaluation measures retrieval of visible active-beacon information. The No-Semantic ablation evaluates the contribution of these semantic inputs to navigation performance.

Reviewer_7Pf2OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Vision-Only Long-Horizon Navigation (VoLN), which studies how an agent can complete long-range navigation from goal images, onboard visual observations, and proprioception, exploiting semantic cues encountered along the route without separate language route instructions. The authors position VoLN as an intermediate setting between image-goal navigation and instruction-following navigation, and instantiate it for continuous 3D UAV flight with the VoLN-UAV benchmark, which contains 7,210 episodes from 17 simulated environments. Active beacons tied to the reference route provide guidance, while passive beacons introduce competing cues; together they are intended to test cross-view goal matching, cue selection, and sustained closed-loop execution.

The authors further propose a reference method, VoLN-MLLM. It first aligns DINOv3 visual features to the CLIP embedding space and retrieves relevant concepts from a fixed semantic bank of 64 descriptors. The goal images, the eight most recent observations, the retrieved semantics, and the proprioceptive state are then fed to a LoRA-adapted Vicuna-7B, which predicts eight body-frame waypoints and a stop signal. In the experiments, the method improves consistently over all compared baselines on every metric. The authors also explicitly acknowledge that absolute success rates remain low, and list history integration, cue-relevance assessment, and recovery from accumulated errors as future directions.

Critical strengths

1. The task has a clear robotics motivation, and the setting is of value.

Navigating a UAV with local environmental cues while the goal remains out of view for long periods, and while viewpoint and scale change continuously, is a meaningful research problem. The paper combines visual goal specification, en-route semantic cues, and continuous 3D motion, so that the task jointly involves perception, goal matching, decision-making, and execution. The paper also clearly explains how the setting relates to ImageNav and VLN, and draws a clear input boundary: no separate language route instructions are provided.

2. The benchmark is constructed systematically and includes a clear unseen-environment evaluation.

The paper describes the full pipeline: reference-trajectory collection, decision-point selection, active and passive beacon generation, multimodal data recording, difficulty stratification, and data splitting. Test-Unseen uses five environments outside the training pool, drawn from a held-out scene source, and the appendix describes checks for trajectory overlap and scene-source separation. Both the beacon generation rules and the random seeds are deterministic, which provides a basis for controlled follow-up experiments.

The benchmark covers urban, forest, mountain, tunnel, and industrial environments, among others. These environments differ in layout, visibility, altitude change, and landmark density, and include both open areas and constrained passages such as tunnels and industrial corridors, which helps assess methods under diverse scene conditions.

3. Within the current comparison scope, the gains are clear and the input conditions are well controlled.

All learned baselines share the observation history, goal views, proprioceptive state, retrieved semantics, visual encoder, waypoint action interface, and supervision targets. These controls improve the interpretability of the comparison.

According to Table 1, the success rates on Test-Unseen are:

DifficultyLAG-VGVoLN-MLLMAbsolute gain
Easy2.3%7.4%+5.1 pp
Normal1.2%4.5%+3.3 pp
Hard0.4%1.8%+1.4 pp

The proposed method also leads consistently on NE, OSR, and SPL, indicating that the SR improvement is not an isolated effect. It leads on nDTW as well, but the definition of that metric needs clarification.

4. Method and training details are well documented, supporting reproduction and follow-up comparison.

The appendix specifies:

  • the visual adapter architecture, the semantic vocabulary, and the retrieval procedure;
  • the composition of the input sequence, the LoRA configuration, and the prediction-head architecture;
  • the training hyperparameters and the procedure for selecting the stop threshold. An anonymized code link is also provided. The authors report five independent runs together with ranges of standard deviations, and carefully delimit what the low success rates and the preliminary physical demonstration can support. This improves the transparency of the work and provides a foundation for reproduction and future comparisons.
Critical weaknesses

1. The empirical characterization of what the benchmark measures remains insufficient.

The paper emphasizes that the agent must interpret en-route cues in light of its goal and history, but the experiments do not separate the roles of the goal images, recent observations, and semantic cues at different stages. For example, the current results cannot tell whether the goal images mainly inform route choices along the way or mainly serve terminal recognition and stopping. The eight-frame history may provide short-term motion information, local scene integration, or retention of earlier cues, but the paper contains no ablation that shortens or removes the history, so its actual role cannot be determined.

The authors describe VoLN-MLLM as a reference baseline (Sec. 6.1). It uses only a fixed window of eight observations rather than the full interaction history hth_t on which the policy π(⋅∣ht,V)\pi(\cdot \mid h_t, \mathcal{V}) is defined in Sec. 3, and the authors list history integration as future work. This positioning is reasonable in itself. Nevertheless, a modest set of input ablations, or an analysis targeted at key decision points, would help clarify which capabilities the benchmark mainly tests.

The nature of goal matching also needs clarification. Appendix A.3 states that a target asset, scaled by a factor of 1.6, is placed 40 m forward of the terminal reference pose, while the goal set V={oT−2,oT−1,oT}\mathcal{V} = \lbrace o_{T-2}, o_{T-1}, o_T \rbrace consists of the last three frames of the reference trajectory. In Fig. 4 and Fig. 8(b)(d), some goal views are dominated by a salient marker (e.g., a glowing object or a "DESCEND DOWN" sign). The paper does not state whether this asset appears in the goal views. If it does in most episodes, goal matching may rely mainly on recognizing this marker rather than on cross-view scene matching.

In addition, difficulty is stratified by reference path length, but the number and placement parameters of the active beacons change with difficulty at the same time. Performance differences across difficulty levels therefore reflect several factors jointly and cannot isolate the effects of path length, decision complexity, or history dependence.

2. Non-semantic differences between active and passive beacons may confound the interpretation of "cue-relevance judgment."

The appendix specifies the generation rules for both beacon types in detail, and the two differ systematically in anchor selection, positional offsets, and orientation.

  • Active beacons:
    • They are anchored at reference step max⁡(0,i−2)\max(0, i-2), i.e., two steps before the selected decision point ii.
    • All lateral offset intervals are negative — [−5,−2][-5, -2], [−7,−3][-7, -3], and [−10,−4][-10, -4] m for Easy, Normal, and Hard, respectively — so active beacons always lie on the same side of the route.
    • The vertical offsets are 88, 1010, and 1212 m downward under the NED convention (except in the BrushifyUrban scene).
    • The yaw is offset by 90∘90^\circ from the anchor heading.
  • Passive beacons:
    • Anchors are sampled from the environment's reference states.
    • The forward, lateral, and vertical offsets are drawn uniformly from [−20,40][-20, 40] m, [−30,30][-30, 30] m, and [−5,5][-5, 5] m, respectively.
    • The yaw equals the anchor yaw plus 180∘180^\circ, and the positions are fixed across episodes within an environment. As a result, the two beacon types may be distinguishable from their left/right position in the image, their relative height, or their orientation alone. The paper does not analyze the distributions of distance, projected size, image position, or viewing angle of the beacons that are actually visible. The current results neither confirm that the model exploits such geometric shortcuts nor rule this out. If some relevant beacons can be identified mainly through non-semantic features, interpreting the performance gains as goal-directed semantic cue selection requires more direct evidence.

I suggest first reporting visibility and geometric statistics for both beacon types in the actual observations; if the differences are substantial, a limited controlled experiment could then test whether the conclusions still hold. Such analyses would strengthen the benchmark's interpretive power with respect to semantic distraction and cue selection.

3. The baselines and ablations are not yet sufficient to isolate the main sources of the proposed method's gains.

The learned baselines are Seq2Seq-VG, CMA-VG, and LAG-VG. Although the inputs and the action interface are well controlled, these baselines still differ substantially from the Vicuna-7B-based method in model capacity, pretrained knowledge, and planning architecture. Adding a directly relevant image-goal or visual-waypoint Transformer baseline, adapted to the paper's sensor restrictions and 3D action interface, would make the comparison more convincing.

The three ablations in Table 2 show that alignment, planner adaptation, and the choice of visual encoder all affect the full system, but they do not directly isolate the contribution of the explicit semantic tokens. In particular, No-Align changes both the visual representation and its consistency with the retrieval space, so it cannot substitute for a control that keeps the aligned visual features and removes only the semantic inputs. Moreover, semantic retrieval uses only the global aligned embedding of the current frame (Appendix B), and the paper does not report retrieval accuracy on frames containing beacons, which makes it hard to judge whether the semantic tokens actually carry beacon information.

The training setup of No-Align is also ambiguous. The main text states that it retains the dimensional projection but removes CLIP-teacher supervision, whereas the appendix specifies that the main method's adapter is frozen during planner training. The authors should state whether the projection layer in No-Align is frozen and, if it is trainable, which loss optimizes it. This directly affects how the ablation result should be interpreted.

4. Several definitions of state inputs, execution timing, and evaluation metrics need to be provided.

State inputs. The nine-dimensional state in Appendix B consists of body-frame linear velocity, angular velocity, and relative displacement, but the paper does not specify the reference time or origin of the relative displacement, or how this quantity is obtained in simulation and on the physical platform. This definition is also inconsistent with the proprioception described in Sec. 3, Appendix A.1, and Figs. 1 and 3 (IMU, altitude, velocity, and orientation): the nine-dimensional state contains no explicit altitude or orientation. If the relative displacement is measured from the episode start, it effectively provides odometric position (the Odom annotation in Fig. 2(g) also includes position), and the authors should explain how this is consistent with the stated exclusion of GPS and world-frame position.

Execution timing. The paper states that the policy acquires a new observation and replans only after executing the full predicted waypoint segment, and it gives the Δt=2\Delta t = 2 s sampling interval of the demonstrations as well as the 2 s planning budget per decision. However, a computational budget and physical execution time are different quantities. The time parameterization of the waypoints, the actual execution duration of a segment, and how the sampling interval of the online eight-frame history corresponds to the demonstration data all remain unclear. These details matter for understanding the closed-loop feedback rate and the accumulation of errors.

EER and CT. EER aggregates time-budget violations and invalid outputs, but no criterion for an "invalid output" is given. Since the trajectory head is a regression MLP (d→d→24d \to d \to 24; Table 5), text-parsing failures cannot occur, which makes this criterion all the more necessary. No-LoRA has a CT of 1.45 s, well below the 2 s budget, so its 5.8% EER most likely stems mainly from invalid outputs; the criterion therefore bears directly on the conclusion that planner adaptation improves output reliability. Two CT differences are also unexplained:

  • CLIP-Input and DINOv3 are both ViT-B/16 backbones operating on 224×224224 \times 224 inputs, yet CT increases from 1.42 s to 1.98 s, close to the 2 s budget.
  • No-Align has the same architecture as the full model, yet its CT is 1.36 s; the gap exceeds the reported CT standard deviation (below 0.03 s).

Evaluation metrics. The paper does not give the distance threshold or the normalization used for nDTW. In Table 1, the random policy's nDTW on Test-Unseen (30.1/22.7/15.1) is higher than that of both Seq2Seq-VG (28.9/21.4/13.0) and LAG-VG (28.1/20.5/14.0) at all three difficulty levels. Under the definition of [1],

nDTW(R,Q)=exp⁡(−DTW(R,Q)∣R∣⋅dth),\mathrm{nDTW}(R, Q) = \exp\left( -\frac{\mathrm{DTW}(R, Q)}{|R| \cdot d_{\mathrm{th}}} \right),

with dthd_{\mathrm{th}} set to the success radius ϵ=4\epsilon = 4 m, a random policy whose terminal error is about 270 m should obtain an nDTW close to zero. Until the computation is clarified and this phenomenon is explained, nDTW differences should not serve as primary evidence of improved trajectory quality.

[1]. Ilharco G, Jain V, Ku A, et al. General evaluation for instruction conditioned navigation using dynamic time warping[J]. arXiv preprint arXiv:1907.05446, 2019.

5. The analysis of failure sources, qualitative evidence, and statistical stability could be strengthened.

The paper reports SR and OSR and presents a qualitative case in which the agent is misled by passive beacons, but it lacks a quantitative breakdown of failure types. In Table 2, the overall Test-Unseen SR is 5.7% and the OSR is 11.8%. On the one hand, some episodes are lost between entering the goal region and stopping successfully; on the other hand, close to 90% of trajectories never enter the goal region at all (1−0.118=0.8821 - 0.118 = 0.882). The current metrics cannot separate the effects of cue selection, goal localization, trajectory tracking, premature stopping, and decision timing.

A coarse failure taxonomy, together with a diagnostic that replays the reference waypoints through the same controller, would help identify which components future work should prioritize. When interpreting the SR–OSR gap, one should also note that the policy decides whether to stop only after each waypoint segment has been executed, so a trajectory may pass through the goal region between two decision steps.

The provenance of the qualitative evidence also needs clarification. The caption of Fig. 8 states that the upper rows show recorded reference routes and that the lower rows illustrate deviations, and Fig. 4 does not state which policy produced its two paths. Moreover, the frame in Fig. 4 labeled "MISLED BY PASSIVE DISTRACTOR" also contains the "FLY OVERHEAD ↑" sign that the upper row labels as Active Beacon 2, so the figure does not make clear which type of beacon caused the error. If these cases are not actual VoLN-MLLM rollouts, their value for illustrating failure modes is limited.

Regarding statistics, the authors report overall ranges of standard deviations over five runs but do not state what varies across the runs: Appendix C sets both the construction seed and the training seed of the reference configuration to 7, and the construction seed also determines the beacon layout. The paper also does not report the number of episodes per split and difficulty level. The full-model row of Table 2 matches the 52/36/12-weighted average of the Test-Unseen results in Table 1; for SR, for example,

0.52×7.4+0.36×4.5+0.12×1.8≈5.7.0.52 \times 7.4 + 0.36 \times 4.5 + 0.12 \times 1.8 \approx 5.7 .

This suggests that the difficulty mix of Test-Unseen is close to the aggregate one, so that TU-Hard contains roughly 0.12×1,081≈1300.12 \times 1{,}081 \approx 130 episodes. On this estimate, the 1.8% SR of VoLN-MLLM corresponds to about 0.018×130≈20.018 \times 130 \approx 2 successes per run, and the 0.4% SR of LAG-VG to fewer than one (0.004×130≈0.50.004 \times 130 \approx 0.5). The ranking of methods on the Hard subset therefore rests on single-digit success counts.

Furthermore, Validation-Seen is used both for checkpoint selection and for calibrating the stop threshold τ\tau, and it shares passive-beacon layouts with the training set. Its results should therefore be regarded as an optimistic reference, with Test-Unseen being the only fully held-out evaluation. I suggest reporting the number of episodes, the number of successes, and the corresponding variability for each unseen environment and each difficulty subset. These are supplementary suggestions and are less important than the capability-attribution and interface-definition issues raised above.

Questions for authors

Q1. At which stages do the goal images and the observation history play their roles?

The paper explains how the goal views and the eight-frame history are organized in the input sequence (Appendix B; for the online sampling interval, see Q4). Could the authors use either of the following to separate their contributions to en-route route selection and to terminal confirmation/stopping?

  • a small set of input ablations applied consistently at training and test time;
  • an analysis of decision points chosen via the decision-point (beacon) indices stored with each episode, focusing on those near competing passive beacons. Such controls should keep the goal and the beacon guidance consistent, to avoid introducing additional task conflicts by arbitrarily replacing the goal.

In addition, does the target asset described in Appendix A.3 (placed 40 m forward of the terminal pose and scaled by 1.6) appear in the goal views? If so, in what fraction of episodes? Could SR be reported separately depending on whether the asset is visible?

Q2. Can active and passive beacons be distinguished by non-semantic features?

Appendix A.3 and Table 3 indicate that the generation rules of the two beacon types differ systematically in lateral position, relative height, and orientation (see W2). Could the authors provide statistics on the distance, projected size, image position, and viewing angle of the beacons that are actually visible? If the differences are substantial, could a limited controlled experiment (e.g., re-evaluating with both types drawn from the same placement distribution) show how much the current results depend on them?

Q3. How large is the independent benefit of explicit semantic retrieval beyond the aligned visual representation?

Could the authors add a control that keeps the aligned visual features and removes only the semantic tokens, and report retrieval accuracy on frames containing beacons? Please also specify the initialization, freezing status, and training objective of the projection layer in No-Align. A directly relevant image-goal or visual-waypoint planning baseline would further help determine whether the gains come from the planner's capacity or from the specific components introduced in this paper.

Q4. How are the state, the execution timing, and the evaluation metrics defined?

  1. To which time step or origin is the relative displacement in the nine-dimensional state referenced? How is it obtained in simulation and on the physical platform? How does it relate to the altitude and orientation signals that Sec. 3 lists as proprioception?
  2. What is the time parameterization of the waypoints, or the actual execution duration of a segment? What is the sampling interval of the online eight-frame history, and how does it correspond to the demonstrations sampled at Δt=2\Delta t = 2 s?
  3. What is the criterion for an "invalid output" in EER? Could the authors report time-budget violations and invalid outputs separately, state which stages CT includes, and explain the CT differences of CLIP-Input and No-Align?
  4. Please specify the distance threshold dthd_{\mathrm{th}} and the normalization used for nDTW, and explain why the random policy's nDTW on Test-Unseen exceeds that of Seq2Seq-VG and LAG-VG.

Q5. Can the sources of failure be further quantified, and can the stability of the results be clarified?

  1. Beyond the existing qualitative cases and SR/OSR, could the authors report the proportions of the main failure types? Where feasible, could they also report the results of replaying the reference waypoints through the same controller, to separate the effects of high-level planning, trajectory execution, and stopping time?
  2. Are the cases shown in Fig. 4 and Fig. 8 actual VoLN-MLLM rollouts?
  3. Which sources of randomness vary across the five runs?
  4. Could the authors report the number of episodes, the number of successes, and the variability for each unseen environment and each difficulty subset?
6 comments
AuthorsOpenReview ↗

Reply to Reviewer 7Pf2 — Part 1A (Q1)

We thank the reviewer for the suggestions on the task definition, the roles of different inputs, and beacon configuration. Below, we address Q1 and Q2, followed by the remaining points in W1 concerning difficulty stratification.

Q1. At which stages do the goal images and the observation history play their roles?

(1) Target placement and success criteria

The target asset is placed on the ground ahead of the reference trajectory's endpoint, along its final heading, with a scale factor of 1.61.6. The reference endpoint serves as an observation location for the target. The original criterion used a 44 m neighborhood around that endpoint to approximate the target discovery region. However, the target can be observed and confirmed from multiple locations. We therefore revised the criterion to combine target confirmation with a spatial condition: an accepted detection interrupts waypoint execution and terminates the episode; success requires correct confirmation of the designated target and termination within 4040 m of the target asset's 3D center.

Accordingly, SR requires correct target confirmation and termination within this region, whereas OSR only requires the executed trajectory to enter the same region, without requiring confirmation or termination. Sections 3, 4, and 6.1 now use these consistent definitions. All methods share the same target-confirmation procedure and success criteria. We recomputed the navigation metrics for the original and revised methods accordingly.

(2) Roles of goal images and observation history

Goal views provide an appearance reference for target recognition and stopping. We compared the original method, which feeds goal views into waypoint planning, with the revised method, which separates waypoint planning from target confirmation. The Test-Unseen results are:

MethodGoal views in waypoint plannerCT/s ↓EER/% ↓NE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
OriginalYes1.420.5119.010.014.045.87.0
RevisedNo1.250.594.615.016.949.910.7

The revised method achieves better overall performance. In this comparison, including goal views in waypoint planning did not yield a performance advantage. Goal views depict the scene near the destination, whereas planning along the route requires the current local environment, motion state, and route beacons. This difference in scene content may interfere with local decisions.

OSR increases by 2.9 pp and SR by 5.0 pp, while the OSR–SR gap narrows from 4.0 to 1.9 pp. Thus, the improvement reflects both more frequent goal-region entry and fewer episodes that enter the region but fail to complete target confirmation and successful termination.

To evaluate the role of history, we added the Current-only ablation. During both training and testing, Current-only retains only the visual, semantic, and constructed-state inputs from the current observation. Full retains the synchronized history of these three inputs over the eight most recent observations, with all other settings unchanged. Table 2 reports the following Test-Unseen results:

SettingCT/s ↓EER/% ↓NE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
Full1.250.594.615.016.949.910.7
Current-only1.050.9134.87.810.243.85.3

The performance drop with Current-only supports the benefit of recent history for successive waypoint predictions. For example, a directional beacon may leave the camera's field of view after the UAV begins turning, while the turn is still incomplete. With current inputs alone, the planner no longer has access to the earlier beacon observation. An eight-observation history retains that cue together with subsequent visual changes and motion states. This provides context for subsequent waypoint predictions and may help maintain consistency across planning updates.

Based on these comparisons and ablations, we adopt the revised method as the reference baseline. Its waypoint planner uses eight observations of visual, semantic, and constructed-state history for local decisions, while a separate target-confirmation branch uses the goal images. This branch employs frozen YOLOE to perform prompt-free detection in the three goal views. The instance ID of the preplaced target asset is used to select valid target regions, which are encoded as visual prompts for the same target. During navigation, the branch checks newly sampled RGB observations every 22 s. A detection satisfying the configured confidence threshold interrupts waypoint execution and terminates the episode. Section 5 and Fig. 3 now consistently describe these information flows.

AuthorsOpenReview ↗

Reply to Reviewer 7Pf2 — Part 1B (Q1 continued; Q2 and W1)

Q1 (continued)

(3) Target visibility in goal views

Target visibility is a condition of dataset construction. The goal set consists of the final three observations of the reference trajectory, and at least one contains the designated target asset in every episode. Visibility is therefore 100% at the episode level; the target need not appear in all three views.

The Test-Unseen breakdown is:

SubsetEpisodesProportion/%SR/%
Target visible in at least one goal view1,08110015.0
Target absent from all goal views00N/A

The visible subset is therefore the complete test set. Target confirmation refers to recognizing, in onboard observations, the asset specified by the goal views. Section 4 and Appendix A.3 now clarify this construction condition and target definition.


Q2. Can active and passive beacons be distinguished by non-semantic features?

We thank the reviewer for identifying this issue. The original configuration placed active beacons exclusively on the left. We measured the camera distance, projected area, image position, and viewing angle of visible active and passive beacons in that configuration. Except for observation counts and maximum distances, the following values are means ± standard deviations:

MeasureActive beaconsPassive beacons
Visible observations77,868216,347
Camera distance/m18.6 ± 11.720.1 ± 12.9
Maximum distance/m46.348.7
Projected area/%2.7 ± 2.12.2 ± 1.9
Horizontal position u/Wu/W0.38 ± 0.200.51 ± 0.23
Vertical position v/Hv/H0.48 ± 0.180.47 ± 0.20
Viewing angle/°26.8 ± 16.429.5 ± 18.2

These statistics show a leftward image-position bias for active beacons in the original layout. To reduce this bias, we adopted equal-probability left/right placement as the standard configuration in the revision. Forward and vertical placement and the passive-beacon layout retain their existing construction rules. Appendix A.3 and Table 4 have been updated accordingly.

The horizontal-position statistics after this change are:

Beacon typeHorizontal position u/Wu/W
Active0.49 ± 0.21
Passive0.51 ± 0.23

We further evaluated three beacon configurations using the same final checkpoint, paired Test-Unseen episodes, and runtime seeds, without retraining. Left-only fixes active beacons on the left; No-Passive removes passive beacons. All other layout settings remain unchanged in each comparison. Table 3 reports:

SettingActive-beacon placementNE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
StandardRandom side94.615.016.949.910.7
Left-onlyFixed left98.214.216.148.710.0
No-PassiveRandom side91.816.118.051.211.4

The slight performance decrease with Left-only indicates a modest effect of beacon placement geometry on navigation decisions. With active guidance unchanged, No-Passive improves route adherence and goal-region entry, indicating that passive cues introduce additional navigation difficulty.

These comparisons evaluate the effects of active-beacon lateral placement and passive beacons on navigation performance. The settings, results, and analysis are included in Section 6.3 and Table 3.


Additional response to W1: Benchmark capabilities and difficulty stratification

The roles of goal images and recent history, together with the target-recognition setting, are addressed in Q1 above. VoLN-MLLM uses a fixed window of eight observations as the history input of the reference baseline, and Current-only evaluates the contribution of this recent multimodal context. The contribution of explicit semantic inputs is addressed further in Q3 of Part 2.

Regarding difficulty stratification, Section 4 and Appendix A.3 now clarify its relationship to beacon configuration. Episodes are first grouped by reference-path length and then assigned the corresponding active-beacon counts and placement parameters:

DifficultyReference-path lengthActive beaconsForward offset/mVertical offset/m (NED)
EasyBelow 300 m3328
Normal300 m to below 450 m44510
HardAt least 450 m55512

The revised lateral placement samples either side with equal probability from symmetric intervals, while forward placement retains uniform jitter of ±3\pm 3 m. Longer routes therefore contain more beacon-guided decision locations and changes in how cues are presented. Cross-difficulty results reflect the combined effects of navigation distance, the number of guidance points, and observation conditions. The role of history is evaluated separately through the Current-only ablation in Q1, with consistent input settings during training and testing.

AuthorsOpenReview ↗

Reply to Reviewer 7Pf2 — Part 2

Q3. How large is the independent benefit of explicit semantic retrieval beyond the aligned visual representation?

We added the No-Semantic ablation and the TravelUAV-VG baseline, and clarified the training settings of the alignment adapter and planner in Appendix B.

(1) Independent benefit of explicit semantic inputs

No-Semantic retains the aligned visual features and removes only the retrieved-descriptor tokens and their similarity embeddings. The same input configuration is applied during training and testing. The visual encoder, alignment module, eight-observation visual and constructed-state history, planner, training budget, and target-confirmation procedure remain unchanged. The Test-Unseen results are:

SettingCT/s ↓EER/% ↓NE/m ↓SR/% ↑OSR/% ↑nDTW/% ↑SPL/% ↑
Full1.250.594.615.016.949.910.7
No-Semantic1.401.1128.98.712.735.55.0

Removing explicit semantic inputs reduces navigation performance while preserving the aligned visual representation. This supports an additional benefit from retrieved semantic information beyond visual alignment. The settings and results are included in Section 6.3, Table 2, and Appendix B.

(2) Retrieval evaluation on observations containing active beacons

For each observation, the method retrieves from the semantic bank using the aligned global image embedding. The retrieved description and its similarity embedding are stored with the visual and state tokens from the same observation. We evaluated whether the retrieved description matches a visible active beacon:

Observations containing active beaconsCorrect matchesActive-beacon concept hit rate/%
77,86873,68394.63

An observation is counted as correct when the retrieved description matches the attributes or instruction content of a currently visible active beacon. Each observation is counted once. Descriptions matching only background scenes, landmarks, or passive beacons do not count as active-beacon concept hits.

This evaluation measures whether retrieval captures information about visible active beacons; No-Semantic evaluates the contribution of these inputs to navigation.

(3) Optimization settings for No-Align

No-Align retains the same adapter architecture as Full but skips the first-stage visual–semantic alignment training. Full uses the adapter trained with the CLIP teacher, whereas No-Align uses PyTorch's default parameter initialization. In both settings, the adapter remains frozen throughout waypoint-planner training, while the downstream input projectors, structural markers, waypoint head, and LoRA parameters are trained with the waypoint regression loss. This comparison evaluates the contribution of alignment pretraining to navigation performance. Appendix B specifies these training settings.

(4) A directly related planning baseline

Table 1 includes TravelUAV-VG, adapted from the OpenUAV implementation. We replace its language condition with goal images, adapt its inputs to visual history and constructed states, and predict eight body-frame 3D waypoints for the shared controller. TravelUAV-VG and VoLN-MLLM use the same Vicuna-7B language backbone and share data splits, waypoint supervision, target confirmation, and evaluation budgets.

The Test-Unseen results are:

DifficultyMethodNE/m ↓SR/% ↑SPL/% ↑
EasyTravelUAV-VG102.010.97.4
EasyVoLN-MLLM79.616.412.1
NormalTravelUAV-VG146.87.74.6
NormalVoLN-MLLM101.814.19.9
HardTravelUAV-VG202.03.11.6
HardVoLN-MLLM138.211.57.1

This baseline provides a direct comparison with a recent UAV planning method.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their careful reading and constructive feedback. Their comments helped us clarify the task formulation, strengthen the experimental comparisons, and improve the analysis. The revised manuscript incorporates the following changes.

1. Stronger comparisons and controlled experiments.

We added TravelUAV-VG as an additional aerial navigation baseline under the shared VoLN interface. We also added controlled ablations of observation history and explicit semantic retrieval. The beacon experiments now examine lateral placement, passive distractors, and local active-guidance removal. Following the follow-up suggestion, we evaluated One-Active-Masked, which masks the first active beacon along each reference route, on all 1,081 Test-Unseen episodes using the same five runtime seeds.

2. Clearer task, method, and evaluation definitions.

We revised the reference method to separate waypoint planning from visual target confirmation and clarified how goal images are used. We specified the construction of the nine-dimensional motion state, the reference frame of predicted waypoints, and the timing of observation sampling, asynchronous planning, and target confirmation. We also clarified the success criteria and metric definitions, including target-confirmed termination within the 40 m goal region, and applied the shared evaluation protocol across methods.

3. Expanded failure and runtime analysis.

We added quantitative failure-event categories and reported episode counts, successful executions, and repeated-run variability across unseen environments and difficulty levels. We also separated timeout and invalid-output events in the execution-error analysis.

The point-by-point responses provide the corresponding results and manuscript locations. We appreciate the reviewers’ suggestions, which have helped us make the benchmark and reference method more clearly specified and the experimental findings easier to assess.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.