acceptodds
Under review as a conference paper at ICLR 2027

AdaFly: Adaptive Temporal Memory for Onboard Aerial Vision-Language Navigation

Abstract

Many aerial VLN systems rely on large multimodal models, making fully onboard deployment difficult on resource-constrained UAVs. Therefore, we propose AdaFly, a lightweight onboard aerial VLN approach that combines a compact 545M-parameter policy with adaptive temporal perception. Our design is motivated by a controlled analysis of 490,090 AeroBrain decisions, which reveals that the value of additional visual history is strongly stage-dependent. Extending history from two to four observations reduces action prediction error by 12.29% near upcoming turns and stops, but increases it by 2.72% during non-critical motion. AdaFly exploits this asymmetry with two components. An Adaptive Temporal History Selector determines whether additional older observations should be encoded before action-chunk generation, while a Context-Conditioned Temporal Memory Compressor condenses the selected history into a fixed set of task-relevant memory tokens. On AeroBrain, AdaFly achieves a 44.94% navigation success rate, representing a 69.0% relative improvement over a representative high-performing aerial VLN baseline despite using a substantially smaller model. The proposed memory compressor reduces the four-frame historical representation from 144 visual tokens to only 16 memory tokens, an 88.9% reduction, while improving navigation performance over uncompressed history. We further deploy the complete system fully onboard a DJI Matrice 4TD with a Manifold 3 (Jetson Orin NX) and validate it across 40 real-world flight trajectories without offboard policy inference.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Unmanned Aerial Vehicles, Temporal Visual Memory, Onboard Deployment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Wy2NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AdaFly targets fully onboard UAV vision-language navigation with a 545M-parameter policy. It selects between two and four historical frames and compresses them into 16 memory tokens using the current navigation context. The paper evaluates the method on AeroBrain with component ablations, and reports approximately 270 ms inference on a Jetson Orin NX and 40 real-world flight trials.

Critical strengths

1: Fully onboard navigation is a useful practical target. The reported deployment includes inference latency and memory usage, providing evidence that the compact policy can run within onboard computing limits. 2: The history-utility analysis and budget-matched random-selection baseline support the idea that when additional history is used matters. The compression ablation also shows benefits from conditioning memory on the current task.

Critical weaknesses

1: The contribution seems incremental, and the benefit of adaptive selection is modest. The method combines a turn/stop predictor for history selection with context-conditioned attention for compression. Both are fairly standard, and the new insight beyond their combination is unclear. Compared with fixed four-frame history, success increases from 43.22% to 44.94%, while latency drops from 297.24 ms to 268.44 ms. The practical importance of adding the selector needs stronger support.

2: The real-flight evaluation needs more detail and stronger comparisons. The supplementary video shows real drone operation, but the paper says little about the task difficulty, success criteria, interventions, and failures across the 40 trials. This makes the reported 30% success rate difficult to interpret. Without a fixed-history comparison, the benefit of adaptive memory in real flight also remains unclear. Missing training and execution details further limit reproducibility.

Questions for authors

1: What is new beyond combining history selection and compression? Both components use fairly standard designs. Please clarify what technical insight the method adds beyond existing approaches, and which experiments support that contribution.

2: What do the real-flight experiments establish? The supplementary video shows real drone operation, but the evaluation remains briefly described. How were the 40 tasks selected, what counted as success, and how were interventions handled? Please provide the task breakdown and main failure cases to help interpret the approximately 30% success rate. As additional validation, a fixed-history flight comparison would help show the method’s practical benefit.

3: How important is adaptive selection in practice? Compared with fixed four-frame history, success increases from 43.22% to 44.94%, while latency drops from 297.24 ms to 268.44 ms. What practical advantage do these gains provide that justifies the additional selector? Also, why does fixed H4 with conditioned compression achieve 45.03% in Table 4(b), compared with 43.22% in Table 4(a)? Please explain any differences in training or evaluation.

4: Can the missing implementation details be provided? Please specify the training data and splits, training objectives and settings, turn/stop label definitions, selection thresholds, and history sampling intervals. The real-flight action-chunk execution and handling of human intervention also need clarification. These details are needed to reproduce the results and assess whether the comparisons use consistent settings.

Reviewer_eAMHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AdaFly is a lightweight aerial vision-language navigation system for fully onboard inference. It uses an Adaptive Temporal History Selector (ATHS) to choose how much visual history to retain and a Context-Conditioned Temporal Memory Compressor (CTMC) to represent past observations efficiently. The authors evaluate the method on AeroBrain, report a 44.94% overall success rate with a 545M-parameter policy, and demonstrate deployment on a physical UAV across 40 flight trajectories.

Critical strengths
  1. The task-dependent treatment of temporal history is well motivated. The analysis of decisions near turns and stops provides useful evidence that additional history is not uniformly beneficial, which supports the proposed adaptive design.
  2. The work considers deployment constraints explicitly. Combining benchmark evaluation with fully onboard execution and runtime measurements makes the system contribution more convincing than simulation-only results.
Critical weaknesses
  1. The gains do not transfer consistently to unseen environments. Table 1 reports 49.63% SR on seen scenes but 26.89% on unseen scenes, compared with 27.67% for AeroVLA on the latter. Thus, the overall advantage is driven mainly by seen environments, which limits the generalization claim.
  2. The gain from adaptive history allocation over a fixed-history configuration is relatively modest. In Table 4(a), ATHS increases SR from 43.22% (fixed H4) to 44.94%. Without variability estimates or a more focused analysis of when ATHS selects the correct history length, it is difficult to assess the robustness of this specific component.
Questions for authors
  1. What types of failures account for the relatively low unseen-scene SR, and does ATHS behave differently between seen and unseen environments?
  2. Can the authors report variability across runs or selector accuracy specifically around turn-critical and stop-critical decisions, to clarify the reliability of the 1.72-point gain?
Reviewer_sXDuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AdaFly, a lightweight (545M-parameter) onboard aerial vision-language navigation system. The key motivation is a large-scale paired analysis (490,090 decisions) showing that the utility of extended visual history is strongly decision-dependent: extending history from 2 to 4 observations reduces action-prediction error by 12.29% near maneuver-critical moments (turns/stops) but increases it by 2.72% during non-critical motion. Building on this, AdaFly combines (i) an Adaptive Temporal History Selector (ATHS) that gates between 2- and 4-frame history budgets based on predicted turn/stop criticality, and (ii) a Context-Conditioned Temporal Memory Compressor (CTMC) that compresses the selected history from 144 visual tokens into 16 memory tokens using context-conditioned learned queries. On the AeroBrain benchmark, AdaFly reports 44.94% SR (a 69.0% relative improvement over the strongest baseline), an 88.9% token reduction, and full onboard deployment on a DJI Matrice 4TD with a Jetson Orin NX, validated on 40 real-world flight trials at ~30% success.

Critical strengths
  1. Well-grounded motivation with rare analytical scale. The paired-decision analysis (identical checkpoint, observation, instruction, state, and flow noise, varying only the history budget) over 490k decisions is a clean, convincing quantification of decision-dependent temporal utility, and it directly and honestly drives the method design.
  2. Careful experimental controls. The Random 2/4 baseline matches the average history budget of ATHS (2.82 frames), ruling out the confound that ATHS merely wins by using less history; the compression ablation (Raw H4 26.78% -> Unconditional CTMC 40.25% -> Conditional CTMC 45.03% SR) cleanly separates the contribution of compression itself from context conditioning.
  3. Unusually complete deployment evidence. Mean/P95 onboard latency, memory footprint, FP16 precision, a 20 Hz flight-control loop via asynchronous chunk execution, and 40 fully onboard real-world flights with a per-step latency/history-budget timeline (Fig. 3). Fully onboard execution without offboard inference is a genuine and rare contribution for aerial VLN.
Critical weaknesses
  1. The benchmark and two central baselines are uncited, breaking verifiability. AeroBrain (the evaluation benchmark, referenced 7 times) does not appear in the 17-entry reference list, and neither do OmniNav, the denominator of the headline 69.0% relative improvement, nor DualVLN, a main comparison row in Tables 1-2. Without citations, the protocol, splits, and provenance of all baseline numbers are untraceable. This undermines the soundness of every quantitative claim in the paper.
  2. Tables 4(a) and 4(b) are internally inconsistent, leaving ATHS's net contribution ambiguous. Section 4.3 states Table 4(b) isolates compression "under a fixed H4 history budget," yet Raw H4 scores only 26.78% SR there while Fixed H4 in Table 4(a) reaches 43.22%. The paper never states whether the Table 4(a) variants include CTMC, and if so which variant. If Fixed H4 in 4(a) already uses the conditioned CTMC, then ATHS (44.94%) is actually worse than fixed H4 + conditioned CTMC (45.03%), meaning the adaptive selector contributes no SR gain, only ~44 ms/decision latency savings, and the claim "ATHS improves SR over Fixed H4" is an artifact of an unstated comparison configuration.
  3. Statistical rigor and generalization evidence are insufficient. Results are "averaged over three runs" with no variance or confidence intervals; evaluation covers only 12 test scenes on a single benchmark; on unseen environments AdaFly (26.89% SR) trails AeroVLA (27.67%), so the headline gains come almost entirely from seen scenes. Thresholds (τturn\tau_{turn}, τstopτ_{stop}) and design choices (M=16, K=10) receive no sensitivity analysis.
  4. ATHS itself is never evaluated. There is no precision/recall of criticality prediction, no oracle-gated upper bound, and no analysis of the closed-loop cost when ATHS wrongly selects H2 at a critical decision, so the value of the adaptive mechanism versus its failure modes cannot be assessed. Real-world validation is weak as evidence. The 40 physical flights achieve only ~30% success with no definition of the success criterion, no failure taxonomy, and no analysis of the gap to the 44.94% simulated SR; Figure 3 shows only successful rollouts, which reads as cherry-picking. Reproducibility and responsible-research gaps. Training details (dataset size, expert-trajectory source, optimizer, hyperparameters, hardware) are absent; the policy architecture is claimed to be described in Sec. 3.1 but that section only gives the problem formulation; no code/weight release is mentioned. There is no limitations section and no safety/airspace-compliance discussion for real UAV flights.
Questions for authors
  1. What is the formal citation for the AeroBrain benchmark, including its protocol, the definition of the 12 test scenes, and the seen/unseen split? For Table 1, are the OmniNav and DualVLN numbers taken from their original papers or reproduced by you? (Neither work appears in the references.)

  2. In Table 4(a), do the Fixed H2/Fixed H4 variants use raw history, unconditional CTMC, or conditioned CTMC? Why does Fixed H4 in Table 4(a) (43.22% SR) differ so dramatically from Raw H4 in Table 4(b) (26.78%)?

  3. Given that conditioned CTMC at fixed H4 reaches 45.03% SR, slightly above ATHS's 44.94%, how do you characterize ATHS's net contribution? Is the adaptive selector essentially trading ~0.1 SR points for ~44 ms/decision of latency?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.