AdaFly: Adaptive Temporal Memory for Onboard Aerial Vision-Language Navigation

Many aerial VLN systems rely on large multimodal models, making fully onboard deployment difficult on resource-constrained UAVs. Therefore, we propose AdaFly, a lightweight onboard aerial VLN approach that combines a compact 545M-parameter policy with adaptive temporal perception. Our design is motivated by a controlled analysis of 490,090 AeroBrain decisions, which reveals that the value of additional visual history is strongly stage-dependent. Extending history from two to four observations reduces action prediction error by 12.29% near upcoming turns and stops, but increases it by 2.72% during non-critical motion. AdaFly exploits this asymmetry with two components. An Adaptive Temporal History Selector determines whether additional older observations should be encoded before action-chunk generation, while a Context-Conditioned Temporal Memory Compressor condenses the selected history into a fixed set of task-relevant memory tokens. On AeroBrain, AdaFly achieves a 44.94% navigation success rate, representing a 69.0% relative improvement over a representative high-performing aerial VLN baseline despite using a substantially smaller model. The proposed memory compressor reduces the four-frame historical representation from 144 visual tokens to only 16 memory tokens, an 88.9% reduction, while improving navigation performance over uncompressed history. We further deploy the complete system fully onboard a DJI Matrice 4TD with a Manifold 3 (Jetson Orin NX) and validate it across 40 real-world flight trajectories without offboard policy inference.