World-Value-Action Model with Future Target Grounding for Language-Goal Aerial Object Navigation

Language-goal aerial object navigation is a formidable task that requires an unmanned aerial vehicle (UAV) to find a language-specified object without knowing its location in advance. World-action models (WAMs), capable of coupling the prediction of future observations (video frames) with action generation, serve as one possible approach to aerial object navigation. However, directly deploying WAMs to this task is non-trivial due to two key challenges: (i) it is challenging to ensure that the predicted future observations contain information about the target object; and (ii) it is challenging to evaluate how the predicted future observations contribute to target search given the limited supervision provided by sparse navigation rewards. To address these challenges, we present FG-WVAM, the first World-Value-Action Model for language-goal aerial object navigation, to the best of our knowledge. Specifically, FG-WVAM employs Future Target Grounding to jointly predict future observations and heatmaps indicating the target's expected location. During training, a frozen Grounding DINO teacher provides target annotations from ground-truth future frames, teaching the model to predict both scene changes and the target's location in future observations. Furthermore, a Future Value Expert evaluates the predicted visual and target-location representations for their contribution to the search goal, using dense supervision derived from successful expert trajectories and unsuccessful search rollouts. During inference, FG-WVAM generates multiple predictions in latent space and ranks them by their value scores. The Action Expert then uses the representations associated with the selected prediction through internal cross-attention to predict the corresponding actions. Experiments on UAV-ON and AerialDojo demonstrate improved navigation success, including in unseen scenes and for unseen target categories.