acceptodds
Under review as a conference paper at ICLR 2027

World-Value-Action Model with Future Target Grounding for Language-Goal Aerial Object Navigation

Abstract

Language-goal aerial object navigation is a formidable task that requires an unmanned aerial vehicle (UAV) to find a language-specified object without knowing its location in advance. World-action models (WAMs), capable of coupling the prediction of future observations (video frames) with action generation, serve as one possible approach to aerial object navigation. However, directly deploying WAMs to this task is non-trivial due to two key challenges: (i) it is challenging to ensure that the predicted future observations contain information about the target object; and (ii) it is challenging to evaluate how the predicted future observations contribute to target search given the limited supervision provided by sparse navigation rewards. To address these challenges, we present FG-WVAM, the first World-Value-Action Model for language-goal aerial object navigation, to the best of our knowledge. Specifically, FG-WVAM employs Future Target Grounding to jointly predict future observations and heatmaps indicating the target's expected location. During training, a frozen Grounding DINO teacher provides target annotations from ground-truth future frames, teaching the model to predict both scene changes and the target's location in future observations. Furthermore, a Future Value Expert evaluates the predicted visual and target-location representations for their contribution to the search goal, using dense supervision derived from successful expert trajectories and unsuccessful search rollouts. During inference, FG-WVAM generates multiple predictions in latent space and ranks them by their value scores. The Action Expert then uses the representations associated with the selected prediction through internal cross-attention to predict the corresponding actions. Experiments on UAV-ON and AerialDojo demonstrate improved navigation success, including in unseen scenes and for unseen target categories.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
aerial object navigation, world models, future prediction, value-guided decision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_geKeOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies language-goal aerial object navigation: without prior knowledge of the target's location, a UAV must use multi-view visual observations, pose history, and a language description to search for the target, approach it, and stop at a valid position. The authors propose FG-WVAM, which integrates future visual prediction, future target grounding, value estimation, and action generation into a single joint model.

During training, the world branch learns future visual representations via conditional flow matching, and a frozen Grounding DINO produces target pseudo-labels on ground-truth future frames. These labels supervise future target heatmaps and also up-weight target regions in the video-generation loss. The Future Value Expert is supervised with stage-aware progress labels constructed from successful expert trajectories and failed search rollouts, and it scores candidate futures.

At inference, the model generates a batch of candidate futures from different noise initializations and selects one by its value score; the Action Expert then reads that candidate's visual, grounding, and value hidden states to generate actions. This process requires neither running Grounding DINO nor decoding future RGB frames or heatmaps.

The authors report same-scene and unseen-scene results on UAV-ON and AerialDoJo. In the same-scene evaluation, FG-WVAM improves the success rate over AeroAct, the strongest compared baseline, by 12.3 and 10.2 percentage points, respectively, with consistent improvements in OSR, SPL, and DTS. Component ablations and the candidate-count study support the contributions of future target supervision and value-guided selection.

Critical strengths
  1. The problem is well motivated, and the method design closely matches the needs of aerial target search.

The paper targets two practically relevant problems: future predictions that look plausible overall may not preserve the target cues that drive navigation decisions, and different predicted futures need to be evaluated with respect to the search task. Future target grounding and task-value estimation address these two problems respectively, so the link between motivation and architecture is clear. Separating pre-discovery exploration from post-discovery approach also provides a concrete scheme for process-level supervision. Although these stage-aware labels remain a proxy for progress, their definition is clearly tied to the structure of the object-navigation task.

  1. The closed-loop gains are large and not limited to a single metric.

In the same-scene comparison, OSR, SPL, and DTS also improve consistently. On UAV-ON, FG-WVAM's SR drops by 3.3 percentage points from the same-scene to the unseen-scene evaluation, less than AeroAct's 5.2-point drop, which offers some support for cross-scene generalization. The AerialDoJo unseen-scene results (Table 3) raise a split-comparability issue; see Weakness 1. The authors report means and standard deviations of SR and OSR over three runs (SPL and DTS are reported as means only), and the appendix specifies 300 evaluation episodes per run for each benchmark and split, together with the termination conditions.

  1. The ablations provide substantive support for the two main components.

All variants in Table 4 share the same backbone, observation and action interfaces, and training data. Even without target-weighted generation, adding future grounding raises SR from 25.6% to 32.8% on UAV-ON and from 19.1% to 25.2% on AerialDoJo. The module consisting of the Future Value Expert and candidate selection also yields gains of 5.0 and 3.8 percentage points on the two benchmarks, respectively. Figure 4 further shows that, within the tested range, increasing the number of candidates continues to improve success rates. The existing experiments therefore support the roles of future target supervision and value-guided selection, although the correspondence between the K=1 result and the component ablations still needs clarification (see Weakness 1).

Critical weaknesses

1. Two experimental results need clarification, as they bear directly on how the main conclusions should be interpreted.

(a) Comparability of the AerialDoJo unseen-scene split. §5.1 mentions an additional unseen-scene evaluation only on UAV-ON, and §5.3 is titled “Generalization to Unseen UAV-ON Scenes”, yet Table 3 reports unseen-scene results on AerialDoJo. More notably, on AerialDoJo every retrained method achieves a clearly higher SR on unseen scenes than in the same-scene setting:

MethodSame-scene SRUnseen-scene SR
OpenFly8.4%21.5%
ANWM9.2%27.4%
AeroAct18.6%32.4%
FG-WVAM28.8%43.2%

This suggests that the two splits are not comparable in difficulty or composition, so Table 3 can hardly be read as evidence that the advantage carries over to unseen scenes. Under the official protocol of the AerialDojo-200K paper, 20% of the tasks in out-of-distribution (OOD) scenes may be used for adaptation; moreover, short-range Base tasks (reference trajectories of 5–30 m) make up about 46% of the OOD test set, compared with about 36% of the in-distribution test set. Please clarify whether the unseen-scene split in this paper corresponds to that OOD split, whether the adaptation subset was used, and how the two test sets compare in difficulty.

(b) Correspondence between the candidate-count study and the component ablations. In Figure 4, the SR at K=1K=1 (32.8% on UAV-ON, 25.2% on AerialDoJo) is identical to that of the “+ Future grounding” variant in Table 4, which has no value expert. However, at K=1K=1 the full model still contains the Future Value Expert, and according to Eq. 12 the Action Expert still reads the value hidden states Z^V\hat{Z}^{V}. An exact match on both benchmarks at once is hard to attribute to coincidence.

If the K=1K=1 result comes from the full model, the value hidden states have no measurable effect on action generation, and the entire benefit of the value expert comes from candidate selection. If the K=1K=1 point reuses the grounding-only result, the improvement from K=1K=1 to K=3K=3 in Figure 4 conflates two factors: adding the value expert and increasing the number of candidates. Either case affects how the value expert's contribution should be attributed; please clarify.

2. Attributing component contributions requires additional controls and training details.

(a) The standalone contribution of target-weighted generation is not quantified. The grounding-only configuration explicitly omits the target-weighted video loss, whereas the full model includes both this weighting and the value-related modules, so the current results cannot isolate how much the weighting contributes (§5.4, Table 4). Since this design is presented as an important component of the method, I suggest one targeted control: keep the full model's future grounding, Future Value Expert, Action Expert, and candidate count unchanged, and set only λh=0\lambda_h = 0, so that video-generation errors are no longer up-weighted in target regions.

(b) Whether training budgets are matched across variants. Training is staged: visual dynamics are learned first, then future grounding and target-weighted generation are introduced, and finally the value and action experts are trained (§4.3). The paper states that all variants share the backbone, interfaces, and training data, but not whether the number of training steps is matched. If Base WAM does not go through the corresponding second stage, the full model's video branch effectively receives additional training, which would affect the comparisons in both Table 4 and Table 5.

In Table 5, PSNR rises from 18.71 dB to 24.46 dB and LPIPS falls from 0.288 to 0.094, while FID drops only from 167.96 to 158.12. It is somewhat counterintuitive that auxiliary grounding supervision plus a weighting applied only to target regions (wi∈[1,4]w_i \in [1, 4]) would yield such large whole-image gains. Please report the number of training steps for each variant and describe the evaluation protocol of Table 5, e.g., which candidate is decoded, which future frames are evaluated, and which split is used.

(c) Uncertainty reporting. Table 4 and Figure 4 provide no standard deviations or error bars. SR standard deviations in the main tables are about 2–3 percentage points, whereas some differences are clearly smaller, e.g., the 0.7- and 0.5-point gains from K=3K=3 to K=4K=4. In addition, the “three runs” in Appendix A.1 correspond to evaluation seeds 42, 43, and 44, and it is not stated whether training randomness is included, so the reported standard deviations may reflect evaluation randomness only.

3. The definitions of the value labels and the ranking score are incomplete, and several equations are inconsistent.

(a) The computation of the exploration score ete_t is underspecified. Before the target is reliably grounded, the value label is determined entirely by ete_t (Eq. 4). The “Exploration score” paragraph in Appendix A.5 only states that this score combines inter-step motion and changes in target evidence and is normalized to [0,1][0, 1]; it does not give the definition of each component, the combination weights, or the normalization procedure. Since exploration is a major part of this task, different implementations could produce markedly different supervision signals.

(b) The ranking score is defined inconsistently. In Eq. 11, VψV_\psi is a velocity-field predictor that takes YτY_\tau and τ\tau as inputs; in Eq. 14, the same VψV_\psi directly outputs the scalar SkS_k without YτY_\tau or τ\tau; and Eq. 5 writes the ranking score as DV(Z^V,k)D_V(\hat{Z}^{V,k}). Please specify the readout actually used: is the scalar taken at a particular future step, aggregated over the value sequence, or produced by a separate readout head? How does it connect to the sequence-level flow-matching supervision in Eqs. 10–11, and which time steps does the value sequence cover?

In addition, Figure 5C in Appendix A.2 shows a paired-rollout configuration whose value loss combines regression and pairwise ranking and whose action loss is success-only cross-entropy. The paper does not state whether any reported result uses this configuration.

(c) Whether the value expert's conditioning inputs match its labels during training. Appendix A.6 states that the value expert is conditioned on generated latent futures, whereas the value labels are computed from ground-truth trajectories. If the value and the future are denoised jointly at the same noise level, i.e., conditioned on noised ground-truth future latents, the labels and the conditioning correspond by construction; if the conditioning instead comes from a separately integrated generated future, that future may differ from the ground-truth one, and the label would no longer describe the candidate being conditioned on. Since this detail determines whether the value supervision is sound, please provide brief pseudo-code showing how Z^F\hat{Z}^{F} and Z^G\hat{Z}^{G} are obtained during training and how the related losses are computed.

(d) Task outcome does not enter the value labels. Eq. 4 applies the same formula to successful and failed trajectories; before the target is discovered, steps from both kinds of trajectories are labeled only by the local score ete_t, so labels in this stage carry no information about whether the exploration eventually leads to success. Please explain what information the failed rollouts actually contribute to value learning, and how the value expert distinguishes productive from unproductive exploration.

4. Category-level teacher annotations are mismatched with instance-level language goals, and the teacher's input is described inconsistently.

The paper is inconsistent about the teacher's input: Appendix A.1 states that Grounding DINO is prompted with the target category name followed by a period, whereas §4.1, the caption of Figure 2, and Eq. 8 in Appendix A.4 all describe the teacher as taking the language goal gg as input. Please clarify which is actually used.

If the category name is used, this conflicts with how both benchmarks define the task. UAV-ON defines an instance-level ObjectNav task whose instruction consists of the target's category, an estimated size, and an instance-level visual description, and success requires stopping within 20 units of that target. Goals in AerialDojo-200K are likewise instance-level, consisting of the target name, its nearest landmark, the target's direction relative to that landmark, and a semantic description (or reference image), and success requires stopping within 3 m of that instance's anchor; the example goal in the paper's own Figure 2, “Find Yellow geometric lattice-frame tower crane”, also contains color and structural attributes.

When several objects of the same category are present, a category-level teacher may place supervision on distractors: the heatmap supervision and target weighting would emphasize the wrong object, and the stage boundary tgt_g, which requires detections in two consecutive frames, may be triggered prematurely. Because the distance dtd_t is computed from the ground-truth target position xgx_g, such a mismatch mainly affects tgt_g, the reference distance dtgd_{t_g}, and the stage assignment rather than the distance itself. Please explain how category-level boxes are mapped to the target instance specified by each episode, report how often multiple same-category instances co-occur in the training data, and provide results or annotation examples on a subset where attribute-based disambiguation is required.

5. The correspondence between the theoretical analysis in Appendix B and the actual algorithm needs clarification.

The derivations of Propositions 1 and 2 are sound, but the object of the analysis maps only loosely onto the actual algorithm. Appendix B assumes that each candidate corresponds to an executed action prefix αk\alpha_k and compares the true and model future distributions PkP_k and P^k\hat{P}_k.

In the actual algorithm, however, the world branch is conditioned only on the history Ht\mathcal{H}_t and the language embedding ege_g, not on actions (Appendix A.3), and candidates differ only in their noise initialization (Appendix A.1); each candidate is a single sample rather than a distribution, and only the first action of the 10-step prediction is executed. Please explain how αk\alpha_k and P^k\hat{P}_k map onto the implementation, or tone down the claim that Eq. 22 explains when prediction accuracy translates into better candidate selection.

6. All experiments are conducted in simulation, with no real-world validation.

Both UAV-ON and AerialDojo-200K are simulation benchmarks built on Unreal Engine, and the paper reports no results on a real UAV or on real imagery. This matters particularly for this method: the world branch learns future prediction entirely from simulated renderings, and candidate selection relies on these generated futures, so the appearance gap of real imagery could affect future prediction, value scoring, and action generation at once. The paper's own introduction notes that AeroAct, its main reference baseline, demonstrates real-world quadrotor flight, whereas the conclusions here are limited to simulation.

The computational setting is also far from onboard deployment. The latency in §5.7 is measured on an A100 (0.54 s per request at K=3K=3) for a model built on the 2B-parameter LTX-Video that runs 10 integration steps over four camera views, while each decision executes only a single 1 m translation or 15° yaw action. It is unclear whether this closed loop can meet real-time requirements with onboard compute or with ground-station inference over a wireless link.

I encourage the authors to add a small-scale real-world or real-imagery validation, e.g., evaluating future prediction and grounding quality on real aerial sequences. If this is not feasible in the short term, the paper should at least add a limitations discussion (it currently has no dedicated limitations section) covering the sim-to-real gap, and report or estimate inference latency on an embedded platform (e.g., a Jetson-class device).

7. Reproducibility details and other minor issues.

The paper already reports the numbers of training and test trajectories, the number of evaluation episodes, the random seeds, the sampling ratio of successful to failed trajectories, and the unified observation and action interfaces of the main baselines. Still missing are reproducible definitions of the scene and category splits, the policy used to collect failed rollouts, and the pretraining initialization and task-adaptation details of the main retrained baselines.

Several minor issues:

  • The baseline descriptions are inconsistent. §5.1 states that the baselines, including GPT-5.6, were retrained, but GPT-5.6 Sol is a closed-source model (the AerialDojo-200K paper lists it among closed-source MLLM baselines), and the paper does not explain how it was trained or adapted. Table 3 marks its result as quoted from the original paper, while Table 1 reports it with standard deviations over three runs. The model is also named inconsistently (GPT-5.6 in the text, GPT5.6sol in the tables), and the APEX results in Table 1, quoted from the original paper, lack the † marker mentioned in the text.
  • The abstract claims improvements on unseen target categories, but the main text only states that test targets include categories unseen during training; no results are broken down by seen versus unseen categories.
  • In Figure 1(c), the value curve during the exploration stage appears to lie around 0.4, above the β=0.25\beta = 0.25 cap imposed by Eq. 4; please double-check. On mechanism analysis: Table 4 and Figure 4 already support the closed-loop benefit of value-guided selection, and Figure 3 provides qualitative examples. Comparing candidate scores with the actual short-horizon progress after executing the corresponding actions, on a set of resettable simulator states, would explain more directly why value ranking works. Similarly, analyzing prediction quality separately for cases where the target is actually visible versus not visible would add target-specific evidence that whole-image generation metrics do not capture.
Questions for authors

Q1. AerialDoJo unseen-scene split. Does the unseen-scene split in this paper correspond to the official OOD split of AerialDojo-200K, and was the 20% adaptation subset used? Please describe the task composition of this split, explain why every retrained method achieves a higher SR on it than on the same-scene split, and reconcile this with §5.1, which mentions an unseen-scene evaluation only for UAV-ON.

Q2. The K=1K=1 result in Figure 4. The SR at K=1K=1 is identical to that of the “+ Future grounding” variant in Table 4. Does this result come from the full model with the Future Value Expert? If so, does this mean that the value hidden states read by the Action Expert have no effect on actions?

Q3. Component attribution and training budget. Could you add results for the full model with only λh=0\lambda_h = 0, and report uncertainty for Table 4 and Figure 4? Please also state whether training steps are matched across variants, and describe the evaluation protocol of Table 5.

Q4. Exploration-stage labels. Please give the complete formula for ete_t, including the component weights and normalization, ideally with an annotated example of a short trajectory segment. In addition, what information do failed rollouts actually contribute to value learning?

Q5. Value readout and cross-branch conditioning. Which of Eq. 5 and Eq. 14 reflects the actual implementation? How is the scalar score read out from the value sequence, and how does it connect to the supervision in Eqs. 10–11? Is the paired-rollout configuration in Figure 5C used for any reported result? Could you provide brief pseudo-code showing how Z^F\hat{Z}^{F} and Z^G\hat{Z}^{G} are obtained during training?

Q6. Teacher input and target instances. Does the teacher actually use the category name or the full language goal? Both benchmarks judge success with respect to a specific target instance; when several same-category objects are present, how are category-level boxes matched to the instance specified by the episode, and how do you ensure that the detections used to determine tgt_g correspond to that instance?

Q7. Theory versus implementation. Given that the world branch is not action-conditioned and each candidate is a single sample, what do αk\alpha_k and P^k\hat{P}_k in Appendix B correspond to in the implementation?

Q8. Reproducibility. Please provide the names of the standard splits used, or lists of the scenes, categories, and test episodes; describe the policy used to collect failed rollouts and the initialization and task adaptation of the main retrained baselines; and specify the setup of the GPT-5.6 baseline (zero-shot prompting or any form of training).

Q9. Real-world validation and the sim-to-real gap. Could you provide a small-scale real-world or real-imagery validation, e.g., evaluating future prediction and grounding quality on real aerial sequences? In addition, what is the model's inference latency on an embedded platform, and do you expect the closed loop to meet the real-time requirements of onboard control?

Reviewer_VcAROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper addresses language-goal aerial object navigation, where a UAV must search for and stop near a language-specified target object under partial observability using multi-view RGB cameras and pose history. The authors observe that directly applying World-Action Models (WAMs) to aerial ObjectNav suffers from two limitations: (1) generic video-prediction objectives focus on whole-scene reconstruction and neglect small or newly emerging target regions, and (2) sparse terminal navigation rewards fail to guide intermediate search decisions. To overcome these issues, the paper proposes FG-WVAM (Future-Grounded World-Value-Action Model), which shows good results in the selected benchmarks of UAV-ON and AerialDojo, when compared with (V)LLM and SOTA.

Critical strengths

The way to tackle the problem (the pipeline and formulations) backed by results is appreciated.

Using frozen models as components reduce the need for re-training based on the benchmark datasets.

Target-weighted flow matching and grounding supervision is shown by authors to improve visual prediction quality over the Base WAM.

FG-WVAM consistently outperforms AeroAct across SR, OSR, SPL, and DTS on both UAV-ON and AerialDoJo in same-scene and unknown evals.

Critical weaknesses

The flow of the text starting from abstract is difficult to follow.

There is no supplementary codebase or videos to showcase the paper's claims, which is usually good if given for robotic domain.

Fig 2 is a pipeline combining existing SOTA components with no clear novelty in the thought alignment process.

Usually LLM as baselines behavior depends on the prompt and the inputs passed to it.

The Appendix section by reading, we get the feeling that it contains significant portions of AI generated text based on codebase that authors have -- which makes the reading mechanical.

Unexplained Performance Inversion Between Same-Scene and Unseen-Scene Splits on AerialDoJo and Missing ObjectNav Baselines as per Tables in the experimental results section. Kindly cross check and give non-trivial explaination.

There are multiple technical logical gaps across the paper wrt what is written and shown -- too many to list. May be manuscript written in hurry. Typesetting, latex issues I am ignoring.

Questions for authors

What is the utility and scope of the propositions in appendix?

What is the technical philosophy behind the Exploration Score?

Are unsuccessful rollouts also used to train the world model?

https://cvpr.thecvf.com/virtual/2026/oral/40327 - any evaluation wrt to this paper, though refered in related work?

Reviewer_844mOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes FG-WVAM for language-goal aerial object navigation. The model predicts future observations and target locations, scores candidate futures, and uses the selected candidate to generate navigation actions. Its main additions are future target-grounding supervision from a frozen Grounding DINO teacher, a target-weighted video prediction loss, and a value model trained with separate progress labels for exploration and target approach. At inference, it selects among several latent futures without decoding images or running the teacher. Results on UAV-ON and AerialDoJo show higher success rates than the evaluated baselines in both seen and unseen scenes. The paper also includes component ablations and an analysis of candidate count versus inference latency.

Critical strengths

The method addresses a concrete weakness of visual prediction for navigation. Small target objects can receive little emphasis in a whole-image prediction objective. Supervising future target locations and increasing the loss weight around those regions provides a direct way to make the learned representation more relevant to the search task.

The reported navigation gains are substantial across two benchmarks. FG-WVAM improves success rate over AeroAct by 12.3 percentage points on UAV-ON and 10.2 points on AerialDoJo, with improvements also reported in unseen scenes.

Critical weaknesses

None

Questions for authors

How is the exploration score ete_t in Equation (4) computed? Appendix A.5 mentions motion and changes in target evidence, but does not specify how these quantities are measured, combined, or normalized. Please provide the calculation procedure, since it determines the value supervision before target discovery.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.