acceptodds
Under review as a conference paper at ICLR 2027

AirWorld: Language-Conditioned Predictive World Models for Generative UAV Exploration

Abstract

We study open-vocabulary target search by a single micro aerial vehicle (UAV) instructed in natural language. Semantic exploration systems such as STEM propagate semantic priorities to frontier viewpoints and optimize a prescribed planning objective. We propose AirWorld, a language-conditioned predictive world model that maintains a belief over geometry, semantics, and task intention from RGB-D observations and UAV state. An action-conditioned latent model predicts future evidence, while a conditional flow-matching policy generates candidate control sequences. Belief-space rollouts rank candidates by discovery, coverage, expected information gain, and risk; a receding-horizon controller executes a short verified segment. We formulate the belief update, constrained trajectory generation, and training objectives, and report matched single-UAV benchmark results for the proposed method and the listed baselines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, embodied AI, language-conditioned exploration, flow matching, belief-space planning, information gain, UAV navigation, open-vocabulary target search

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_LVLSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies open-vocabulary target search by a single UAV: given a natural-language instruction, the drone must find the target in an unknown indoor environment using only RGB-D and its own state, without any prior map. AirWorld plans in belief space with a receding horizon. A Gaussian latent belief summarizes the environment, an action-conditioned world model predicts future observations for candidate actions, conditional flow matching generates candidate control sequences, and after a geometric feasibility check the candidates are ranked by a weighted sum of discovery, coverage, information gain and collision risk. On a matched benchmark in three simulators against 10 baselines, the model reaches 92.5% SR versus 89.0% for the strongest baseline (AirHunt). Component ablations and robustness diagnostics are also provided. A hardware platform and a demo video are shown, but no real-world numbers are reported.

Critical strengths

The problem formulation and system design are sensible. Imagining the consequences of candidate paths in a world model, ranking them by information gain and risk, and executing only segments that pass geometric verification, with a fallback otherwise, is a reasonable way to combine learned prediction with safety checks.

Critical weaknesses

The information-seeking mechanism the paper emphasizes contributes little. In Table 7, removing the information-gain and uncertainty terms lowers SR only from 92.5 to 88.0; in Table 2, removing the world model gives 79.0 and removing language conditioning 83.0. This does not match the weight given to the term in contribution (iii) and the conclusion.

Questions for authors

Where do the flow-matching demonstrations and region-instruction annotations come from, which planner generated them, and did it see the full map? Sec. 5.1 says learned controls share the demonstrations; did the other baselines get equivalent data?

1 comment
Reviewer_LVLSOpenReview ↗

Official Review of by Reviewer LVLS

Thank you for the response and the additional experiments. I appreciate the matched-supervision control and the plan to revise contribution and the conclusion. I will keep my rating. Please also state in the final version which planner generated the demonstrations and whether it had access to the full map.

Reviewer_DJC4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AirWorld addresses open-vocabulary target search by a single UAV from natural-language instructions. It maintains a Gaussian latent belief over geometry, semantics, and task intention, updated from RGB-D observations and UAV state. A conditional flow-matching policy proposes candidate control sequences, an action-conditioned latent world model rolls out their consequences, and the imagined futures are scored by expected discovery, new coverage, information gain, collision risk, and residual uncertainty. Only a geometrically verified, risk-bounded segment is executed in a receding-horizon loop. Training combines a variational world-model objective with flow matching, alignment, and a disjoint calibration stage. On a matched single-UAV benchmark against ten baselines, AirWorld reports 92.5% success, 0.68 SPL, and 4.0% collisions, improving on AirHunt (89.0%, 0.64, 4.5%) at a slightly longer failure-penalized time (121 vs. 118 s). Ablations attribute large gains to the world model and the information-gain objective. Limitations include Gaussian and linearization assumptions, calibration-dependent uncertainty, and flow-sampling cost.

Critical strengths
  1. The paper casts language-conditioned aerial search as belief-space predictive control and specifies the belief update, the discounted objective with discovery, coverage, information-gain, risk, and uncertainty terms (Eq. 1), the flow-matching loss, and the geometric feasibility constraints. Each component maps to a stated term rather than an unexplained pipeline stage, which makes the design auditable and the ablations meaningful.
  2. The protocol equalizes RGB-D sensing, pose estimates, field of view, low-level controller, collision checking, start poses, targets, and time/battery budgets, with no privileged map and paired episode seeds. Ten baselines span geometric exploration, semantic navigation, generative control, and 2025-2026 aerial systems, and the UAV adaptations are documented rather than compared through published numbers. The component and objective ablations further isolate the world model, language conditioning, flow matching, future prediction, uncertainty modeling, gain reward, and calibration stage.
  3. AirWorld reports 92.5% SR, 0.68 SPL, and 4.0% collisions versus 89.0%, 0.64, and 4.5% for AirHunt, and the advantage persists on unseen HM3D, Isaac, and AirSim scenes and on harder relational instructions (87.0% SR on held-out compositions). Robustness results degrade gracefully under 30% depth dropout and 0.10 m / 2 deg pose noise. The authors also report calibration diagnostics (risk Brier score 0.069 to 0.044) while explicitly stating that calibration does not certify safety.
Critical weaknesses
  1. The paper argues that learning an action-conditioned world model to predict future evidence is the primary driver of its exploration performance. However, in Table 2, removing the world model (w/o World Model) simultaneously disables multi-step future prediction (Future: ×), reverting the system to myopic scoring on the current map. Consequently, of the reported 13.5 pp performance drop (from 92.5% to 79.0%), 8.5 pp stems simply from the loss of lookahead planning itself, which may conflate the generic benefit of multi-step planning with the specific contribution of the world model. A more informative control group—retaining competitive, training-free 3D lookahead planning (e.g., TSDF volumetric ray-casting or frontier-based multi-step rollouts) without the learned world model—would substantially clarify the true marginal value of the proposed framework over established planning paradigms.

  2. The paper highlights real-world quadrotor deployment and closed-loop flight protocols across the Abstract, Introduction, and key figures (Figures 1 and 3). However, across all quantitative benchmark tables (Tables 1 through 13), the reported empirical results are derived exclusively from simulation suites (Habitat-Sim, Isaac Sim, and AirSim), without providing quantitative performance metrics on the physical UAV platform (such as real-world success rate, SPL, collision rate, or flight duration). Consequently, the physical deployment remains primarily a qualitative feasibility demonstration, making it difficult to substantiate the system's closed-loop robustness against real-world aerodynamic disturbances, sensor latency, and onboard compute constraints. Providing quantitative statistics from physical flight trials (even on a modest episode set), or appropriately calibrating the narrative to clarify the qualitative scope of the hardware trials, would substantially strengthen the alignment between the claims and the presented evidence.

  3. The paper highlights "constrained conditional flow-matching control generation" as a core contribution, arguing that it produces smoother, multi-modal continuous aerial trajectories over rigid, classical motion primitives. However, in the matched-budget comparison under a 100 ms planning deadline (Table 8), conventional discrete motion primitives already achieve a 90.0% success rate, whereas the flow-matching generator—requiring continuous vector-field training and a 20-step ODE integration—reaches 92.5%, yielding a marginal gain of only 2.5 percentage points. This suggests that the drone's collision avoidance and search capability may largely be governed by the downstream geometric feasibility verification rather than the generative sampler itself. Given the substantial training complexity and numerical integration overhead of flow matching, this modest performance differential makes it difficult to justify the necessity and computational cost-effectiveness of the generative policy over established, lightweight primitive libraries.

  4. The paper presents its prior-to-posterior information-gain objective as a primary theoretical driver of active exploration. However, in Table 7, completely disabling both information gain and uncertainty penalties (κ = η = 0) still retains an 88.0% success rate (a modest 4.5 pp drop from 92.5%). Moreover, optimizing information gain alone (η = 0) actually increases the collision rate from 4.0% to 5.0% and degrades task success. These empirical results suggest that information-gain maximization alone may encourage riskier flight behavior with limited standalone gains, making its necessity as an indispensable exploration engine less clear.

Questions for authors

See weakness.

Reviewer_k8sJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies open-vocabulary target search by a single UAV in initially unknown 3D environments under natural-language instructions. AirWorld constructs a Gaussian latent exploration belief from RGB-D observations, UAV states, and language instructions, and uses an action-conditioned world model to predict future observation features and search outcomes. A conditional flow-matching policy generates sequences of velocity and yaw-rate commands. After geometric feasibility checks, the system performs belief-space rollouts and ranks candidate trajectories based on target discovery, coverage, expected information gain, collision risk, and residual uncertainty. It then executes a validated short-horizon control sequence and replans in a receding-horizon manner. On a single-UAV benchmark with standardized online inputs, AirWorld reports a success rate of 92.5% and an SPL of 0.68. The paper also presents component and objective-function ablations, sampler comparisons, generalization evaluations, prediction calibration analyses, robustness tests, and online planning latency analyses.

Critical strengths
  1. The core hypothesis underlying predictive planning is supported by reasonably thorough experimental validation. Through component ablations and a two-factor experiment, the paper distinguishes the contributions of future prediction and language conditioning to search performance. The results indicate that ranking actions based on predicted future observations can effectively improve search performance. Moreover, the analysis of prediction quality further supports the world model's ability to evaluate candidate actions, providing reasonably convincing empirical evidence for the effectiveness of predictive planning.

  2. Practical UAV constraints and computational costs are carefully considered. The paper integrates motion constraints, collision checking, and safety fallback mechanisms into the planning pipeline. Through ablation studies and comparisons across computational budgets, it analyzes the effects of uncertainty modeling, prediction calibration, and action generation methods. The prediction calibration and online latency experiments further demonstrate the method's potential practical value in terms of reliability and real-time performance.

Critical weaknesses
  1. The advantage over the strongest adapted baseline lacks evidence quantifying statistical uncertainty. Compared with AirHunt, AirWorld achieves a 3.5-percentage-point higher success rate, an SPL improvement of 0.04 , and a 0.5-percentage-point lower collision rate. However, AirWorld does not lead on every metric: AirHunt has a shorter failure-penalized search time (118 s vs. 121 s), while VISTA achieves higher terminal coverage (77% vs. 75%). Although the paper uses five training seeds, stratified 95% confidence intervals, and paired tests with corrections for multiple comparisons, the main results do not present the corresponding intervals or test outcomes. Nor are the numbers of scenes and episodes in each test suite reported. The performance differences may be genuine, but the available results are insufficient to determine whether the advantage over recent competing approaches is robust.

  2. Key implementation details and comparative experimental settings are insufficiently documented. Although the paper attempts to standardize the observation and evaluation conditions across methods, some baselines have been adapted, potentially with different pretraining resources and adaptation costs, which may affect the fairness of the comparisons. In addition, the paper omits some key training parameters, model configurations, and dataset sizes, limiting reproducibility. Certain differences between ablation results and sampler comparisons remain unexplained, and quantitative results from real-hardware experiments are also lacking. These issues leave the reliability of the reported performance advantage and the method's practical effectiveness in need of further validation.

Questions for authors
  1. (Additional statistical results) Could the authors report the numbers of experiments for each test scenario, variation across random seeds, and confidence intervals or significance-test results for AirWorld versus the strongest baseline on metrics including success rate, path efficiency, search time, and collision rate? This would help establish the reliability of the reported improvements.

  2. (Clarification of method and comparison settings) Could the authors provide additional details on the model architecture, key training hyperparameters, training dataset size, sources of pretrained weights, and the adaptation procedures and computational budgets for each baseline? In addition, some ablations and sampler comparisons yield different results under seemingly similar settings. Could the authors explain the relevant differences in experimental conditions?

  3. (Clarification of evaluation scope) Are the main experimental results reported in the paper obtained exclusively in simulation? If real-UAV tests were conducted, could the authors provide quantitative results on task success rates, collisions, and human interventions to further clarify the method's practical deployment performance?

Reviewer_wuFJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AirWorld combines a Gaussian latent belief, predictive world model, and flow-matching control proposals for language-guided UAV target search. It ranks candidates by predicted utility and verifies geometric feasibility before execution. The paper reports 92.5% benchmark success, with ablation, generalization, and calibration studies.

Critical strengths

The ablations separate language conditioning from predictive ranking while keeping the representation and generator fixed. Adding prediction improves reported success from 84.0% to 92.5%. Sampler comparisons under fixed proposal counts and planning deadlines further help assess the contribution of flow sampling.

Critical weaknesses
  1. Equation 4 predicts marginal first-discovery probabilities, while Eqs. 6 and 20 mask post-termination samples. This appears to mix marginal and survival-conditioned targets, potentially biasing candidate scores.
  2. Missing scene/episode counts and confidence intervals make the 3.5-point success gain over AirHunt difficult to assess.
  3. Backbone identities, network dimensions, training-data quantities, and loss/utility weights are unspecified, limiting reproducibility.
  4. The primitive sampler achieves 86.0% success in Table 2 and 89.0% in Table 8, without a clear explanation of the differing settings.
Questions for authors
  1. Do discovery losses supervise conditional hazards or marginal probabilities, and how are terminated sequences masked?
  2. Can you report scene/episode counts and paired confidence intervals for the main comparisons?
  3. Can you provide the missing architecture, training-data, and hyperparameter details?
  4. What differences in the primitive-sampler settings explain the results in Tables 2 and 8?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.