acceptodds
Withdrawn from ICLR 2027

AerialWAM: Reconstructive Aerial World Action Model for Open-world Semantic-level Object-goal Search

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Open-world semantic-level object-goal search is an important yet challenging task for aerial agents. Existing approaches rely on spatio-semantic representations and multimodal large models to reason over historical observations, failing to maintain semantic-object alignment during long-horizon search in open-world environments. To solve this problem, this paper presents AerialWAM, the first reconstructive aerial world action model, which can autonomously search for semantic-level object-goals with semantic-object alignment. Specifically, AerialWAM first constructs a object-conditioned aerial world action model that couples semantic understanding and action generation using a Mixture-of-Transformers to achieve long-horizon prediction and action execution. We further design a semantic-object reconstructor that generates an goal-centric object image from current observations and the semantic description. During search, each new observation refines this object image and feeds it back to the world action model for the next action, forming a closed loop to achieve semantic-object alignment. We conducted extensive experiments on AerialDojo-200K. The result shows AerialWAM achieves average success rates 5.77× and 23.98× those of existing methods in in-distribution and out-of-distribution settings, respectively.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Aerial Agent, World Action Model, Object-goal Search

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_qmeAOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper researchs open-world semantic-level object-goal search for aerial agents. The authors propose "AerialWAM," a semantic-object reconstructor that generates a goal image based on current observations and language instructions. This goal image then serves as a condition for a subsequent world action model, which outputs the corresponding actions required to search for the target. The model is trained in three stages, respectively optimizing the object-image reconstruction, world action model, and the full closed-loop system. Experiments are conducted in two simulation scenes from AerialDojo-200K. The authors claim that their model outperforms the VLM-based baselines.

Critical strengths

This paper integrates a generated target representations into the online, closed-loop decision-making process for open-world object-goal search tasks. The model updates the goal image (which provides the world action model with visual information about the next action output) using real-time observations from the surrounding environment and language instructions. This mechanism allows the world action model to directly understand the target obeject, enabling more effective search planning.

Critical weaknesses
  1. The effectiveness of the "semantic-object reconstructor" proposed in the paper lacks validation of experiments. The paper lacks ablation experiments to evaluate the effectiveness of the semantic-object reconstructor. For instance, authors could conduct a simple comparative experiment by testing the "world action model" with and without the "goal image" generated by the semantic-object reconstructor as an input. Furthermore, when the target object is not present in the model's input observations, do the goal images outputed by the semantic-object reconstructor align with the actual target object? If they do not align, does this lead the subsequent world action model to generate incorrect actions?

  2. Benchmark coverage is limited, and there is a lack of baselines for agent systems. The evaluation is limited to two island scenarios within AerialDojo-200K. Testing should be expanded to more AerialDojo scenarios or external benchmarks (such as UAV-ON[1]). Furthermore, please add some relevant agent baselines (e.g., AOA-F[1] and APEX[2]). Comparing a world-action model trained specifically for certain tasks against a basic VLM controller does not constitute a fair comparison.

  3. The results reported in Tables 1 through Table 4 require further clarification, and the corresponding supporting records should be provided. In some cases, the reported percentages do not align with the stated number of test trials. For example, Claude’s success rate (SR) on the NIsland_0 Base task is reported as 5.88%; however, this figure cannot be derived from an integer number of successful trials out of 295 total trials. (17/295 is approximately 5.76%, while 18/295 is approximately 6.10%.) Besides, in the case of Grok, the episode counts for out-of-distribution (OOD) cannot even be reconciled with a common integer denominator of 334 or less. Could the authors provide the exact inputs, prompts, history, and output formats used in these benchmarks, as well as the specific episode IDs and episode counts?

[1] Xiao J, Sun Y, Shao Y, et al. UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agents[J]. arXiv preprint arXiv:2508.00288, 2025.

[2] Zhang D, Chen P, Xia X, et al. APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigation[J]. arXiv preprint arXiv:2602.00551, 2026.

Questions for authors
  1. When the target is invisible and its semantic description is ambiguous, what evidence shows that the reconstructed object image matches the actual target? Could you provide some visual examples?

  2. See 1, 2 and 3 in the Weakness.

Reviewer_VSYNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AerialWAM, a reconstruction-based World Action Model for UAVs, designed for semantic-level object search in open-world environments. The approach integrates a diffusion-model-based semantic object reconstructor with a dual-expert world action model to facilitate both object searching and action generation.

Critical strengths
  1. Research-worthy task formulation. The paper addresses the problem of semantic-level object search for UAVs in open-world environments—a challenging task with significant practical utility. The authors’ motivation—maintaining alignment between semantic information and target objects during long-range exploration—is well-founded.

  2. Sound framework design. Combining a semantic object reconstructor with a world action model is an intuitive and logical design. The proposed "observe-reconstruct-act-execute" paradigm offers a promising solution for continuously updating target representations and guiding subsequent actions.

  3. Strong simulation results. On the AerialDojo-200K benchmark, the proposed method achieves a notable improvement in success rate compared to several baselines based on large multimodal models. The experiments also cover tasks of varying lengths and report a comprehensive set of navigation metrics.

Critical weaknesses
  1. Lack of ablation studies.The paper does not provide ablation studies to validate the effectiveness of key components, such as the semantic object reconstructor, the dual-expert architecture, or the module for predicting future latent states. In particular, the absence of direct evaluation regarding reconstructed image quality and the accuracy of semantic-target alignment makes it difficult to gauge the specific contribution of the proposed reconstruction mechanism.

  2. Insufficient experimental comparisons and generalization analysis. Quantitative experiments are primarily compared against general-purpose large multimodal models, lacking benchmarks against methods specifically designed for UAV navigation or world-model-based navigation. Furthermore, while the paper claims to conduct out-of-distribution (OOD) evaluation on "NIsland_1," the model was actually fine-tuned using 137 training trajectories from that specific environment; therefore, this experiment fails to demonstrate the model's generalization capability to entirely unseen environments.

  3. Absence of real-world validation and a high collision rate. All experiments were conducted exclusively in a simulation environment; no real-world drone experiments or sim-to-real transfer evaluations were performed. Furthermore, the model’s collision rate remains high—reaching 84.09% in the "Long" task within the NIsland_0 environment—which raises concerns regarding the method's robustness and its feasibility for practical application.

  4. Poor image quality. Several figures in the paper are blurry and difficult to interpret, particularly the framework diagram and the trajectory visualizations. These appear to be rasterized images rather than vector graphics, leading to a significant loss of clarity when enlarged. It is recommended that the authors increase the resolution of these figures and, where appropriate, use vector graphics to improve the overall readability of the paper.

Questions for authors

Refer to the weaknesses.

Reviewer_njAhOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework called AerialWAM for open-world semantic-level target-object search based on aerial agents. This framework combines a semantic-object reconstructor with a dual-expert world action model. The reconstructor generates target-centered object images based on current multi-view observations and linguistic targets, while the world action model uses this representation—along with actual observations and predicted future features—to select navigation actions or STOP.

The experiments were conducted on two scenarios in AerialDojo-200K, covering the Base, Standard, and Long tasks. The authors compared AerialWAM with several multimodal large language models and reported a significant improvement in navigation success rates.

Critical strengths

This paper explores a practical and challenging problem in the field of semantic target search in the air: agents are required to actively determine when to terminate their search while exploring an unknown environment.. Integrating reconstructed target representations into predictive action models is a promising research direction. If the evaluation methodology and comparison results are properly validated, the success rate reported in this paper is quite impressive.

Critical weaknesses
  1. Lack of experimental validation for key technical contributions.

The paper introduces several components, including a semantic-object reconstructor, a two-expert world action model, future latent prediction, and a three-stage training procedure. However, the paper does not report any ablation studies for these components.

Specifically, the paper neither compares the full model against a version with the reconstruction module removed, nor does it clarify the individual contributions of the "future prediction" and "two-expert architecture" components. Furthermore, although object reconstruction is central to the proposed method, the paper provides neither a quantitative evaluation of the reconstruction performance nor a qualitative analysis of the actual reconstruction outputs.

In the absence of such experiments, it is difficult to determine whether the reported performance gains stem from the proposed mechanisms or from other factors. This significantly weakens the evidence supporting the paper's primary contributions.

  1. The justification for baseline comparisons is insufficient, and the assessment of generalization capability is quite limited.

Section 4.1 states that all comparisons utilize the same episode manifest and evaluation protocol. However, the percentages presented in Tables 1 and 3 indicate that the number of episodes varies across different methods. For instance, the results for STEP3-VL in Table 1 under the "3 m" metric (SR 0.50%, OSR 3.52%, CR 38.19%) correspond to 1, 7, and 76 episodes, respectively—out of a total of 199 episodes—rather than the "Base test set" of 295 episodes mentioned in the text. Similar inconsistencies exist for other baseline methods, yet the paper fails to specify the actual number of episodes used in their evaluations. While this does not necessarily imply an unfair comparison, further clarification is required to accurately assess the claimed performance gains. Furthermore, the comparisons focus exclusively on multimodal language models, lacking baselines—such as navigation models trained for specific tasks—for broader context.

The evaluation on NIsland_1 incorporates 137 training trajectories derived from the evaluation environment itself; the authors acknowledge that this does not constitute "zero-shot transfer." Consequently, these results merely reflect the model's performance within a specific setting, offering very limited evidence regarding its ability to generalize to unseen environments.

In summary, these issues weaken the experimental support for the framework's effectiveness and applicability.

Questions for authors
  1. Contributions of individual components: Could the authors provide ablation study results to evaluate the respective roles of the "semantic-object reconstructor," "future latent prediction," and "two-expert architecture"? Specifically, how does the full model perform compared to a version of the World Model (WAM) that excludes the reconstruction module?

  2. Baseline evaluation statistics: Could the authors specify, for each baseline model in Tables 1 and 3, the total number of episodes evaluated and the number of successful episodes? How were the percentages reported in the paper calculated according to the stated "matched-comparison protocol"? Are these results averaged over multiple runs, or were they evaluated on different subsets of the data?

  3. Consistency in evaluation: In Table 3, the results for Grok-4.5 (including a 10.42% collision rate and a 1.96% success rate at 5 meters) appear difficult to derive from a consistent, integer-based total number of episodes under the defined metric framework. Could the authors clarify the raw numerical statistics and the aggregation methods used to compute these metrics?

  4. Training pipeline: The paper outlines three training stages and mentions the potential use of a "score-function estimator." Which optimization algorithm was actually employed in Stage III? Furthermore, could the authors provide details regarding model configuration, the prediction horizon, settings for diffusion sampling, and the computational resource requirements?

Reviewer_4vQ8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AerialWAM addresses aerial semantic object-goal search by coupling a diffusion-based object reconstructor with a two-expert world action model. At each step, a reconstructed target image conditions future latent prediction and action selection alongside real observations. Three-stage training combines prediction, reconstruction, and terminal search success. Experiments on two AerialDojo-200K scenes report higher success rates than the evaluated multimodal language model baselines.

Critical strengths
  1. Updating a goal-centered image after each observation offers a plausible way to connect semantic grounding with predictive control. Retaining real observations alongside reconstruction makes the proposed interface well motivated.
  2. The reported Base-task success rates substantially exceed the strongest listed baselines: 33.90% versus 6.12%, and 47.01% versus 2.83% (Tables 1 and 3). These gains make the approach worth further consideration.
Critical weaknesses
  1. The comparisons do not establish an advantage over existing aerial search systems. Tables 1 and 3 use MLLM baselines, while Tables 2 and 4 report only AerialWAM. Representative methods discussed in Section 2, such as APEX and OctMem-Agent, are not evaluated in the supplied manuscript. Since the motivation targets their limitations, a compatible system-level comparison would be consequential. Please provide any existing results or explain the interface differences that prevent a fair comparison.
  2. The evidence does not isolate the claimed alignment and stopping benefit. Tables 1–4 evaluate the complete system, without a matched reconstruction ablation in the supplied manuscript. Moreover, FSR = OSR − SR (Section 4.1) includes episodes that reach the target and later collide, so it does not measure incorrect STOP decisions specifically. A matched comparison with reconstruction disabled or held fixed, plus a breakdown of existing stopping outcomes, would clarify whether the central mechanism explains the gains.
  3. The evaluation supports a narrower claim than the OOD framing suggests. Sections 4.1 and 4.3 explicitly disclose 137 Island_1 training trajectories, so these results do not establish unseen-scene transfer. Although matched evaluation interfaces are stated, per-baseline episode counts and task-specific training exposure are not clear from the supplied tables. These details matter when interpreting the large gains. Please clarify the comparison population and training access, and consistently describe this setting as evaluation after scene-specific training.
Questions for authors
  1. Are existing results available for a representative aerial search system discussed in Section 2, such as APEX or OctMem-Agent? If not, which task or interface differences prevent a fair comparison?
  2. Is there an existing comparison with the same WAM trained without reconstruction or with a fixed target image? Please report or locate it, and clarify whether Stage III retains any reconstruction or prediction losses beyond Equation 15.
  3. For Tables 1 and 3, what episode count and task-specific training exposure apply to each model? Were the reported comparisons computed over identical episode sets, as specified in Section 4.1?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.