acceptodds
Withdrawn from ICLR 2027

Rethinking Dense Spatial Features: Compact Spatial Queries for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World Action Models (WAMs) jointly model future visual observations and robot actions, transferring video priors to robotic manipulation. However, their image- derived representations leave interaction-relevant 3D geometry implicit; dense 3D features can provide complementary spatial cues, but may also introduce manipulation-irrelevant redundancy and additional learning and computational costs. We therefore systematically investigate whether WAMs benefit from ad- ditional spatial conditioning, which representations and supervision targets are effective for action generation, and how compact an effective spatial representa- tion can be. To test a compact and manipulation-aligned alternative within this study, we develop QuestWAM, which learns latent spatial queries through super- vision from future 3D bounding boxes, interaction points, and semantics, organiz- ing spatial information around where task-relevant objects will be and where the robot should interact with them. These queries condition action generation with- out requiring future annotations or dense 3D reconstruction at inference. Quest- WAM achieves 93.14% and 92.18% success on RoboTwin 2.0 Clean and Random, respectively, and improves LIBERO-Plus success from 49.43% for original Fast- WAM to 53.48%. Compared with online VGGT conditioning at the same token budget on RoboTwin 2.0 Clean, QuestWAM improves success by 1.00 percentage point. Controlled comparisons favor structured supervision over dense-feature distillation and latent queries over decoded attributes. Spatial interventions fur- ther show that the queries encode scene-dependent future geometry. These find- ings suggest that compact and manipulation-aligned spatial conditioning improves WAM action generation, with its value depending on how relevant information is selected and organized, beyond representational richness alone.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Robot Learning, Latent Visual Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2CpZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces QuestWAM which is a spatial-query framework for pretrained WAMs. The spatial query module is trained on 3 different supervision signals which should help the WAM model to perform better on different benchmarks (RoboTwin 2.0 and Libero-Plus): future 3D bounding boxes, interaction signals with task-relevant objects and object semantics. The overall method performs 2-3 percent better on RoboTwin 2.0 and 3-4 percent better on Libero-Plus compared to FastWAM. They further investigate their hypothesis that WAMs need a better understanding of geometry and interaction around task-relevant objects instead of reconstructing every little detail of a scene using a systematic representation study.

Critical strengths
  • Real robot experiments
  • Hypothesis which would be important in the field of robotics and WAMs
  • The latent vs decoded readout results are a useful finding
Critical weaknesses

Weaknesses

  • It is hard to follow the method section along, as it seems that the supporting figures and text are using different names for the same representations. Making this the same for both would benefit the clarity of the paper.
  • It is also not understandable what the actual architecture of QuestWAM is. Is the model build upon the architecture of FastWAM and includes this query module or is it its own architecture all together, where any WAM can be plugged in? If so, then more WAM variants need to be evaluated to confirm that this additional querying process actually increases performance.
  • The “Systematic Representation Study” is not yet a standalone contribution and hovers between ablations and an actual study. To make it a proper study it needs explicit hypotheses comparisons, multiple seeds per model, evaluation beyond RoboTwin Clean and a control for the extra training stage.
  • There is only one real other baseline included (FastWAM), but no other models, which are also mentioned in related work, like DreamVLA, WAM4D, 4D-WAM or GaussianWAM. How do these models perform compared to the proposed method? Also is the evaluation the same for all models (e.g. same checkpoint choice, etc.)? Comparing against more baselines would strengthen the paper.
  • The gain of the supervision components is not significant and it does compare the exclusion of only one at the same time, but it does not evaluate if two are missing or all 3 and we only have a plain model. The conclusion that these supervision signals help performance is therefore not conclusive. It is also not clear if the models were trained using different seeds, if only one seed was used the results and the advantages could be in the range of the variance.
  • It is not clear if the stage 1 and stage 2 training is responsible for the higher success rate of QuestWAM and FastWAM should get the same training time

Minor Weaknesses

  • Figure 1 is too overloaded and the right part is far too small. Only using the left part would benefit the paper much more and the right part can be an extra figure (if space allows it)
  • Empirical results are not a contribution, but required in this field to prove that the proposed method is working
  • Avoid empty Sections like 3 and just put the problem formulation directly below it without having the subsection there
  • Figure 2 is cut at the bottom and top and has a mismatch with “…” between blocks. In general the font is also rather small.
  • There is no table for the real-world robot results, instead a figure displaying the tasks. Moving this figure into the Appendix and adding a table with success rates would be more helpful.
Questions for authors
  • How does the method identify task-relevant objects?
  • What are semantic states?
  • Where do you get the supervision information for the query model? How are interaction points defined? Are supervision information missing or complete?
  • What exactly is used from the query module (because there is one arrow directly from the query model to the DiT action head and one from the outputed Queries)?
  • What is RoboTwin 2.0 random? What modifications are included?
  • FLEX-π uses a corrected Libero-Plus evaluation setup, where FastWAM increases performance from 49 to over 70 percent success. Could you report your results using the correct evaluation setup for FastWAM and QuestWAM?
  • Why are the latency numbers for the QuestWAM model different in table 1 and 6 to 5 and 7?

Please also consider all weaknesses also as a question that should be answered.

Reviewer_A5qROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

QuestWAM learns compact queries from future object geometry, interaction points, and semantics, then freezes the query module while adapting the action model. The study compares representation sources, supervision targets, token budgets, and action-conditioning interfaces on simulated and real-robot manipulation.

Critical strengths
  1. Matched conditioning interfaces and a training-only distillation control make the representation comparisons informative.

  2. The intervention study transparently reports evidence that limits the proposed interpretation, rather than treating geometric encoding alone as proof of geometry-guided control.

Critical weaknesses
  1. Figure 1 would be easier to follow if the rows were ordered as VGGT, VGGT+DINO, and finally QuestWAM, presenting the baselines before the proposed approach. Its internal text and plot annotations are also too small, while Figure 2 contains cropped text along its lower boundary.

  2. Section 3.4 primarily describes the experimental comparisons, ablations, and intervention analyses rather than the proposed method. It would fit better in the experimental section, alongside the corresponding results, to avoid repetition and improve the separation between methodology and evaluation.

  3. In Table 4, geometry and illumination interventions produce nearly identical trajectory changes, without clear geometry-aligned action responses. Moreover, removing individual supervision signals yields only small success-rate drops (0.16–0.62 percentage points), with no training-variability estimates to establish their distinct contributions.

  4. Although the losses for future 3D boxes, interaction points, and semantic categories are specified, the paper does not clearly explain how these annotations are obtained in simulation or on the real robot. In particular, the operational definition and extraction procedure for interaction points, the 3D coordinate system, and the annotation cost are necessary to assess reproducibility and scalability.

  5. Prior work such as S-VAM already uses geometric and semantic foresight to support action prediction. QuestWAM differs by learning compact queries through structured object and interaction supervision, but the paper should better articulate the technical insight beyond this change in supervision and representation interface. The matched VGGT-distillation comparison is useful, although its 1.16-percentage-point gap without variability estimates does not establish a robust advantage.

Questions for authors
  1. How should the near-equal geometry- and illumination-donor responses be reconciled with the proposed spatial-conditioning mechanism?

  2. How are future boxes and interaction points obtained for real-robot training, and what annotation resources are required?

  3. Which findings remain stable across independently trained checkpoints and checkpoint-selection choices?

Other suggestions (not decisive for my recommendation): Frame the findings as comparisons of supervision and representation construction, since dense-source baselines also use compact conditioning budgets; explain the different QuestWAM latency values across Tables 1, 5, 6, and 7; distinguish intervention-checkpoint settings from the policy settings producing the main success rates.

Reviewer_E33mOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents QuestWAM, which learns compact spatial queries supervised by future 3D bounding boxes, interaction points, and semantics, and uses these queries to condition action generation in world action models.

Critical strengths

The research question is practically relevant. Organizing spatial information around task-relevant objects and interaction locations provides a reasonable auxiliary conditioning design for action models with video priors, without requiring future annotations or dense 3D reconstruction at inference.

Critical weaknesses
  1. The manuscript contains clear proofreading and figure-presentation problems. The contribution list on page 3 includes duplicated text, such as “QuestWAM. QuestWAM.” and the repeated “Systematic representation study.” In addition, the action-token annotation at the bottom of Figure 2 on page 4 is clipped, and some figure labels are difficult to read.

  2. The evaluation protocols are incompletely reported. Appendix E specifies 100 episodes per task for RoboTwin Clean, but trial counts are not consistently documented for other comparisons and ablations. LIBERO-Plus also lacks evaluation counts and seed settings, limiting reproducibility.

  3. The annotation sources, physical definitions, coordinate frames, and missing-label handling for future boxes and interaction points are insufficiently specified.

Questions for authors
  1. Why is the AGILE logo on the microwave visible in some images but absent in others?

  2. For the real-world experiments, could the authors provide details of the data collection, annotation, training, and evaluation protocols?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.