acceptodds
Desk rejected at ICLR 2027

AWAM: Agentic World Action Model for Embodied Intelligence

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing Vision-Language-Action (VLA) models and World Action Models (WAMs) typically rely on end-to-end decision-making architectures without explicit agentic reasoning. Although WAMs provide foresight through future prediction, they lack agentic closed-loop reasoning and remain vulnerable to error accumulation in long-horizon, compositional, and out-of-distribution tasks. To address this limitation, we introduce AWAM, an Agentic World Action Model whose dual-brain architecture comprises an Agentic Reasoning Brain and a World–Action Brain, integrating high-level reasoning, world modeling, and action generation. The Agentic Reasoning Brain reasons over task goals, observations, and retrieved experience to generate and refine structured intents. Conditioned on these intents, the World–Action Brain jointly predicts future world states and executable actions, distilling the predicted physical consequences of these actions into world evidence that is fed back to the Agentic Reasoning Brain. This interaction establishes a unified closed loop in which reasoning and world modeling constrain and refine each other to guide action generation. During training, AWAM generates multiple candidate intents conditioned on the same physical state and evaluates them through simulation rollouts. At inference time, AWAM adopts a single-intent execution strategy and re-deliberates when necessary. Experiments across multiple benchmarks demonstrate AWAM’s superior manipulation performance, with strong capabilities in long-horizon execution, compositional generalization, out-of-distribution robustness, and failure recovery.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Intelligence

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WyczOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an agentic framework for WAMs comprising two components: the Agentic Reason Brain and the World Action Brain. An auxiliary system supports WAM execution, and the two components are jointly trained to improve performance.

Critical strengths
  1. The work proposes an agentic method to support the WAM’s generation process, incorporating a harness tailored to the WAM’s characteristics to improve overall performance.
  2. The experimental results show substantial improvements over the baselines.
  3. The entire pipeline is jointly trained, with both systems trained together to achieve the final results.
Critical weaknesses
  1. Agentic design is a common and broadly applicable approach, and in most cases, VLAs and WAMs do not differ within this paradigm. The experiments do not compare the proposed agentic design with existing agentic designs in terms of either design differences or empirical performance.
  2. The component designed around the WAM’s ability to imagine future states exploits a distinctive advantage of WAMs. However, there is insufficient experimental evidence and visualization to demonstrate its effectiveness. This capability should also be a key distinction between harnesses designed for WAMs and those designed for VLAs.
  3. The agent design is not evaluated using different models for the Agentic Reasoning Brain, particularly more powerful closed-source models. This makes it difficult to disentangle the gains from the agentic design itself from those attributable to the underlying model’s capabilities.
  4. The real-world robot experiments lack complete video comparisons and more extensive results.
Questions for authors
  1. Compare the proposed approach with other agentic designs, highlighting the effectiveness and intermediate processes of the components specifically designed for WAMs.
  2. Provide additional visualizations showing how the WAM’s imagined futures are refined.
  3. Compare results using other language models as the Agentic Reasoning Brain, particularly powerful closed-source models.
  4. Provide more extensive demonstrations and quantitative results from real-world robot experiments.
4 comments
AuthorsOpenReview ↗

Response to Reviewer Wycz

Thank you for carefully reviewing our manuscript. We appreciate your feedback and respond to each comment below.

Q1. How does AWAM compare with existing agentic approaches in design and performance, and what distinguishes its WAM-specific harness?

R1. Thank you for the question. AWAM builds on the complementary strengths of LLMs, VLAs, and WAMs. A pretrained LLM can interpret new tasks and compose familiar operations but struggles with fine-grained contact control. VLAs and WAMs can learn precise manipulation from demonstrations, but generalizing to unfamiliar combinations of skills in long-horizon tasks remains challenging. Two contemporaneous approaches, HarnessVLA and HarnessWAM, focus on coordinating existing execution capabilities. HarnessVLA combines a frozen VLA with analytic primitives, while HarnessWAM maps plans to WAM-supported skills and uses execution feedback to revise those plans.

Inspired by how people combine familiar movements to accomplish new tasks, we introduce a Task–Skill–Primitive hierarchy for long-horizon bimanual manipulation. The LLM decomposes tasks into skills, selects primitives for each arm, and determines which operations can proceed in parallel and which must wait for the other arm to complete a step. Primitive-level annotations align the operations invoked by the planner with the demonstration segments used to train the WAM. This division allows the WAM to focus on learning basic manipulation operations and their local dynamics. For a new task, the LLM can recombine these primitives while the WAM reuses the operations it has learned. This may reduce reliance on large datasets of whole-task bimanual demonstrations.

The distinguishing feature of AWAM’s WAM-specific harness is bidirectional physical feedback through the primitive interface. Given a primitive intent from the LLM, the WAM predicts a future representation that guides action generation. The Physical Head uses the same representation to predict manipulation outcomes, interaction states, and collision risks. After execution, AWAM records these predictions alongside the observed outcomes in experience memory. The LLM can then use discrepancies between expected and observed outcomes when selecting or revising subsequent primitives.

Because we could not locate an official HarnessWAM implementation at the time of evaluation, we constructed Harness+WAM by retaining the HarnessVLA agent and replacing its VLA policy with Fast-WAM. With DeepSeek-v4.1-flash as the reasoning backbone for both systems, AWAM-Pro outperforms Harness+WAM on BiCoord by 24.8 percentage points in SR and 19.8 percentage points in SSR. Mean trajectory length on successful trials decreases from 616 to 408 steps. Table 1 reports this system-level comparison, which we have also added to Table 2 of the revised manuscript.

Table 1. Performance of agent-augmented methods on BiCoord.

MethodReasoning backboneSR (%) ↑SSR (%) ↑TL (steps) ↓
Harness+WAMDeepSeek-v4.1-flash24.440.7616
Agent + WAMQwen3.5-27B32.046.5628
AWAM (Full)Qwen3.5-27B38.253.8491
AWAM-ProDeepSeek-v4.1-flash49.260.5408

With Qwen3.5-27B held fixed, full AWAM outperforms Agent + WAM by 6.2 percentage points in SR and 7.3 percentage points in SSR. These gains reflect the combined contributions of experience memory, physical feedback, and reinforcement learning. Table 3 of the revised manuscript quantifies the incremental contribution of physical feedback. With reasoning and experience memory already enabled, adding physical feedback raises SR from 34.5% to 36.0% and SSR from 49.0% to 51.5%.

Q2. What experimental and visual evidence shows how AWAM uses predicted futures to guide decisions?

R2. Figure 6 in the revised manuscript shows two block manipulation examples, one per row. Each panel contains two consecutive frames. Panels (a) and (b) show the WAM’s predictions for the original and revised primitives, respectively. Panel (c) shows the robot executing the revised primitive.

In the top row, the WAM predicts from the gripper’s position relative to the middle block that the gripper will knock the block out of place. The planner raises the gripper’s target pose, and the updated prediction shows the gripper clearing the block. During execution, the block remains in place. In the bottom row, the original prediction includes an extra block that is absent from the scene. After the planner revises the primitive, the updated prediction shows the correct number of blocks, with the light blue block aligned directly above the purple block. The execution frames confirm this alignment.

AuthorsOpenReview ↗

Response to Reviewer Wycz

Q3. How sensitive is AWAM to the reasoning backbone, and how are backbone gains distinguished from those of the agentic design?

R3. We have added the BiCoord results for AWAM-Pro (DeepSeek-v4.1-flash) to Table 2 of the revised manuscript. A core contribution of AWAM is its Task–Skill–Primitive hierarchy for long-horizon bimanual manipulation. The LLM decomposes tasks, composes primitives, and coordinates the two arms, while the World–Action Brain focuses on learning basic operations and their local dynamics. This division allows a stronger reasoning backbone to improve how AWAM combines learned operations across tasks.

To assess sensitivity to the reasoning backbone, we compared both full configurations under the same BiCoord evaluation protocol. AWAM-Pro achieved 49.2% SR and 60.5% SSR, compared with 38.2% SR and 53.8% SSR for AWAM (Full) with Qwen3.5-27B. The DeepSeek configuration thus improved SR by 11.0 percentage points and SSR by 6.7 percentage points.

Table 3 of the revised manuscript evaluates the contributions of individual agentic components with Qwen3.5-27B held fixed. Adding experience memory increased SR and SSR by 2.5 percentage points each. Adding physical feedback yielded further gains of 1.5 percentage points in SR and 2.5 percentage points in SSR. We have not yet evaluated a GPT backbone because repeated API calls during online training and evaluation would incur substantial costs.

Table2 BiCoord performance of full AWAM with different reasoning backbones

Reasoning backboneSR (%) ↑SSR (%) ↑TL (steps) ↓
Qwen3.5-27B38.253.8491
DeepSeek-v4.1-flash49.260.5408

Q4. Can the authors provide more real-world demonstrations, complete videos, and quantitative comparisons?

R4. We have expanded the real-world evaluation from two tasks to five by adding Arithmetic and Logic, Chessboard Setup, and Letter Formation to the original Block Stacking and Number Sorting tasks. Table 5 of the revised manuscript compares AWAM and π0.5\pi_{0.5} on each task and reports their mean success rates across all five tasks. The supplementary material includes uninterrupted videos of AWAM performing each task. We have included uninterrupted videos of AWAM performing all five tasks in the supplementary material, with playback at 1.5× speed throughout.

Table 3. Success rates (%) on five real-world manipulation tasks

MethodBlock Stacking (%)Number Sorting (%)Arithmetic and Logic (%)Chessboard Setup (%)Letter Formation (%)Avg. (%)
π0.5\pi_{0.5}(22/30) 73.3(18/30) 60.0(13/30) 43.3(21/30) 70.0(18/30) 60.061.3
AWAM(25/30) 83.3(23/30) 76.7(26/30) 86.7(25/30) 83.3(23/30) 76.781.3

Dear Reviewer,

Thank you for the time and care you have devoted to reviewing our work. As the discussion period draws to a close, we would be grateful if you could take another look at our responses and revised manuscript when your schedule permits. Your feedback has helped us improve the paper, and we would welcome any further questions or concerns.

We appreciate your time and consideration.

Best regards, The Authors

Dear Reviewer,

With the discussion period closing in a few hours, we would be sincerely grateful if you could revisit our responses and revised manuscript, if your schedule permits, and consider whether they address your concerns and warrant an updated assessment.

We appreciate the time and care you have devoted to reviewing our work. Your feedback has helped us improve the paper, and we would be happy to clarify any remaining questions.

Thank you again for your thoughtful consideration.

Best regards,
The Authors

Reviewer_U4AuOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper highlights a missing closed loop among reasoning, world modeling, and action; i.e., reasoning should identify futures worth imagining, while world modeling should assess their physical feasibility.

To address this, the authors propose AWAM, a dual-brain architecture with an Agentic Reasoning Brain and a World-Action Brain:

  1. The Agentic Reasoning Brain generates structured physical intents from goals, observations, and retrieved experience; whereas,
  2. World-Action Brain jointly predicts future world states and executable actions conditioned on the intents, and feeds back physical evidence to refine high-level decisions.

The authors evaluate AWAM across two main benchmarks and show its superiority to the baselines.

Critical strengths
  • Bidirectional Interaction Between Reasoning and Modeling Brains sounds interesting as it unifies high-level reasoning, world modeling, and action prediction; which allows conditioning world-action brain on relevant intents and revising decisions using predicted consequences
  • The AWAM offers a framework for recovery from failures
Critical weaknesses
  • Frozen Reasoning Brain: I am wondering what's the motivation behind this, as it may impact the overall performance.
  • Computation Cost: AWAM leverages various models, including reasoning and world models, in its architecture. All and all, this increases the computation overhead, which may limits any real world application.
  • Future Context and Aggregation (Lines 286-290): What is Fω\mathcal{F}_{\omega}? Does this lead to any information loss?
Questions for authors
  • Why didn't you include any WAMs in your BiCoord experiments (referring to Table 2).
  • Please move the specifications of your underlying models to the main content, i.e., Qwen3.5 for Agentic Reasoning, Wan2.2-5B for WM, etc.

Some other notes:

  • What's ztRz_t^R in Fig 3? Is it a typo? Shouldn't it be ztjz_t^j?
  • Please review all your mathematical notations and equations and ensure they are consistent.
  • Please check your figure/equation/table labels and references, they are missing, e.g., lines 184, 237, and 256.
  • Fix the typo on line 223, the it takes
  • Use the correct citation commands - there are inconsistencies for them too
4 comments
AuthorsOpenReview ↗

Response to Reviewer U4Au

Thank you for carefully reviewing our manuscript. We appreciate your feedback and respond to each comment below.

Q1 Rationale for freezing the Agentic Reasoning Brain and its effect on performance

R1. We freeze the Agentic Reasoning Brain to leverage the pretrained LLM’s ability to reason across tasks and compose familiar operations, while concentrating learning from robot demonstrations in the World–Action Brain (WAM). This division of responsibilities draws on how people plan sequences of familiar movements to accomplish new tasks. Our Task–Skill–Primitive hierarchy and primitive-level annotations link each primitive the LLM invokes to the demonstration segments used to train the WAM. The LLM decomposes tasks into skills and coordinates primitives across both arms, while the WAM learns to execute each primitive and predict its local dynamics. By reusing primitives across task combinations, AWAM aims to reduce the need for large datasets of whole-task demonstrations.

Even with its parameters frozen, the LLM can revise plans using physical feedback and retrieved experience. The reasoning brain sends a primitive intent to the WAM, which predicts a future representation conditioned on that intent. The Physical Head uses this representation to predict physical outcomes. AWAM combines these predictions with observed execution outcomes to update experience memory. The reasoning brain then uses feedback and retrieved experience to select or revise subsequent primitives.

With the reasoning brain frozen, full AWAM achieves an SR of 38.2% and an SSR of 53.8% on BiCoord. Table 3 in the main text shows that, with the same frozen backbone, adding experience memory raises SR from 32.0% to 34.5% and SSR from 46.5% to 49.0%. These gains quantify the contribution of memory under a frozen backbone.

Q2 Computational overhead, inference latency, and real-world deployment

R2. In our Piper X deployment, AWAM uses a cloud-hosted LLM for high-level planning and runs the World–Action Brain on a single NVIDIA RTX 4090, with peak local GPU memory usage of 23.5 GB. The mean end-to-end latency from receiving an observation to generating a 32-step action chunk was 1,750 ms, including the API round trip and local WAM inference. The robot executed the first 28 steps of each chunk before requesting the next. With this configuration, AWAM completed all five tabletop tasks reported in Section 4.3.

Table 1. Inference latency across WAM and agentic control paradigms. Hardware and timing scopes vary by method.

MethodParadigmDeploymentLatency (ms)
Fast-WAM [1]WAMNVIDIA H20470
GigaWorld-Policy [2]WAMNVIDIA A100360
Cosmos-Policy [3]WAMNVIDIA A1001,413
Motus [4]WAMNVIDIA A1003,231
HarnessVLA [5]Agent + VLACloud Agent + RTX 40902,136
AWAM (Ours)Agentic WAMCloud Agent + RTX 40901,750

The Fast-WAM latency is from its official implementation, and the A100 measurements are from the GigaWorld-Policy paper. We measured HarnessVLA and AWAM in our own experiments. As shown in Table 1, policy inference latency ranges from 360 to 3,231 ms for the three WAMs evaluated on an A100. For AWAM, the reported 1,750 ms covers the cloud planning call. The full observation-to-action-chunk latency also includes local action generation.

Efforts to accelerate cloud inference for robotics are already underway. Google DeepMind’s Gemini Robotics uses a distilled cloud VLA backbone with a response latency of under 160 ms. Combined with an on-robot action decoder, the system produces an action chunk approximately 250 ms after receiving an observation. Similar advances in model distillation and inference optimization could shorten the delay between action chunks in systems such as AWAM.

References

[1] T. Yuan et al., “Fast-WAM: Do world action models need test-time future imagination?” accepted for presentation at the Conf. Neural Information Processing Systems (NeurIPS), 2026.

[2] C. Ni et al., “GigaWorld-Policy: An efficient action-centered world–action model,” in Proc. 19th Eur. Conf. Computer Vision (ECCV), 2026, pp. 315–333, doi: 10.1007/978-3-032-37553-7_18.

[3] M. J. Kim et al., “Cosmos Policy: Fine-tuning video models for visuomotor control and planning,” in Proc. Int. Conf. Learning Representations (ICLR), 2026.

[4] H. Bi et al., “Motus: A unified latent action world model,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2026, pp. 35101–35113.

[5] Y. Zhang et al., “Harness VLA: Steering frozen VLAs into reliable manipulation primitives via memory-guided agents,” accepted at the CoRL 2026 Workshop on Agentic Robotics, Austin, TX, USA, 2026.

AuthorsOpenReview ↗

Response to Reviewer U4Au

Q3: Future context aggregation and potential information loss

R3: In lines 286–290, FωF_\omega denotes the world-feature aggregation module parameterized by ω\omega. Given the world representation WtW_t predicted for the current manipulation intent, the module uses Interval KV-Fusion to aggregate features across groups of world-model layers. The resulting future context ft=Fω(Wt)f_t = F_\omega(W_t) conditions action denoising at eight interaction stages in the 30-layer action Transformer (Appendix A.1, Table 6).

The aggregation is designed to retain predicted dynamics relevant to action generation. Because FωF_\omega is not constrained to be invertible, aggregation may discard fine-grained details of WtW_t. For a fixed WtW_t, AWAM computes ftf_t once and reuses it at each denoising step, so caching introduces no additional information loss. AWAM recomputes ftf_t when the observation or manipulation intent changes.

Q4: WAM baselines on BiCoord

R4: At submission, we were unaware of any officially reported BiCoord results for WAMs, so Table 2 included the baselines from the original BiCoord paper. We have since trained and evaluated Fast-WAM using the same BiCoord training data and evaluation protocol as AWAM. The table below summarizes overall performance. We have added a task-by-task comparison of Fast-WAM and AWAM across all 18 BiCoord tasks to Table 2 of the main manuscript, where we identify the Fast-WAM results as our own measurements.

Table1. Comparison of Fast-WAM and AWAM on BiCoord.

MethodSR (%) ↑SSR (%) ↑TL (steps) ↓
Fast-WAM29.243.4474
AWAM (Qwen-3.5)38.253.8491
AWAM -Pro (DeepSeek-v4.1-flash)49.260.5408

Q5: Model specifications in the main text

R5. We have added the model specifications to the Implementation Details paragraph in Section 4. The Agentic Reasoning Brain uses a frozen Qwen3.5-27B backbone in AWAM and a frozen DeepSeek-V4.1-Flash backbone in AWAM-Pro. The World–Action Brain initializes its video DiT, text encoder, and VAE with pretrained weights from Wan2.2-5B. Its 30-layer action Transformer has a hidden dimension of 1024 and uses the predicted future context and current robot state to generate 32-step action chunks.

Q6 Notation, cross-references, typo, and citation commands

R6: Thank you for pointing out these issues. The superscript RR in Fig. 3 was a typo. We have replaced ztRz_t^R with ztjz_t^j, where jj indexes the deliberation round at decision step tt, consistent with Section 3.2. We have checked the notation across the text, figures, and equations and corrected inconsistent symbols and indices. We have also checked all figure, table, and equation labels and restored the missing references at lines 184 (Fig. 2), 237 (Fig. 4), and 256 (Eq. 4) of the original submission. Finally, we have corrected “the it takes” at line 223 and standardized the citation commands throughout the manuscript.

I thank the authors for addressing my questions and comments. I wish to see the paper at the conference; at the same time, I intend to keep my scores. I request the authors to reflect their answers to my questions/comments in the camera-ready version upon acceptance. Thank you.

We sincerely thank you for your support and for the time and care you have devoted to reviewing our work. Your thoughtful questions and constructive suggestions have helped us strengthen both the paper and the clarity of our presentation. If the paper is accepted, we will ensure that all the clarifications and revisions discussed in our responses are fully incorporated into the camera-ready version.

Reviewer_SdigOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AWAM, a dual-brain architecture that combines agentic reasoning with world-action modeling for robotic manipulation. A frozen reasoning model generates structured intents, while a world-action model predicts physical consequences and produces executable actions. The system also uses experience memory and physical feedback to refine decisions during execution. Training combines supervised learning with reinforcement learning from simulated rollouts. Experiments on RoboDojo and BiCoord show improved task success, particularly for long-horizon manipulation and bimanual coordination.

Critical strengths

I think connecting high-level reasoning with predictive world modeling is a useful direction. The proposed architecture provides a reasonable way to incorporate physical feedback into task planning, which could be particularly valuable for long-horizon manipulation.

The experimental results are promising across both benchmarks. The improvements on BiCoord are particularly encouraging, and the component ablations provide some evidence that reasoning and physical feedback contribute to the overall performance.

Critical weaknesses

I am not convinced that the proposed dual-brain interaction is the main source of improvement. AWAM introduces a large frozen reasoning model alongside several additional components, but the comparisons focus primarily on standalone VLA and WAM models. A stronger comparison with existing agent-augmented methods would help establish the novelty and effectiveness of the proposed interaction mechanism.

The paper provides limited direct evidence that world prediction actually improves reasoning. Although the ablations show gains from physical feedback, they do not isolate predictive world evidence from ordinary execution feedback. There is also little quantitative analysis of prediction accuracy or how often the model successfully revises its plans. This makes the central closed-loop reasoning claim difficult to assess.

Questions for authors

How does AWAM compare with an agent-augmented WAM using the same reasoning backbone but without the proposed bidirectional interaction? This would help isolate the central contribution.

Could the authors compare predicted physical feedback against ordinary execution feedback and report the accuracy of the Physical Head?

6 comments
AuthorsOpenReview ↗

Response to Reviewer Sdig

Thank you for carefully reviewing our manuscript. We appreciate your feedback and respond to each comment below.

W1 & Q1. How does AWAM compare with agent-augmented baselines, and how much does bidirectional interaction contribute when the reasoning backbone is held fixed?

R1. Thank you for the suggestion. AWAM’s overall improvement reflects several design choices. Our two main contributions are a Task–Skill–Primitive hierarchy for long-horizon bimanual manipulation and bidirectional physical feedback through the primitive interface.

First, AWAM builds on the complementary strengths of LLMs, VLAs, and WAMs. A pretrained LLM can interpret new tasks and compose familiar operations but struggles with fine-grained contact control. VLAs and WAMs can learn precise manipulation from demonstrations, but generalization to unfamiliar long-horizon tasks and skill combinations remains challenging. HarnessVLA and HarnessWAM, two contemporaneous approaches, focus on coordinating existing execution capabilities. HarnessVLA combines a frozen VLA with analytic primitives, while HarnessWAM maps high-level plans to skills supported by an existing WAM.

Inspired by how people combine familiar movements to accomplish new tasks, we propose a Task–Skill–Primitive hierarchy for long-horizon bimanual manipulation. The hierarchy assigns task decomposition, primitive composition, and bimanual coordination to the LLM, simplifying the role of the World–Action Brain (WAM) so that it can focus on learning and executing basic operations. We annotate whole-task demonstrations at the primitive level to align the primitives invoked by the planner with the action segments used to train the WAM. From these segments, the WAM learns the local dynamics of each primitive and the actions needed to execute it. For a new task, the LLM recombines primitives according to dependencies between the two arms, while the WAM reuses the operations it has learned. This offers a complementary route to embodied generalization that may reduce reliance on large demonstration datasets covering complete bimanual task combinations.

Second, bidirectional physical feedback allows the LLM to use predicted consequences when revising its plans. The LLM sends a primitive intent to the WAM, which predicts a future representation that guides action generation. The Physical Head uses the same representation to predict manipulation outcomes, interaction states, and collision risks. After execution, AWAM records these predictions alongside the observed outcomes in experience memory. The LLM can then draw on discrepancies between expected and observed outcomes when selecting or revising subsequent primitives.

For a direct comparison with an agent-augmented method, we constructed Harness+WAM by retaining the HarnessVLA agent and replacing its VLA policy with Fast-WAM, as we could not locate an official HarnessWAM implementation at the time of evaluation. With DeepSeek-v4.1-flash as the reasoning backbone for both systems, AWAM-Pro (Full) outperforms Harness+WAM on BiCoord by 24.8 percentage points in SR and 19.8 percentage points in SSR. These results quantify the overall performance difference with the reasoning backbone held fixed. Table 1 reports this comparison, which we have also added to Table 2 of the revised manuscript.

Table 1. Performance of agent-augmented methods on BiCoord.

MethodReasoning backboneSR (%) ↑SSR (%) ↑TL (steps) ↓
Harness+WAMDeepSeek-v4.1-flash24.440.7616
Agent + WAMQwen3.5-27B32.046.5628
AWAM (Full)Qwen3.5-27B38.253.8491
AWAM-ProDeepSeek-v4.1-flash49.260.5408

With Qwen3.5-27B held fixed, AWAM (Full) outperforms Agent + WAM by 6.2 percentage points in SR and 7.3 percentage points in SSR. These gains reflect the combined contributions of experience memory, physical feedback, and reinforcement learning. Table 3 of the main manuscript quantifies the incremental contribution of physical feedback. With reasoning and experience memory already enabled, adding physical feedback raises SR from 34.5% to 36.0% and SSR from 49.0% to 51.5%.

AuthorsOpenReview ↗

Response to Reviewer Sdig

W2 & Q2: Contribution of predictive evidence beyond execution feedback, prediction accuracy, and plan revision success rate

R2: In AWAM, the action policy generates actions from the future representation predicted by the WAM. The Physical Head uses this representation to predict task outcomes, interaction states, and collision risks. Under the default single-intent setting (K = 1), AWAM combines these predictions with observed execution outcomes to construct feedback and update experience memory. The reasoning brain uses this feedback and retrieved experience when selecting or revising subsequent primitives.

Table 3 of the manuscript evaluates the overall contribution of the Physical Head by removing it and retraining the model. With reasoning and experience memory enabled, adding the Physical Head increases SR from 34.5% to 36.0% and SSR from 49.0% to 51.5%.

To isolate the contribution of predictions to memory updates, we use the same pre-RL AWAM checkpoint, initial memory, test seeds, and execution budget in both conditions. Only the information written to memory varies. One condition records observed execution feedback alone, and the other also records Physical Head predictions. Both use the same mechanisms for action generation, explicit feedback, replanning, and memory retrieval.

Table 2. Effect of the information used to update experience memory on BiCoord performance

Information used to update memorySR (%) ↑SSR (%) ↑TL (steps) ↓
Execution feedback only34.949.5542
Execution feedback + Physical Head predictions36.051.5504

We evaluate the Physical Head on held-out BiCoord rollouts by comparing each prediction recorded before execution with the outcome observed over the same time horizon. For each prediction target, accuracy is calculated over executed action segments with a valid label for that target.

Table 3. Physical Head prediction accuracy on held-out BiCoord action segments

Prediction targetAccuracy (%)
Task outcome97.5
Forbidden collision85.1
Interaction formation78.4

To quantify how often plan revisions succeeded, we manually reviewed 100 revisions from BiCoord test rollouts. We counted a revision when the planner changed the selected primitive or its arguments while pursuing the same subtask goal. In 88 of the 100 cases, executing the revised primitive achieved that goal without another revision, yielding an observed revision success rate of 88%.

Figure 6 in the revised manuscript illustrates two examples of plan revision. Each example shows the WAM’s predictions for the original and revised primitives, alongside the observed outcome after the revised primitive is executed.

Thanks for providing the agent-augmented comparison. This addresses the baseline concern in Weakness 1. For Weakness 2 : adding predictions to memory raises SR from 34.9% to 36.0%, but their role in plan revision remains unclear to me, i.e. whether the execution-only condition still exposes predictions to the planner through another route.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Kind Request to Review Our Responses and Continue the Discussion

Dear Reviewers,

Thank you for your time and thoughtful feedback. We would appreciate it if you could take a look at our responses and the updates to the manuscript when you have a chance. We would be very happy to discuss any remaining concerns and provide further clarification. Thank you again for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.