acceptodds
Under review as a conference paper at ICLR 2027

HarnessWAM: Bridging Prediction and Deliberation in World Action Models

Abstract

World Action Models (WAMs) jointly learn environmental dynamics and robot actions, introducing priors over physical evolution into embodied control. However, finite-horizon prediction and action generation are insufficient for complex embodied tasks that require global planning, cross-stage state maintenance, execution verification, and failure recovery. We refer to this mismatch as the prediction-deliberation gap. To address this gap, we propose HarnessWAM, an agentic framework for WAMs. HarnessWAM employs a vision-language-model-based Task Manager to maintain an evidence-grounded scene belief and organize the global instruction into a structured task graph. Capability-conditioned executable-space projection then compiles the graph into WAM-supported skill sequences that satisfy task dependencies and embodiment-state constraints. During execution, HarnessWAM couples a lightweight progress estimator with event-triggered Task Manager deliberation. The estimator supplies frequent execution cues, while the Task Manager uses observations, task state, and interaction history to verify outcomes and decide whether to advance, acquire evidence, revise the plan, or recover. Local recovery preserves acquired scene knowledge so that execution can resume after failure. HarnessWAM achieves state-of-the-art average performance on RoboMemArena, with full-task and subtask success rates of 64.4% and 76.8%, respectively, and reaches 23.7% success on RoboCerebra Ideal. On a physical UR3 arm, full-task success reaches 95% for shape-conditioned tabletop tidying and 90% for memory-dependent box exploration. These results demonstrate the effectiveness of model-external agentic orchestration in bridging the prediction-deliberation gap for complex embodied tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World action model, agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NEcfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents HarnessWAM, an external agentic framework for organizing a World Action Model’s local execution capabilities into long-horizon task execution. A VLM-based Task Manager maintains scene beliefs, evidence memory, and a task graph with partially bound variables. A capability-conditioned compiler maps proposed operations to supported skills while checking represented dependencies and state constraints. During execution, progress events trigger outcome verification, task-state updates, revision of the unexecuted plan suffix, and recovery that preserves previously acquired evidence. The framework is evaluated on RoboMemArena, RoboCerebra Ideal, and two UR3 manipulation tasks.

Critical strengths
  1. The framework connects evidence memory, delayed target binding, task dependencies, and recovery within a consistent execution process. This design is well motivated for partially observable tasks, where acquired evidence must remain available for subsequent planning and manipulation.

  2. The same-checkpoint comparison improves full-task success from 52.5% with static planning to 64.4% with the complete framework . Results across two WAM backbones and two physical tasks further support the usefulness of the overall system, although they do not independently establish methodological novelty.

Critical weaknesses
  1. Limited methodological differentiation from existing agentic frameworks. Memory-guided skill orchestration is central to Harness VLA [1], while explicit state maintenance, precondition checking, and feedback-driven planning have precedents in Statler [2], ProgPrompt [3], and Inner Monologue [4], respectively. Although [1] is described as concurrent work, these broader precedents also warrant clearer differentiation. HarnessWAM’s task graph, constrained compilation, and suffix revision are concrete design choices, but their substantive advance over established orchestration principles remains insufficiently demonstrated. It is also unclear whether deliberation exploits WAM-specific predictive capabilities or treats the WAM primarily as a generic skill controller. The paper includes agentic comparisons, notably Harness VLA [1], HPE [5], and a free-form control. However, insufficient adaptation details, differences in model configurations, and the limited structure of the free-form control make attribution difficult. A matched comparison with a faithfully adapted [1] or [5], using the same planner, controller, and comparable budgets where feasible, would better isolate the proposed mechanisms.

  2. Deployment overhead and latency are insufficiently characterized. The framework introduces additional VLM calls for planning, verification, and recovery, but the available manuscript does not quantify their latency, frequency, or effect on physical execution. Event-triggered scheduling may reduce overhead, and fewer failures could offset additional computation; however, success rates under environment-step budgets do not establish this trade-off. The paper should report Task Manager latency and call counts, whether deliberation blocks execution, achieved action-update frequency, and end-to-end task time against direct WAM execution and a matched agentic baseline. These measurements are important for assessing practical responsiveness and throughput.

Questions for authors
  1. What specific capability or consistency property does the proposed task graph and suffix revision provide beyond the orchestration mechanisms in [1–4]? Does high-level deliberation explicitly use WAM predictive outputs? Which components would require substantive changes if the WAM were replaced by a VLA exposing a comparable skill interface?

  2. How does end-to-end execution time compare with direct WAM execution and a matched agentic baseline on the physical tasks?

References

[1] Yixian Zhang et al. Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents. arXiv:2607.08448, 2026. arXiv

[2] Takuma Yoneda et al. Statler: State-Maintaining Language Models for Embodied Reasoning. IEEE International Conference on Robotics and Automation (ICRA), 2024. arXiv:2306.17840. arXiv

[3] Ishika Singh et al. ProgPrompt: Program Generation for Situated Robot Task Planning Using Large Language Models. Autonomous Robots, 47:999–1012, 2023. DOI: 10.1007/s10514-023-10135-3. Springer

[4] Wenlong Huang et al. Inner Monologue: Embodied Reasoning through Planning with Language Models. arXiv:2207.05608, 2022. arXiv

[5] Songhao Han et al. RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation. NeurIPS, 2025. arXiv:2506.06677. The relevant comparator is its Hierarchical Planning and Execution (HPE) framework. arXiv

5 comments

Thank you for recognizing the consistent execution design connecting evidence memory, deferred binding, task dependencies, and recovery. Below, we address your concerns about the methodological contribution and the focus on WAMs through the properties of the framework, ablations with a fixed WAM, and additional matched experiments.

W1. Relationship to existing agentic frameworks and our contribution

Our core contribution is to identify and address the capability gap that WAMs face when moving from local prediction and action execution to long-horizon tasks. WAMs can generate local actions, but long-horizon tasks also require identifying targets from earlier observations, organizing operations under skill and state constraints, and continuing the remaining task after failure. HarnessWAM builds an external harness around these shortcomings, providing evidence across stages, executability constraints, task updates grounded in feedback, and progress-driven scheduling. Experiments with a fixed WAM examine the need for and effectiveness of this task-level support. To our knowledge, this is the first work to systematically investigate and address WAM shortcomings in long-horizon tasks through a harness.

The works you cite, Statler [1], ProgPrompt [2], Inner Monologue [3], and Harness VLA [4], provide foundations for state maintenance, precondition checking, feedback-driven planning, and memory-guided orchestration. Evaluating our work solely by whether these components have precedents would overlook the WAM problem we study and the solution we provide. Our research asks what continues to limit long-horizon tasks once a WAM has acquired local skills, and whether a harness can effectively address those limitations. Evidence-grounded binding, executable-space projection, and suffix revision that preserves history address target identification across stages, skill executability, and task continuity after failure, respectively, making these problems explicit and experimentally testable.

With the same WAM checkpoint held fixed, Static Plan achieves a full-task success rate of 52.5%, while HarnessWAM reaches 64.4%, showing that task-level shortcomings remain after initial skill decomposition. Full projection raises the normalization-only success rate from 54.0% to 64.4%, demonstrating the need to address execution constraints even after instruction expressions have been standardized. Retaining memory, binding, verification, and recovery while disabling only suffix revision reduces success to 57.1%, further demonstrating the need to adapt the remaining task to execution outcomes. Together, these experiments identify specific bottlenecks in long-horizon WAM execution and validate the harness's effectiveness in addressing them.

The lightweight progress estimator addresses the coordination needed between continuous WAM execution and high-level task decisions. It dynamically tracks subtask progress and triggers VLM decisions through execution events, reducing the number of VLM calls needed for continuous progress checking. This links task-level intervention to actual execution progress. We ask that our contribution be assessed in terms of the WAM problems identified and addressed, focusing on how the harness extends existing local prediction and action capabilities into sustained, reliable long-horizon task execution.

W2. Comparison with existing methods using the same Planner and Controller

We have added comparisons with HPE and PrediMem. All three methods use the same LingBot-VA checkpoint trained on RoboMemArena subtasks and Qwen3-VL-32B-Instruct as the high-level Planner. This enables a fair comparison of orchestration methods while holding both the underlying skill model and the high-level planning model fixed.

Orchestration methodTransfer full-task SROcclusion full-task SRCounting full-task SRSequence full-task SRAverage full-task SRAverage subtask SR
HPE16.362.754.371.354.671.4
PrediMem21.366.855.075.057.973.1
HarnessWAM22.576.859.381.364.476.8

HarnessWAM achieves absolute gains in average full-task SR of 9.8% over HPE and 6.5% over PrediMem. In Occlusion, which strongly depends on evidence across stages, the three methods achieve 62.7%, 66.8%, and 76.8%, respectively. The added comparison directly examines the gains over existing structured methods, while Table 4 of the paper connects these gains to specific execution mechanisms. Together, these results show that addressing WAM task-level shortcomings requires targeted execution support, and that HarnessWAM effectively addresses these issues with the underlying model held fixed.

W3. High-level reasoning overhead and execution efficiency

We have added efficiency measurements on two real-world tasks. Compared with PrediMem + LingBot-VA, HarnessWAM reduces end-to-end task time by 15.1% on Tabletop Tidying and 14.0% on Box Exploration. The time cost of high-level reasoning is an important deployment challenge for existing robotic agent methods. HarnessWAM addresses this issue through lightweight progress estimation and event-triggered scheduling that reduce high-level calls. The following real-world comparisons directly quantify the resulting efficiency improvement.

Real-world taskMethodAverage VLM calls / episodeAverage latency per VLM call (ms)Does high-level reasoning block execution?Achieved action-update frequency (Hz)Average end-to-end task time (s)
Tabletop TidyingDirect WAM: LingBot-VA———1.01337
Tabletop TidyingPrediMem + LingBot-VA213,814Yes0.81417
Tabletop TidyingHarnessWAM5.33,198Yes0.96354
Box ExplorationDirect WAM: LingBot-VA———1.02405
Box ExplorationPrediMem + LingBot-VA263,964Yes0.81508
Box ExplorationHarnessWAM8.63,695Yes0.94437

In this real-world deployment, high-level reasoning blocks execution in both PrediMem and HarnessWAM, and each VLM call takes several seconds. Reducing the number of calls is therefore particularly important for controlling the overall time overhead. HarnessWAM reduces the average call counts on the two tasks from 21 and 26 to 5.3 and 8.6, respectively, corresponding to reductions of 74.8% and 66.9%. These results further demonstrate the role of the lightweight progress estimator: it continuously tracks execution progress and concentrates VLM calls on events requiring task-level judgment, addressing long-horizon WAM shortcomings while controlling reasoning overhead.

Q1. Consistency of the task graph and suffix revision, predictive outputs, and VLA adaptation

The task graph requires target-dependent operations to execute only after evidence-grounded binding, dependency satisfaction, and fulfillment of preconditions. Node verification outcomes determine subsequent advancement. Suffix revision follows Equation (4) in the paper, Gk+1[Vkexecuted]=Gk[Vkexecuted]G_{k+1}[V_k^{\mathrm{executed}}]=G_k[V_k^{\mathrm{executed}}], preserving the operation definitions and dependencies of executed nodes. Scene beliefs are updated from current observations, while unexecuted operations are reorganized and projected. Executed operations and failed attempts are therefore retained as history, while future plans adapt to the actual state. This property corresponds to the recovery example and suffix ablation discussed in W1. Regarding predictive outputs, a WAM's future predictions may differ from actual execution outcomes, and planning directly from them could affect subsequent target and operation choices. The high-level system therefore combines real observations, evidence memory, the task graph, and progress signals to make decisions and update the remaining plan based on actual execution outcomes. The current implementation does not directly read WAM-predicted frames. Our experiments examine the task-level execution shortcomings that remain in a local controller with world-modeling capabilities, and how an external harness can address them.

We have connected the same harness to π₀.₅ trained on RoboMemArena subtasks, achieving a full-task success rate of 44.6% and a subtask success rate of 49.7%. The replacement is at the level of the underlying skill executor, using the same action primitives, task-switching strategy, and budgets. High-level evidence maintenance, task representation, verification, and suffix-update logic remain the same. The following table compares the original results for both controllers with their results under the same harness.

SettingController typeFull-task SR (%)Subtask SR (%)
Original LingBot-VAWAM49.060.0
LingBot-VA + same HarnessWAM64.4 (+15.4)76.8 (+16.8)
Original π₀.₅VLA21.538.7
π₀.₅ + same HarnessVLA44.6 (+23.1)49.7 (+11.0)

Q2. Real-world end-to-end task time compared with Direct WAM and a matched agentic baseline

As shown in the real-world efficiency table in W3, HarnessWAM takes 354 s on Tabletop Tidying and 437 s on Box Exploration, reducing PrediMem's times of 417 s and 508 s by 63 s and 71 s, respectively. Compared with Direct WAM's 337 s and 405 s, the additional times are 17 s and 32 s, corresponding to overheads of 5.0% and 7.9%. Alongside these results, the real-world success rates reported in the paper increase from Direct WAM's 30.0% and 5.0% to HarnessWAM's 95.0% and 90.0%, respectively. These results show that HarnessWAM improves WAM task completion with modest additional time costs and achieves clearly better time efficiency than PrediMem in this matched real-world comparison.

These experiments clarify the capability gaps of WAMs in long-horizon tasks and validate the harness's effectiveness in addressing them. In light of the success-rate gains with a fixed WAM and the real-world efficiency results, we respectfully ask you to reassess our contribution in terms of addressing WAM shortcomings and providing the task-level support they need.

Reviewer_KhmFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a harness for World Action Models, called HarnessWAM, that extends local action generation to long-horizon task execution. A VLM-based Task Manager maintains scene beliefs, memory, and a structured task graph, while a deterministic compiler maps proposed operations to supported skills and checks execution constraints. During execution, a lightweight progress estimator triggers semantic verification, plan revision, and failure recovery while preserving previously acquired evidence.

Critical strengths
  1. Practical integration of planning and execution. HarnessWAM addresses an important limitation of finite-horizon action models by connecting persistent task state, executable skill constraints, outcome verification, and recovery. This integration is particularly relevant to tasks where later actions depend on earlier observations or successful completion of prerequisites.

  2. Substantial improvements with component-level validation. The framework improves performance across two WAM backbones on RoboMemArena and achieves large gains on two physical manipulation tasks. Ablations using the same subtask-trained checkpoint provide evidence that the benefits extend beyond task decomposition, particularly highlighting executable-space projection and event-triggered execution control.

Critical weaknesses
  1. Limited differentiation from existing embodied-agent frameworks. High-level language-model planning, persistent memory, skill preconditions, execution verification, and recovery have substantial precedents. Although the paper compares against Harness VLA and HPE, these comparisons do not isolate the orchestration method from differences in underlying controllers and skill interfaces. Comparisons that adapt established structured orchestration methods to the same WAM checkpoint would better establish whether HarnessWAM offers a substantive methodological advance beyond integrating known components.

  2. Insufficient evidence that the contribution is specific to world-action modeling. The harness primarily uses observations, symbolic task state, progress estimates, and language-conditioned skill execution; the method does not clearly establish how predicted future-world representations contribute to its decisions. Without a matched comparison using a non-world-model action controller, it remains unclear whether the framework specifically bridges prediction and deliberation or provides a general hierarchical control wrapper. This uncertainty weakens the paper’s central WAM-specific positioning.

Questions for authors

Please refer to the weaknesses above.

3 comments

Thank you for recognizing the importance of the problem, the integration of planning and execution, and the mechanism validation using the same checkpoint. We have added comparisons with existing orchestration methods and a non-WAM controller to address your two concerns about the methodological contribution and research focus.

W1. Differences from existing orchestration frameworks and matched comparisons

Our core contribution is to identify and address the capability gap that WAMs face when moving from local prediction and action execution to long-horizon tasks. HarnessWAM provides an external harness to address target identification across stages, skill executability, and task continuity after failure, supplying the task-level support that WAMs need. Target-dependent operations wait for binding supported by observational evidence. Projection translates high-level operations into calls that satisfy the WAM skill interface, dependencies, and holding-state requirements. Execution outcomes determine whether to advance or revise the unexecuted plan. The key issue is how new evidence and failures change subsequent operations, and how the revised plan satisfies execution constraints again.

The combination of a lightweight progress estimator and event-triggered task management is another key execution design. During WAM execution, the estimator dynamically tracks the current subtask's progress and completion trend. Progress milestones or exhaustion of the skill budget trigger the VLM Task Manager to perform semantic verification and decide whether to advance, revise, or recover. The lightweight model handles frequent progress tracking, concentrating VLM calls on events that require task-level judgment and reducing the number of calls needed for continuous progress checking. This design organizes execution monitoring and task decisions at different time scales, closing the loop between local WAM execution and high-level feedback.

The paper already examines this role with the same WAM. Static Plan achieves 52.5%, compared with 64.4% for the full framework. Keeping memory, binding, verification, and recovery while freezing only the suffix structure yields 57.1%. This demonstrates the independent practical value of sustained task management and online plan revision.

We have also added comparisons in which HPE and PrediMem use the same underlying skill executor. All three methods share the same LingBot-VA checkpoint trained on RoboMemArena subtasks and use Qwen3-VL-32B-Instruct as the high-level Planner.

Orchestration methodTransfer full-task SR (%)Occlusion full-task SR (%)Counting full-task SR (%)Sequence full-task SR (%)Average full-task SR (%)Average subtask SR (%)
HPE16.362.754.371.354.671.4
PrediMem21.366.855.075.057.973.1
HarnessWAM22.576.859.381.364.476.8

With the underlying WAM and Planner held fixed, HarnessWAM achieves absolute full-task SR gains of 9.8% over HPE and 6.5% over PrediMem, with higher success rates in all four categories. These results show that, even with the same underlying WAM and Planner, a harness designed around evidence across stages, execution constraints, and task updates can substantially improve long-horizon WAM performance.

W2. The focus on WAMs and the comparison with a controller without a world model

Our central research question is which task-level shortcomings remain after a WAM acquires world-modeling and local action capabilities, and how an external harness can address them. We define the prediction–deliberation gap as the gap between local prediction and action capabilities and sustained task-level decision-making. Experiments with a fixed WAM directly examine this gap and its improvement. Since the future states predicted by a WAM may differ from actual execution outcomes, the Task Manager adjusts subsequent plans using real observations and execution feedback to prevent prediction errors from leading planning away from the actual state. The current implementation therefore does not directly read predicted future frames.

We have added a comparison using a non-WAM controller. π₀.₅ uses weights trained on RoboMemArena subtasks and shares the same action primitives, task-switching strategy, and budgets as LingBot-VA. The high-level task representation, memory, Planner, verification, and plan-revision logic remain the same.

SettingController typeFull-task SR (%)Subtask SR (%)
Original LingBot-VAWAM49.060.0
LingBot-VA + same HarnessWAM64.4 (+15.4)76.8 (+16.8)
Original π₀.₅VLA21.538.7
π₀.₅ + same HarnessVLA44.6 (+23.1)49.7 (+11.0)

This result demonstrates the applicability of HarnessWAM to a non-WAM controller. Starting from the capability gaps of WAMs is compatible with developing mechanisms that apply across controllers. Our contribution to WAMs is to identify their task-level shortcomings and address them through a harness. Experiments across two WAM backbones and with a fixed checkpoint jointly examine the need for and effectiveness of this task-level support. To our knowledge, HarnessWAM is the first work to systematically investigate and address this long-horizon WAM execution problem through an external harness.

We hope the additional matched comparisons and clarification of our research focus help you reassess our contribution to extending local prediction and action capabilities into more reliable long-horizon execution, in light of the task-level shortcomings of WAMs and the need for a harness.

Thank you again for your time and effort in reviewing our submission.

As the discussion period is nearing its end, we would like to kindly follow up on our response, where we have provided additional experiments and clarifications addressing your concerns. We would greatly appreciate any further thoughts you may have and would be happy to address any remaining questions.

Thank you for your time and consideration!

Reviewer_Cq5uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HarnessWAM, an agentic controller built around a World Action Model for long-horizon, memory-dependent manipulation. A VLM Task Manager keeps a scene belief and a task graph. A deterministic compiler maps the plan onto the primitives the WAM supports and checks dependencies and holding state. During execution, a learned progress estimator triggers VLM calls that verify outcomes and decide whether to advance, replan or recover. The method improves both LingBot-VA and FastWAM on RoboMemArena (64.4% full-task SR with LingBot-VA) and reports 23.7% on RoboCerebra Ideal.

Critical strengths
  • The ablations in Table 4 are well controlled. All variants share the same subtask checkpoint and seeds, so the differences can be attributed to the controller itself. The gains also carry over to a second backbone (FastWAM), which suggests the design is not tied to one model.

  • Keeping targets symbolic until evidence is collected, and recording whether a fact was observed or only inferred, is a simple way to deal with occlusion. The occlusion tasks and the real-robot box exploration task both show clear improvements.

Critical weaknesses
  • Limited novelty. The harness uses the WAM only as a prompt-conditioned skill policy. Its predictive part plays no role in planning or verification. What remains is a fairly standard combination of VLM planning, PDDL-style operators, progress-based failure detection and replanning, which is close to prior works, several of which are not discussed. I would like to see either the world model actually used inside the harness, or evidence that the same harness behaves differently on a non-WAM policy (e.g. Pi 0.5).

  • The comparisons do not isolate what the harness contributes. Fine-tuned LingBot-VA alone already beats every baseline in Table 1. The one agentic baseline, Harness VLA, scores 0 on all occlusion tasks, which points to a configuration problem. The projection ablation also seems to mix up two things: mapping VLM outputs onto the WAM's training prompt vocabulary, and the constraint checking that the text gives credit to. The free-form VLM control is probably weak for the same reason. On RoboCerebra the gain is small and there is no backbone-only control, and the real-robot baselines have no memory, so the large margins there are expected. No confidence intervals are reported.

Questions for authors
  • How does the Static Plan baseline decide when to switch subtasks? Also, the "w/o task state" row matches Static Plan exactly in three of the four task families. Is that expected?

  • How was Harness VLA configured, and was its VLA fine-tuned on RoboMemArena?

  • Could you report a projection variant that only normalizes prompts and aliases, without the dependency and holding-state checks? How are the aliases built for each benchmark?

6 comments

Thank you for your careful review and for recognizing the ablations using the same checkpoint and the experiments across backbones. Through additional experiments and implementation details, we would like to clarify our contribution: WAMs already possess local prediction and action capabilities, but organizing these capabilities into sustained, reliable long-horizon execution requires evidence across stages, executability constraints, and task updates grounded in feedback. HarnessWAM systematically investigates and addresses this task-level capability gap.

W1. Novelty, the focus on WAMs, and the non-WAM comparison

Thank you for your comments. Our core contribution is to identify and address the capability gap that WAMs face when moving from local prediction and action execution to long-horizon tasks, and to establish the need for and effectiveness of an external harness in addressing these shortcomings. WAMs can generate local actions conditioned on instructions. However, situations such as inspecting before selecting a target based on the inspection results, or continuing the remaining task after an execution failure, require the system to connect prior evidence, the current physical state, and future operations. Existing WAMs still struggle to handle these requirements reliably in long-horizon tasks. HarnessWAM establishes this connection through evidence-grounded target binding, executable-space projection onto the WAM skill interface, and suffix revision that preserves execution history. Lightweight progress estimation and event-triggered scheduling further coordinate continuous WAM execution with high-level task decisions. To our knowledge, this is the first work to systematically investigate and address WAM shortcomings in long-horizon tasks through a harness.

Regarding the use of predictive outputs, the future states predicted by a WAM may differ from actual execution outcomes. Directly using these predictions as the basis for planning could cause subsequent decisions to diverge from the actual state. The current Task Manager therefore makes decisions from real observations and execution history, without directly reading the future frames predicted by the WAM. The prediction–deliberation gap refers to the capability gap between a local action model with world-modeling capabilities and sustained task-level decision-making. The same WAM trained on subtasks achieves a full-task success rate of 52.5% with Static Plan and 64.4% with the complete framework, directly demonstrating the task-level capabilities that remain to be supplied after initial skill decomposition.

We have also added an experiment connecting the same harness to π₀.₅. Both π₀.₅ and LingBot-VA within the harness use weights trained on RoboMemArena subtasks and share the same action primitives, task-switching strategy, and budgets. The high-level task representation, memory, Planner, verification, and plan-revision logic remain the same. The following table compares the two models with and without the harness.

ControllerTransfer full-task SROcclusion full-task SRCounting full-task SRSequence full-task SRAverage full-task SRAverage subtask SR
Original LingBot-VA11.363.237.168.849.060.0
LingBot-VA + same Harness22.576.859.381.364.4 (+15.4)76.8 (+16.8)
Original π₀.₅20.012.714.360.021.538.7
π₀.₅ + same Harness21.346.442.167.544.6 (+23.1)49.7 (+11.0)

Relative to the original results, the average full-task success rate increases from 49.0% to 64.4% for LingBot-VA and from 21.5% to 44.6% for π₀.₅. These system-level results demonstrate the value of the harness for both WAM and non-WAM controllers. Our research starts from the task-level shortcomings of WAMs, and the resulting mechanisms can also transfer to action models exposing the same skill interface. The focus on WAMs is supported jointly by our problem analysis, targeted design, and experiments with a fixed WAM.

W2. Attribution of the gains in the comparisons

W2.1. Whether the gains mainly come from LingBot-VA

Table 4 of the paper holds the underlying checkpoint fixed to directly examine the contribution of the harness. All variants use the same LingBot-VA checkpoint trained on RoboMemArena subtasks and the same evaluation seeds. Static Plan achieves 52.5%, compared with 64.4% for the full framework. Disabling suffix revision and removing recovery yield 57.1% and 59.0%, respectively. These differences arise from changes to the high-level execution mechanisms. We have also added comparisons with existing orchestration methods. HPE, PrediMem, and HarnessWAM share the same LingBot-VA checkpoint trained on RoboMemArena subtasks, and all use Qwen3-VL-32B-Instruct as the high-level Planner without task-specific fine-tuning. The results are as follows.

Orchestration methodTransfer full-task SROcclusion full-task SRCounting full-task SRSequence full-task SRAverage full-task SRAverage subtask SR
HPE16.362.754.371.354.671.4
PrediMem21.366.855.075.057.973.1
HarnessWAM22.576.859.381.364.476.8

With the underlying WAM and Planner held fixed, HarnessWAM achieves absolute full-task SR gains of 9.8% over HPE and 6.5% over PrediMem, with higher success rates in all four task categories. Together with the ablations using the same checkpoint in the paper, these results demonstrate clear benefits from task-level orchestration on the same underlying WAM.

W2.2. Harness VLA's zero success rate on occlusion tasks

Harness VLA's zero success rate on RoboMemArena occlusion tasks is consistent with evaluations reported in other work. Table 1 of Don't Drop the BATON [1] also reports a TSR of 0.0% and a CSR of 2.3% for Harness VLA on these tasks. The main difficulties are obtaining an initial success through whole-task exploration within a limited budget and managing state transitions between heterogeneous subtasks.

W2.3. Whether the projection gains mainly come from lexical normalization

We have added a normalization-only execution comparison that retains prompt and alias normalization while disabling dependency and holding-state checks. All other runtime modules are shared with Full projection. The table also includes Static Plan with the same WAM checkpoint to show the gains beyond initial skill decomposition.

SettingFull-task SR (%)Subtask SR (%)
w/o executable projection23.145.9
Static Plan52.569.7
Normalization only54.070.3
Full projection64.476.8

The w/o executable projection variant directly uses VLM-generated instructions, many of which fall outside the WAM's supported operation space. Execution is therefore affected by mismatches with the skill interface. Normalization maps synonymous object names and operation descriptions to the interface supported by the WAM. It first repairs these mismatches, explaining the substantial improvement over w/o executable projection.

Compared with Static Plan, normalization-only achieves only a 1.5% absolute gain in full-task SR (52.5% → 54.0%) and a 0.6% absolute gain in subtask SR, so their performance is similar. Full projection raises full-task SR to 64.4%, an absolute gain of 11.9% over Static Plan and a further 10.4% over normalization-only. This shows that, once an executable skill sequence is available, the main additional gains come from the dependency and holding-state checks in full projection, which make the plan compatible with both the skill interface and the current execution prerequisites.

We further compare plan quality at three stages: the raw plan, normalization only, and full projection.

Plan representationSyntax (%)Dependencies (%)Binding (%)Executability (%)
Raw VLM plan60.858.121.313.8
+ normalization and aliasing84.667.563.842.3
+ executable-space projection95.292.988.372.9

Normalization substantially improves Binding from 21.3% to 63.8% (+42.5%), reflecting the role of consistent object names and operation descriptions in argument matching. At the same stage, Dependencies improves only from 58.1% to 67.5% (+9.4%). The main improvement in Dependencies comes from the dependency checks in full projection, which yield a further absolute gain of 25.4% over normalization, reaching 92.9%. Full projection also raises Executability from 42.3% to 72.9%. These results show that lexical normalization primarily aligns names with the skill interface, while full projection additionally enforces dependencies and execution prerequisites so that operations can be executed in a valid order. This mechanism is consistent with the 10.4% absolute full-task SR gain over normalization-only in rollouts, demonstrating the harness's contribution to task-level execution constraints.

W2.4. Gains on RoboCerebra and the backbone-only comparison

RoboCerebra presents challenging long-horizon manipulation tasks, with an average of 9.1 atomic steps and 2,972.4 simulation steps per trajectory. These extended operation sequences place greater demands on consecutive local skill execution and subtask transitions. We have added Direct WAM results on RoboCerebra Ideal. Ideal SR follows the benchmark's definition of the average completion rate of key object-state transitions.

MethodControllerIdeal SR (%)
Direct WAMLingBot-VA3.95
HarnessWAMLingBot-VA23.70

HarnessWAM achieves an absolute SR gain of 19.75% over Direct WAM. The gains reported in the paper over GPT-4o Planner + OpenVLA and HPE are 1.78% and 2.60%, respectively. The added comparison further clarifies the system-level benefit over the underlying WAM. Even in static compositional tasks, high-level task organization is needed to make effective use of local control capabilities.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.