acceptodds
Under review as a conference paper at ICLR 2027

TraceWAM: Task-State Reference-Guided Action Candidate Evaluation for Long-Horizon Robot Manipulation

Abstract

World action models generate robot actions from visual and language context, yet action generation alone does not ensure that the resulting physical state remains aligned with the active stage of a long-horizon task. We study task-state alignment drift, the stage-relative accumulation of deviations in task-critical object poses, contacts, and spatial relations during closed-loop execution. To address this problem, we propose Task-State Reference-Guided Action Candidate Evaluation for World Action Models (TraceWAM), a framework for verifying candidate actions against the current physical task state before execution. TraceWAM maintains stage-wise physical references together with a recurrent progress memory, and predicts candidate-specific alignment deviations and uncertainty in an object-centric task-state space. Given a base action chunk and a bounded residual candidate, TraceWAM evaluates both under the same active reference and selects the residual only when its predicted risk and progress margins satisfy the verification criteria. During training, alternative action-conditioned outcomes are obtained by branching from shared simulator states, providing supervision for candidate-level consequence prediction without altering the deployment procedure. On RoboTwin 2.0 and LIBERO, TraceWAM achieves average success rates of 93.0% and 98.1%, outperforming Fast-WAM by 1.6 and 1.4 percentage points, respectively, and further achieves 41.6% overall success on RMBench. Task-state drift analyses and ablation studies further support the effectiveness of stage-referenced action verification for reliable multi-stage manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world action models,robot manipulation,action verification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EmP2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

TraceWAM addresses task-state alignment drift in long-horizon robot manipulation. It uses stage references and progress memory to evaluate a base action chunk and a bounded residual candidate against the active stage goal. A shared verifier predicts each candidate’s structured physical deviations and uncertainty. The residual is accepted only when the reference is valid and the prediction indicates both lower risk than the base action and progress from the current state; otherwise, the base action is retained. Shared-state simulator branches provide supervision for learning candidate outcomes.

Critical strengths
  1. A clear formulation of task-state alignment drift. The paper characterizes execution errors through changes in task-critical poses, contacts, and spatial relations relative to the active stage goal. Distinguishing current misalignment from further deterioration provides a useful framework for studying how physical errors develop during multi-stage manipulation.
  2. Consistent gains across manipulation benchmarks. TraceWAM improves over Fast-WAM by 1.6 and 1.4 percentage points on RoboTwin 2.0 and LIBERO, reaching 93.0% and 98.1% success, respectively. The larger improvement on RMBench, from 4.0% to 41.6%, demonstrates the practical value of the combined stage references, progress memory, and candidate evaluation on memory-dependent tasks.
Critical weaknesses
  1. Comparison with related methods. EV-WM already uses structured event prediction, uncertainty scoring, and candidate gating, with bounded residual proposals in its contact-sensitive setting. The scalar-verifier baseline does not establish TraceWAM's advantage over this approach. Please compare under matched budgets and clarify the benefit in success rate, latency, or supervision requirements.

  2. Dependence on predefined references. The experiments mainly vary scenes within trained tasks, leaving transfer to new task structures untested. How much additional schema design and demonstration data does a new task require?

  3. Insufficient real robot evaluation. The real world PiPER case lacks training data size, training tasks, success rates, and more details. Please report these and provide a complete demo through an anonymous link (if possible). I would raise my score if the authors address my concerns raised in points 1 and 3.

Meanwhile, replacing the schematic robot interactions in Figures 1 and 2 with images from real robot executions may make the paper more visually appealing.

Questions for authors

Could TraceWAM’s short-term scoring override useful recovery actions, such as withdrawing before retrying an insertion? Can the authors compare recovery success rates with the base policy from the same failure states?

Reviewer_bzJzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

TraceWAM augments a world action model with stage references, progress memory, and a verifier that predicts structured consequences for base and residual action candidates. Simulator branches supervise candidate outcomes, while uncertainty-aware gating selects corrections.

Critical strengths
  1. Predicting object-centric pose, contact, and relation deviations provides an interpretable connection between consequence learning and action selection. Requiring both an advantage over the base action and predicted progress addresses a meaningful distinction: a better candidate may still worsen the current state.
  2. The matched-context experiment holds the backbone, stage context, candidate generator, and branch data fixed when comparing scalar and structured verification. Improvements from 92.0% to 93.1% on randomized RoboTwin and from 37.6% to 41.6% on RMBench provide evidence for the structured consequence representation beyond the full-system comparison.
Critical weaknesses

The method depends on a pretrained task-state reader, deterministic goal parser, schema library, pose prototypes, and stage tolerances. However, the paper does not adequately specify the reader architecture and training, reliability-mask construction, concrete schema examples, or task-specific authoring requirements. Branch-data volume, auxiliary training budgets, and numerical selection hyperparameters are also missing. These omissions affect reproducibility and make it difficult to assess the practical cost and generality of the approach.

Questions for authors

How are the state reader, reliability masks, and schema library implemented and trained? Which components require task-specific annotation or manual authoring, and how accurately does the reader estimate poses and contact predicates?

Reviewer_bpTcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TraceWAM to address task-state alignment drift in long-horizon robotic manipulation. The method uses stage references, progress memory, and a structured verifier to evaluate candidate actions before execution. Experiments on RoboTwin 2.0, LIBERO, and RMBench demonstrate competitive performance.

Critical strengths

1.Practical Motivation: The paper addresses task-state misalignment in long-horizon robotic manipulation, an important problem in robot control. 2.Promising Performance: The method outperforms Fast-WAM across three benchmarks, with notable improvements on RMBench. 3.Reasonable Design: The integration of stage references, action consequence prediction, and risk evaluation provides a reasonable approach to action correction.

Critical weaknesses

1.Unclear Performance Contributions: TraceWAM introduces several additional components and training supervision. Although Table 2 provides useful comparisons, the large improvement on RMBench cannot be fully attributed to structured verification. The contributions of stage modeling and progress memory remain unclear. 2.Reliance on Additional Task Information: Stage references depend on predefined schemas and demonstration data, while the state reader requires additional annotations. The paper provides limited analysis of how these dependencies affect robustness and generalization. 3.Limited Verifier Evaluation: The paper mainly reports task success rates and execution drift. The accuracy of the verifier in predicting action outcomes and ranking candidates has not been directly evaluated.

Questions for authors

1.Additional Ablation Studies: Could the authors clarify the settings of w/o STR and w/o ACV in Table 3, particularly the performance difference between w/o ACV and Branch Residual in Table 2? An additional ablation removing progress memory while keeping other components unchanged would help clarify its contribution. 2.Robustness Evaluation: Could the authors provide an example of stage reference construction and explain how the state reader is trained? It would also be useful to evaluate RoboTwin performance under different levels of pose estimation noise to assess the robustness of these components. 3.Verifier Accuracy: Could the authors evaluate the verifier on held-out simulator branches and report prediction errors and pairwise candidate ranking accuracy? Comparing scalar and structured verifiers would provide more direct evidence for the proposed design.

Comments on the paper

1 comment
Area_Chair_6zQMOpenReview ↗

Reminder: Reviewer Engagement During the Discussion Period

Dear Reviewers,

As the discussion period is drawing to a close, could you please review the authors' rebuttal and provide your feedback at your earliest convenience?

Active engagement in discussions with the authors is an essential part of the peer-review process and plays an important role in ensuring the success of the conference. Your participation would be greatly appreciated.

Thank you for your time and contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.