acceptodds
Withdrawn from ICLR 2027

WECO:Equivariant, Generalizable Robot Actions from Object-Centric World Changes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generalizing robot manipulation beyond the training distribution remains difficult because demonstrations sparsely cover variations in object appearance, geometry, and workspace configuration. We introduce WECO, a world–action framework that separates what should change in the world from how a fixed robot embodiment should realize that change. Given a predicted visual future, WECO extracts a structured object-centric world change consisting of a coarse rigid transformation and sparse contact or alignment references. A geometry-conditioned inverse dynamics model combines this target change with an object-centered mesh of the current state to predict short action chunks in closed loop. Experiments across multiple manipulation primitives show strong robustness to appearance changes and encouraging transfer across object geometry and workspace positions. These preliminary results suggest that structured object-level world changes provide a useful inductive bias for manipulation generalization.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Manipulation, Out-of-Distribution Generalization, World–Action Models, Object-Centric Representation, Inverse Dynamics, Visual Prediction

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_JsmGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a world-action framework named WECO to enable robot accomplish manipulation tasks beyond the training distribution. WECO first predicts a future frame, then constructs an object-centric representation, and finally calculates action chunks leveraging inverse dynamicsd model.

The performance of WECO is evaluated in RoboSuite/MimicGen in simulation. Results show that WECO outperforms PI 0.5 under settings including original, color changes and background changes, but does not perform as well as PI 0.5 with shape changes.

Critical strengths
  1. The paper addresses a significant problem in the field of robot manipulation, and the proposed solution is highly intuitive.
  2. On a specific simulation benchmark involving variations in color, background, and texture, the method proposed in the paper achieved a 100% success rate, thereby validating to some extent that geometry-based representations can enhance generalization capabilities.
Critical weaknesses
  1. No real-world experiments. The paper conducted experiments solely in a simulation environment and did not validate the method on actual robots; consequently, it remains unknown whether the approach would be effective in the real world.
  2. There are serious formatting issues with the paper. The widths of Figure 1, Figure 2, and Table 3 exceed the page margins.
  3. Experimental validation is severely lacking. WECO was evaluated on only a single simulation benchmark and compared against just two baselines: pi05 and seer. Furthermore, no ablation studies were conducted for many of WECO's modules—specifically object-centric geometry, contact/alignment references, pose–skill disentanglement, and closed-loop conditioning.
Questions for authors
  1. How does WECO perform in real-world scenarios?
  2. How does WECO perform on a wider range of benchmarks?
Reviewer_SX52OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes WECO, a framework that connects visual planning and robot action generation through structured, object-centric world changes. The desired change is represented by a coarse rigid transformation together with contact and alignment references. A DiffusionNet encoder processes an object-centered 3D surface mesh, and a Transformer decoder predicts action sequences conditioned on the target change, robot proprioception, and other information. The paper also introduces pose–skill representation disentanglement and remaining-motion conditioning to support closed-loop execution and spatial generalization.

Critical strengths

Combining coarse object motion with contact and alignment references offers a reasonable way to express desired outcomes while retaining information relevant to their physical realization.

Critical weaknesses
  1. The central methodological claims are not clearly connected to the evaluated system, making it difficult to determine whether the complete proposed framework has been validated. The paper presents a pipeline from visual prediction to structured world-change extraction and then to closed-loop action generation. However, it does not specify which world model is used in the experiments or how contact references, alignment references, and object masks are obtained at test time. It is therefore unclear whether the main results evaluate the complete visual-planning-and-control pipeline or an inverse dynamics controller supplied with externally constructed targets and auxiliary information. Furthermore, the paper emphasizes short-chunk closed-loop execution, whereas the best-performing configuration in Table 3 predicts H=96H=96 OSC delta actions and upsamples them to the skill duration. The only explicitly identified multi-chunk closed-loop Stack configuration achieves 0/1 success. These results are not necessarily contradictory, but the paper needs to state which configuration produces Tables 1 and 2. Without this connection, the reported results do not establish the effectiveness of the proposed closed-loop design. The “equivariant” claim in the title is also insufficiently supported. Object centering and a pose-invariance objective applied to part of the latent representation do not by themselves guarantee equivariant action outputs. Even accounting for kinematic constraints that limit strict equivariance, the paper should define the intended behavioral consistency property and validate it through controlled joint spatial transformations.
  2. The experimental scope, sample sizes, and controls are insufficient to support broad manipulation-generalization claims or establish the contribution of individual components. Most conditions contain only five rollouts, and Stack repeatedly uses a single demonstrated initialization. Move and Rotate evaluate empty-hand end-effector motion, while TowelFold uses a hinged proxy. These settings provide limited evidence for generalization of object interactions. In Stack, which involves more substantial contact and multi-stage manipulation, success is 0% under all reported size, shape, and position shifts. Even a ±3\pm 3 cm XY perturbation reduces success in the appearance conditions to 0%. The continuous-target experiment uses a relaxed success threshold whose numerical value is not specified, and skill composition is supported by only one illustrative example. The baseline comparisons and ablations also do not adequately explain the source of performance. For Seer and π0.5\pi_{0.5}, the paper provides insufficient convergence or validation evidence to establish that the fine-tuning configurations are adequately adapted; the number of fine-tuning steps alone cannot establish this. Table 3 simultaneously changes action representation, prediction horizon, execution controller, and evaluation instances, preventing individual effects from being isolated. The paper also lacks direct ablations establishing the gains from the world-change interface, contact/alignment references, mesh representation, and pose–skill disentanglement. Consequently, the reported appearance robustness does not yet establish the distinct contribution of the proposed mechanisms.
Questions for authors

Please address the clarification requests and evidence gaps identified in the Critical Weaknesses above.

Reviewer_APo4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

WECO represents a manipulation goal as a desired object movement, together with contact and alignment information. A controller uses this target, the object’s 3D geometry, and the robot’s state to predict actions. Simulation experiments show robustness to appearance changes, but limited generalization to changes in object geometry and position.

Critical strengths

The method combines desired object movement with contact and alignment information. This is a sensible design because knowing where an object should end up does not fully determine how the robot should interact with it.

Critical weaknesses
  1. The novelty over existing object-centric methods is unclear.

Prior work already separates desired object motion from robot-specific actions. For example, SPOT (ICRA 2025) predicts object pose trajectories and converts them into robot actions with closed-loop feedback. WECO uses a different combination of motion targets, contact references, and a geometry-conditioned controller, but does not clearly explain or demonstrate the benefit of these differences. The paper neither discusses SPOT nor compares against a closely related object-centric method.

  1. The experiments are too limited to support the main claims.

Most conditions have only five trials. Move and Rotate involve an empty gripper, and TowelFold uses a simplified hinged object. Stack fails under every tested size, shape, and position change, including a small ±3 cm positional perturbation. Appearance robustness alone is insufficient evidence because the controller does not use RGB features. There are no controlled ablations showing the benefits of the proposed representation or its main components, and no direct test of equivariance. The limited baseline training details also make the comparisons difficult to assess.

Questions for authors
  1. What is the main technical advance over methods such as SPOT, and what evidence shows the benefit of that advance?
  2. Which world model generates the experimental targets? How are contact/alignment references and object masks obtained at test time?
  3. Which execution setup produces Tables 1–2, and how does it relate to the unsuccessful closed-loop Stack setting in Table 3?
  4. What evidence supports the equivariance claim, and were the baselines trained sufficiently for a fair comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.