- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how explicit metric relations can improve VLA and world-action policies. It first jointly predicts actions and object-relative end-effector trajectories using Interaction-Centric Tokens (ICTs). It then freezes the adapted policy and trains a Metric Action Interaction Field (MAIF), which uses point-cloud features to correct action predictions. The framework is evaluated on RoboTwin 2.0, LIBERO, and five real-robot tasks, with reported average gains of 3.59, 0.80, and 6.80 percentage points.
Critical strengths
The improvements extend to tasks where geometric errors directly affect success, including USB insertion and bimanual retrieval. For π0.5, average real-world success rises from 69.4% to 77.0%; Dual-USB Insertion improves from 47% to 59%.These execution results make a stronger case for the approach than the offline trajectory-error plots alone.
The evaluation covers five backbones on 50 RoboTwin tasks, including the geometry-aware X-WAM, whose average success increases from 78.41% to 82.01%. The ICT-only and full-method comparisons also suggest that the second stage contributes beyond ICT adaptation.
Critical weaknesses
The full method adds simulator depth for several RGB-based baselines, an approximately 200M-parameter adapter, and physical supervision. Table 3 includes ICT-only, flow-only, and extra-training controls, but does not compare object-relative supervision with an equivalent absolute end-effector target, or MAIF with a simpler adapter using the same geometry, capacity, and waypoint loss.
The real-robot success rates support practical usefulness, but the precision claim is assessed mainly through training/offline ADE and a small barrier-height analysis. Lower error relative to a demonstration does not necessarily imply more accurate object alignment or safer obstacle clearance during execution.
Equation (1) requires a full object pose at each time step, but the real-world annotation procedure is described only as 2D annotation followed by depth-based lifting. It remains unclear how object orientations are recovered, how reference objects are assigned during object switches or simultaneous manipulation of different objects, and how occluded frames are handled. These choices directly determine the ICT targets.
Appendix A.2 states that the Fast-WAM baselines on RoboTwin are taken from the public leaderboard, while the proposed variants are evaluated locally. The authors report that their local reproductions were close, but do not provide those numbers. The main results also lack variability across training seeds. This makes the exact improvement harder to assess, particularly on LIBERO, where the average gains are 0.55 and 1.05 percentage points.
Questions for authors
MAIF processes the action and ICT streams separately, and only its action branch is used at inference. How much does Stage 2 ICT supervision contribute beyond the action-position loss? An ablation with λICT = 0, while keeping the Stage 1 ICT-adapted backbone and all other settings unchanged, would help clarify its independent contribution.
How is the reference object selected during object switches or when the arms manipulate different objects? How do real-world 2D annotations and depth yield full object orientations, and how are occlusions handled?
Beyond binary task success, could you report execution-level measurements such as alignment error before USB insertion or minimum obstacle clearance during barrier crossing? These would help establish whether the lower ADE translates into more precise interaction with the actual objects.
Since local reproductions of Fast-WAM was already conducted, could you report their exact RoboTwin results alongside the proposed variants under matched evaluation conditions?
2 comments
Gentle follow-up before the discussion deadline
Dear Reviewer XekY,
As the discussion period closes on Nov 18 (AoE), we would like to briefly follow up. Following your suggestions, we added the absolute end-effector target control, a parameter-matched MLP adapter (MAIF +4.50 points), the λ_ICT = 0 ablation, USB alignment error measurements, exact local RoboTwin reproductions, and expanded annotation details. We would greatly appreciate it if you could let us know whether these address your concerns, and if so, whether you would consider updating your assessment. Thank you!
Response 3
We thank you and address concerns below.
W1 & Q1: Matched controls and Stage 2 ICT supervision.
We added Absolute End-effector Tokens (AET), replacing object-relative targets while retaining the token architecture and supervision objectives. Stage 1 results on RoboTwin 2.0:
ICT improves average success over AET by 1.82/1.43 percentage points for π0.5/Fast-WAM, supporting object-relative supervision beyond absolute end-effector targets.
We also replace only MAIF with a parameter-matched MLP adapter, keeping identical geometry inputs, frozen Stage 1 checkpoint, losses (including waypoint supervision), and training settings. Separately, we set λICT=0 only in Stage 2, retaining action-flow and -position supervision and all other settings.
With geometry, capacity, and losses matched, MAIF exceeds the MLP by 4.50/8.88 average/randomized percentage points. Stage 2 ICT supervision adds 2.10/2.78 points through the shared MAIF trunk, without an auxiliary MAIF ICT branch at inference.
W2 & Q3: Execution-level precision.
We agree that offline ADE alone does not establish execution-level precision. We compare π0.5 with π0.5 + Ours using planar USB alignment errors relative to the end-effector position for correct socket alignment. Each method uses five trials per task: five insertions for Task 2 and ten for Task 3.
These small-sample results provide limited evidence of improved alignment; protocols and per-trial results are in the appendix. Mean peak barrier-crossing height decreases from 307.151 to 282.621 mm at 100% success for both methods. This indicates reduced lifting; minimum obstacle clearance was not measured, so improved safety is not established.
W3 & Q2: Orientation, reference assignment, and occlusion.
The expanded ICT Annotation appendix details task-defined frames. For non-collinear keypoint direction d=p2−p1 and consistently oriented surface normal n in the same frame,
\mathbf u=\mathbf d-(\mathbf z^\top\mathbf d)\mathbf z,\quad \mathbf x=\frac{\mathbf u}{\|\mathbf u\|},\quad R=[\mathbf x\;\mathbf z\times\mathbf x\;\mathbf z]\in SO(3).$$ Keypoint ordering and normal signs fix axis directions. Plugs use their longitudinal direction and table normal; ball axes are base-aligned without spin estimation. Corn roll is not independently recovered. References are assigned per arm and phase, switching from grasp target to receiving container/port at visually reviewed boundaries, without interpolation across objects. In microwave manipulation, the left arm uses the handle/box/placement target by phase, while the right uses the door. During occlusion, stationary poses are retained only within stationary intervals. Moving references require tracking; ball-centre interpolation is limited to valid observations at most 20 frames apart. Unsupported intervals and specified uncertain transitions are excluded. **W4 & Q4: Local reproductions and training-seed variability.** We added exact local RoboTwin results alongside official results. Both augmented variants build on the locally trained baselines and use matched evaluation conditions across all 50 tasks: Average gains over local baselines are 1.61 and 4.98 percentage points. | Backbone | Method | Clean (%) | Randomized (%) | Avg. (%) | | --- | --- | ---: | ---: | ---: | | $\pi_{0.5}$ | Local baseline | 70.84 | 48.38 | 59.61 | | $\pi_{0.5}$ | + Ours | **71.26** | **51.18** | **61.22** | | Fast-WAM | Local baseline | 78.28 | 3.32 | 40.80 | | Fast-WAM | + Ours | **80.08** | **11.48** | **45.78** | LIBERO is an established benchmark on which our matched local baselines already exceed 96% success, leaving limited headroom. Our method improves success from 96.85% to 97.40% for $\pi_{0.5}$ and from 96.75% to 97.80% for Fast-WAM. Although modest in absolute terms, these 0.55/1.05-point gains represent 17.5%/32.3% reductions in observed failure rates, supporting the method's benefit even with near-saturated baseline performance. Stage 2 uses seed 42. The 2,000 LIBERO evaluation episodes per model do not quantify training-seed variability; we acknowledge this limitation and do not claim statistical significance across training seeds. We also polish the manuscript. Thank you for your constructive feedback.