- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PointPilot presents a coherent and integrated robotic manipulation pipeline for cross-embodiment policy learning. The work offers a well-motivated framework supported by a substantial set of simulation and real-world experiments. However, I am not yet convinced that the paper establishes a sufficiently significant methodological contribution for ICLR. My primary concerns are that the claimed novelties appear incremental compared to existing approaches, the experimental improvements are relatively marginal, and the ablation studies do not adequately substantiate the benefits of the proposed core components (see the weakness section for more details).
Critical strengths
-
The paper presents an integrated pipeline for learning from heterogeneous action representations. Representing manipulation through task-relevant 3D motion rather than embodiment-specific actions is well motivated and offers a potentially useful approach to cross-embodiment learning.
-
The cross-embodiment transfer results in Table 4 are encouraging, demonstrating the potential of the proposed representation across different robotic platforms and manipulation tasks.
-
The paper and appendix provide sufficient implementation details to understand the proposed framework. I particularly appreciate the inclusion of real-world robotic experiments, which help demonstrate the practical feasibility of the approach. Overall, the paper presents a relatively comprehensive experimental study.
Critical weaknesses
- Presentation and clarity.
-
The methodology would benefit from more explicit connections between the text and the figures. For example, Figure 2 is not directly referenced or discussed in the main methodological descriptions, making it more difficult for readers to follow the overall pipeline.
-
The paper would benefit from a clearer distinction between conceptual novelty and system-level integration.
- Limited methodological novelty and insufficient justification through ablations.
-
The central idea of representing 3D object motion as point flow, including the robot gripper, has already been explored in related work. For example, PointWorld (https://arxiv.org/abs/2601.03782) also represents robotic embodiments and scene dynamics through 4D flow. Moreover, the fundamental distinction between PointPilot and existing approaches such as 3D Diffusion Policy remains unclear. I would appreciate a more explicit discussion of the methodological differences and the specific advantages introduced by PointPilot.
-
Regarding Table 1, I am not fully convinced that the selected features provide a fair comparison of methodological capabilities. For instance, Feature 2 (the absence of a separate grasping module) could potentially be incorporated into existing frameworks without fundamental architectural changes. Furthermore, the distinctions represented by Features 4 and 5 require clarification. Do these indicate that the existing methods were only trained or evaluated in single-arm or bimanual settings, or that they are fundamentally incapable of supporting the alternative setting, even after retraining? This distinction is important when assessing the claimed novelty.
- Marginal and inconsistent experimental improvements.
-
In Table 2, PointPilot exhibits a substantial performance drop on LIBERO-Goal compared with its performance on the other suites. The underlying reasons are insufficiently analysed. Moreover, the authors report a 96.0% success rate after excluding "three tasks", but the calculation and justification for this exclusion are unclear. Compared with SmolVLA, which demonstrates relatively consistent performance across the evaluated suites, the advantages of PointPilot are not convincingly established.
-
In Table 3, the improvement over existing methods is marginal, particularly compared with DP3 (56.6% versus 55.2%). Such a small performance difference provides limited evidence of the superiority of the proposed approach without further statistical analysis.
- Insufficient experimental validation and ablation studies.
-
In Table 4, X-VLA exhibits unusually poor performance on the "Click alarm clock" task with the ALOHA embodiment. I would appreciate an explanation of this observation. In fact, X-VLA performs reasonably well on the remaining seven tasks under ALOHA, which suggests that the results should be interpreted more carefully.
-
The cross-embodiment evaluation only includes X-VLA as a baseline. I believe this comparison is insufficient to support broader claims about superior cross-embodiment generalization. Additional representative baselines would strengthen the experimental conclusions.
-
More importantly, the experimental and ablation results do not clearly demonstrate the specific benefits of using a unified point-flow representation for both grippers and manipulated objects. At present, the proposed formulation appears primarily to provide an alternative representation for predicting robot and object motion. It remains unclear what intrinsic advantages this representation offers over existing approaches.
Questions for authors
Please see the weakness for the details of the questions that I would like to understand from the authors' explanations.
4 comments
Response to Reviewer 4J8V (Part II)
Reply to weakness 3:
LIBERO-Goal performance. The lower success rate is concentrated in three tasks involving frictional contact without fixed gripper–object coupling. We believe this reflects their sensitivity to contact accuracy: errors in point-flow prediction and execution mapping can accumulate and, thus, prevent the gripper from maintaining contact, causing execution failures. The reported 96.0% is the unweighted average success rate over the remaining seven tasks, provided to analyze performance across contact modes.
Comparison with DP3. Although PointPilot’s RoboTwin success rate exceeds DP3’s by only 1.4 percentage points, their training settings differ. DP3 is a vision-action policy trained separately for each task without language input, whereas PointPilot jointly trains a single model on LIBERO and RoboTwin.
To further evaluate cross-embodiment capability, we added more zero-shot transfer experiments including DP3 (EEF). Both methods use ALOHA demonstrations for RoboTwin training and subsequently transfer to ARX and Piper with frozen weights. PointPilot achieves 75.6% and 71.0% success on ARX and Piper, respectively, compared with 23.9% and 10.1% for DP3. These results further support PointPilot’s zero-shot cross-embodiment transfer advantage beyond DP3.
Response to Reviewer 4J8V (Part Ⅲ)
Reply to weakness 4:
Additional baselines. Beyond X-VLA, we added π₀.₅ (joint), π₀.₅ (EEF), and DP3 (EEF), covering both joint-angle and end-effector-pose action representations. These experiments follow the protocol of source training on ALOHA followed by zero-shot evaluation on ARX and Piper with frozen weights. Below are the detailed experiment results:
π₀.₅ (Joint) :
| Robot | Grab roller | Shake bottle | Shake bottle horiz. | Click alarm clock | Click bell | Dump bin into big bin | Stack two bowls | Pick dual bottles | Average | Δ (pp) |
|---|---|---|---|---|---|---|---|---|---|---|
| ALOHA | 100 | 100 | 100 | 98 | 87 | 86 | 79 | 74 | 90.50 | – |
| ARX | 9 | 81 | 79 | 8 | 45 | 4 | 2 | 7 | 29.38 | −61.13 |
| Piper | 0 | 0 | 0 | – | 0 | 0 | 0 | 0 | 0.00 | −89.43 |
π₀.₅ (EEF) :
| Robot | Grab roller | Shake bottle | Shake bottle horiz. | Click alarm clock | Click bell | Dump bin into big bin | Stack two bowls | Pick dual bottles | Average | Δ (pp) |
|---|---|---|---|---|---|---|---|---|---|---|
| ALOHA | 98 | 100 | 100 | 74 | 92 | 85 | 86 | 67 | 87.75 | – |
| ARX | 1 | 67 | 73 | 50 | 30 | 4 | 23 | 31 | 34.88 | −52.88 |
| Piper | 64 | 80 | 71 | – | 84 | 0 | 35 | 10 | 49.14 | −40.57 |
DP3 (EEF) :
| Robot | Grab roller | Shake bottle | Shake bottle horiz. | Click alarm clock | Click bell | Dump bin into big bin | Stack two bowls | Pick dual bottles | Average | Δ (pp) |
|---|---|---|---|---|---|---|---|---|---|---|
| ALOHA | 100 | 98 | 100 | 87 | 100 | 83 | 63 | 63 | 86.75 | – |
| ARX | 87 | 47 | 31 | 8 | 6 | 0 | 0 | 12 | 23.88 | −62.88 |
| Piper | 18 | 4 | 3 | – | 10 | 0 | 0 | 36 | 10.14 | −76.57 |
PointPilot (Point Flow) :
| Robot | Grab roller | Shake bottle | Shake bottle horiz. | Click alarm clock | Click bell | Dump bin into big bin | Stack two bowls | Pick dual bottles | Average | Δ (pp) |
|---|---|---|---|---|---|---|---|---|---|---|
| ALOHA | 98 | 100 | 94 | 75 | 79 | 100 | 54 | 88 | 86.00 | – |
| ARX | 85 | 93 | 90 | 69 | 75 | 67 | 41 | 85 | 75.63 | −10.38 |
| Piper | 79 | 75 | 83 | – | 77 | 69 | 39 | 75 | 71.00 | −16.57 |
Results Revision and task retraining. Thank you for pointing out this discrepancy. With the released official X-VLA checkpoint, we obtained a 17% success rate over 100 ALOHA trials on click_alarmclock. In the failed scene, we observed that the gripper would approach the alarm clock correctly, but couldn't approach deep enough to press the button------the distance between the final gripper and the clock is from 5mm to 10mm. Thus, we think there might be some problem with the official released checkpoint. Then, we retrain X-VLA on this task with our own rolled-out data and get 96% success rate which coincides with the officially reported success rate. And other tasks' success rates also match reported one. With this new checkpoint, the cross embodiment success rate on ARX reduces from 62% to 60%. The table result has been updated.
Contribution of the unified point-flow representation. First, task-relevant object motion serves as a shared prediction target. This decouples task-motion prediction from embodiment-specific execution without requiring a separately learned action decoder for each embodiment. Experiments demonstrate the tasked-focused object motion a better cross-embodiment ability than other action representation, such as end-effector pose or joint action space. Furthermore, including grippers as task-focused objects allows the same interface to cover both pre-grasp gripper motion and post-grasp object motion. Combined with execution-role assignment and subtask planning, this enables automatic target switching, closed-loop and long horizon/multiple stages manipulation in both single-arm and bimanual settings. To our best knowledge, existing object-centric methods (mentioned in the related work) possess limited ability to achieve this goal.
I sincerely thank the authors for their detailed responses and additional experiments, which address several of my concerns regarding the generalization and robustness of the proposed method. In particular, the expanded cross-embodiment evaluations provide stronger evidence of its practical effectiveness. However, my primary concern regarding methodological novelty remains. Compared with existing approaches, PointPilot appears to introduce relatively incremental modifications, mainly through extending the input representations and integrating additional modalities and existing components, rather than proposing a new methodology/framework. While I acknowledge the value of this system-level integration, I remain unconvinced that the paper demonstrates sufficiently substantial methodological contributions for ICLR, particularly given the lack of controlled ablations isolating the benefits of its core unified point-flow representation. Therefore, despite appreciating the authors' efforts and the improved experimental evidence, I have decided to maintain my original rating.
Response to Reviewer 4J8V (Part I)
Thank you for your comments and suggestion which are really useful. Below is the corresponding reply to each weakness and question.
Reply to weakness 1:
Thank you for this suggestion. We referenced Fig. 2 in the method section and explained the pipeline through text and figures. We also clarified the distinction between the following contributions:
Conceptual contribution: PointPilot uses task-focused objects as its basic modeling units, and classifies each grounded object with an execution role. This design associates an object-centric model with different manipulation modes, and provides a unified motion representation across embodiments, supporting joint training on heterogeneous demonstrations with different embodiments and manipulation modes.
System integration: PointPilot integrates subtask planning, semantic grounding, object-flow prediction, and a geometric execution adapter. As the task progresses these components automatically select relevant objects, predict their motion, and convert it into robot actions. This system integration enables autonomous, closed-loop execution of multi-stage tasks. Compared with previous object-centric models that only predict the motion of a single fixed object, PointPilot automatically selects, switches and predicts multipled targeted objects, supporting closed-loop execution of tasks involving different objects within a unified framework.
Reply to weakness 2:
Methodological differences and additional experiments. Thank you for this detailed comment. We agree that representing robot and object motion in 3D has been explored previously, like pointworld. However, there still exists some differences.
Specifically, PointWorld predicts full-scene point flows conditioned on scene point clouds and robot actions. PointPilot instead only predicts subtask-relevant object flows from language and RGB observations, then converts them into executable actions. Besides, PointWorld requires stereo cameras to collect high quality depth images, which is time-consuming and time cost . Such stringent requirements impose high demands on the quality of pretraining data for PointWorld, rendering most existing datasets unusable and thereby limiting its scalability to large-scale pretraining. PointPilot constructs task-relevant object-flow supervision from RGB demonstration videos, together with camera calibration and recorded robot states, without requiring additional stereo-depth capture. This enables PointPilot to use robot demonstrations without depth observations, reducing dependence on depth-sensing equipment. As a VA policy, DP3 inputs point clouds and predicts robot actions, whereas PointPilot is able to accept language input and predicts 3D object flow. The distinction concerns the prediction target and execution mechanism, beyond the use of 3D information.
We additionally evaluated DP3 (EEF) for zero-shot cross-embodiment transfer. Both methods use ALOHA demonstrations for RoboTwin training and transfer to ARX and Piper with frozen weights; PointPilot retains its joint training with LIBERO. PointPilot achieves 75.6% and 71.0% success on ARX and Piper, respectively, compared with 23.9% and 10.1% for DP3. These results further support PointPilot’s transfer advantage on the evaluated tasks.
Comparison criteria in Table 1. We do think it is valuable to define our comparison criteria more precisely. Feature 2 describes whether a method covers pre-grasp and post-grasp manipulation through a unified motion interface without a separate grasping module. For previous object-centric frameworks which don't predict gripper states, incorporating grasping also requires representing gripper motion and handling phase transitions, potentially requiring additional prediction targets, supervision, and data processing beyond a simple model change.
Features 4 and 5 concern whether the original representation and execution interface directly support single-arm and bimanual manipulation, rather than whether a method could support another setting after extension. For example, an interface restricted to one arm cannot jointly trained on heteoregenous data. It requires nontrivial future work to change the model dimension and execution logic to support two arms. PointPilot unifies both modes through execution-role assignment and jointly trains one model on LIBERO and RoboTwin. Existing work didn't show this ability.