- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a novel method for generative action policies - instead of predicting immediate next action trunks, or predicting dense and costly long-horizon structures, the method jointly predicts a compact sketch of the trajectory, as well as an executable dense action trunk, using the same denoiser head. The method is evaluated on two simulator-based benchmarks, where it slights outperforms the strongest baseline methods, as well as in real-world experiments, where it demonstrates real and substantial advatnages.
Critical strengths
- The idea of long-horizon prediction in generative action models has been strongly motivated, and tested in many different ways. PAM-DP takes pushes this idea from B-spline, which directly uses spline coefficients to parameterize executable actions, to the next level of using the spline only for representing the sketch, which is then used to condition a separate action chunk.
- The strongest result is from the real-world experiment, where PAM-DP shows substantial advantage over Diffusion and B-spline.
- The ablation study is quite thorough, covering many meaningful alternatives.
Critical weaknesses
- The main comparisons for the simulations are less convincing, as many of them are taken from other publications rather than reproduced.
- Missing broader discussion and comparison with alternative forms of future-motion guidance action policies. There are keypoint/waypoint based methods, e.g. ChainedDiffuser, Chain-of-Action, RT-Trajectory; WAM-style models like CoT-VLA. The authors should provide a more in-depth analysis and potentially empirical comparison against PAM-DP.
- It would be great for the authors to provide some evaluation on the method's performance for the spline sketch prediction for OOD conditions.
Questions for authors
Would equal-budget phase/stage-sampled waypoints provide similar benefits?
2 comments
I thank the authors for the detailed rebutal. All my concerns are address and I will raise my score.
We thank the reviewer for emphasizing baseline reproducibility and alternative forms of future-motion guidance. We report our reproduced Push-T baselines (W1), new real-world comparisons with ChainedDiffuser and CoT-VLA (W2), evaluation under unseen lighting (W3), and a controlled comparison with equal-budget phase-sampled waypoints (Q1).
W1: Reproduced simulation baselines
We argue that using published scores does not inherently weaken a benchmark comparison. The key requirements are comparable tasks, observation settings, metrics, and evaluation protocols.
Our reproduction of DP-T using its official configuration achieves 0.78 / 0.65 on image-based Push-T, closely matching the published 0.78 / 0.66.
W2: Comparison with alternative future-motion guidance
PAM combines a timing-free remaining-path representation with joint sketch–action generation. The related approaches differ in their intermediate guidance and how it enters action generation:
We adapted the original ChainedDiffuser and CoT-VLA methods to our dual-arm platform and trained them on the same demonstrations. We evaluated handoff and fold cloth with 20 trials per task under the Table 3 protocol.
PAM-DP achieves higher success on both tasks, totaling 31/40, compared with 25/40 for ChainedDiffuser and 23/40 for CoT-VLA.
[1] Xian et al. ChainedDiffuser: Unifying Trajectory Diffusion and Keypose Prediction for Robotic Manipulation.
[2] Zhang et al. Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation.
[3] Gu et al. Robotic Task Generalization via Hindsight Trajectory Sketches.
[4] Zhao et al. CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models.
W3: OOD evaluation under unseen lighting
To assess robustness during deployment, we tested handoff and fold cloth under a lighting condition absent from the demonstrations, with 20 trials per method per task.
PAM-DP achieves 19/40 successes versus 9/40 for Diffusion Policy under the evaluated lighting shift, supporting better closed-loop task performance.
Q1: Equal-budget phase/stage-sampled waypoints
We replaced the B-spline coefficients with positions sampled uniformly in arc-length phase along the same remaining path. Both representations retain 11 predicted 2-D tokens, relative coordinates, and the terminal displacement as the last token.
B-spline coefficients are jointly fitted to 64 phase samples spanning the remaining path, whereas phase-sampled waypoints directly retain selected positions. We compared target reconstruction RMSE using the metric in App. A.3, with linear interpolation between waypoints.
The B-spline targets reconstruct the path more accurately in all evaluated settings, consistent with the higher coverage in the controlled Push-T comparison.