- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Graph2Demo converts PDDL plans into per-robot subtasks and inter-robot dependencies that select target poses, rewards, and synchronization conditions. State-based residual RL builds on a shared pretrained reaching policy to generate demonstrations for per-agent visuomotor diffusion policies. Evaluation covers all eleven RoboFactory tasks with up to four robots in simulation.
Critical strengths
- The framework provides a practical integration of dependency-aware coordination and residual learning. Explicit dependencies permit independent progress while coordinating joint subtasks, and the shared reaching policy supports reuse across tasks and robots.
- Generated demonstrations improve average downstream success across the full benchmark, with substantial gains on several difficult multi-robot tasks and shorter successful rollouts. Three independent runs, matched demonstration counts and policy-training settings, and script baselines with noise and asynchronous execution strengthen the empirical evidence.
Critical weaknesses
- The graph's contribution is not isolated from reward shaping. Removing it also removes intermediate rewards, so the ablation's failure may reflect sparse-reward exploration rather than the specific dependency representation. Alternative scheduling or hierarchical RL with matched targets and rewards would better establish the benefit of the graph design.
- Practical costs and reliability need further characterization. The method requires PDDL specifications, annotated interaction poses, predicate checkers, and reward functions, plus 100 million residual-training transitions per task. Engineering effort and total generation cost relative to scripting are insufficiently quantified. Substantial regressions on Camera Alignment and Take Photo also lack a failure analysis; shorter completion times conditioned on success do not resolve the reliability tradeoff.
Questions for authors
N/A
5 comments
W2 (a): Engineering effort and total generation cost
We apologize for not making the engineering effort explicit in the paper.
PDDL domain and predicates
Our shared PDDL domain comprises 16 predicates and 14 action schemas across all 11 RoboFactory tasks and took approximately 10 hours for writing and debugging.
We also implement state-based checkers for 8 out of the 16 predicates, while the remaining predicates are only used for symbolic planning. We implemented these checkers largely by adapting ManiSkill3’s built-in methods or success evaluation code, e.g., the holding predicate checker reuses ManiSkill’s is_grasping method.
We do not design additional dense rewards for each subtask and only use predicate satisfaction bonuses.
Annotated assets
For annotations, we reuse the object annotations released with RoboFactory without introducing new ones. These annotations define contact and functional frames for computing end-effector targets and can be reused across tasks involving the same objects. RoboTwin also provides publicly available annotated assets, which can be used or extended when introducing new objects.
Per-task marginal effort
For tasks covered by the shared domain, specifying the initial and goal states in predicates takes 5 minutes on average. Connecting the symbolic object names to simulator objects is straightforward and only takes approximately 1–2 minutes per task. Once the task is specified, the PDDL planner automatically comes up with a symbolic plan with the fewest actions. The resulting task graph selects annotation-derived target poses and provides predicate-based subtask rewards, which can be derived from the predicate checkers and annotated assets.
Training and demonstration collection costs
For RL training, we use a GPU-accelerated simulator, i.e., ManiSkill 3, to maximize the throughput. Using one RTX 3090 GPU, residual policy training averaged 6.5 hours per task. The shared base policy requires an additional 2 hours to pretrain once. After training, collecting 150 successful demonstrations with 32 parallel environments takes an average of 5.3 minutes per task. This is due to the high success rate and low trajectory length of the residual policy trained via RL.
Comparison against script-based demonstration generation
Graph2Demo incurs an additional cost for residual policy training, but the trained policies enable demonstration collection across parallel environments in GPU-accelerated simulation. During the collection of the 150 demonstrations, the RL-trained residual policies achieved a 100% success rate, with no rejected rollouts. With parallel execution and reliable rollout generation, Graph2Demo collects 150 successful demonstrations in 5.3 minutes on average across all 11 tasks, compared with 1 hour and 47 minutes on average for one task using the original RoboFactory scripts.
W2 (b): Substantial regressions on Camera Alignment and Take Photo lack a failure analysis
Thanks for highlighting this issue. We evaluate the teacher used for demonstration generation and the students trained on these demonstrations on the same held-out seeds. The teachers achieved success on both tasks, whereas the students achieved on Camera Alignment and on Take Photo. This indicates that the regression arises primarily during student policy distillation and execution, rather than from teacher failures on these evaluation episodes.
We further inspected the rendered teacher and student rollouts on Camera Alignment and Take Photo tasks, and observed that the students learn aggressive gripper motions from the teacher. In failed student rollouts, these aggressive motions lead to collisions with the camera, unstable grasping, or dropping the camera during grasping. To verify this, we add a smoothness penalty to the teacher's reward for each robot, i.e., , to encourage smooth motion, and then collect new demonstrations and retrain the student policies.
The success rate (%) for the students trained on smoothness-penalized demonstrations is shown below:
| Task | Original teacher | Student trained on original demonstrations | Student trained on smoothness-penalized demonstrations |
|---|---|---|---|
| Camera Alignment | 100.0 ± 0.0 | 53.0 ± 3.0 | 86.2 ± 2.2 |
| Take Photo | 100.0 ± 0.0 | 48.3 ± 3.8 | 62.9 ± 2.6 |
The smoothness penalty improves student success by on Camera Alignment and on Take Photo. These results support teacher motion style as a contributing factor to the student's performance regression, and narrow the gap relative to the strongest scripted baselines.
If the authors have addressed your concerns or provided clarifications, please join the discussion to indicate whether your concerns have been resolved or if you have any further questions.
We thank the reviewer for the thoughtful comments on the ablation, engineering costs and performance regression. We address these concerns below.
We agree that the original ablation confounds dependency-aware execution with intermediate reward shaping. To isolate the contribution of the dependency-aware coordination, we added two ablations:
Both ablations retain the same subtask rewards and target poses as Graph2Demo. For each ablation, we train residual policies, collect 150 successful demonstrations, and train diffusion policies using the same procedure as Graph2Demo. The table below reports downstream diffusion policy success rates and mean completion steps over successful episodes:
Since both ablations retain the subtask rewards and target poses, the residual policy can be trained to collect the demonstrations. However, the resulting downstream diffusion policies perform worse than those trained with Graph2Demo demonstrations. Global Sequential produces substantially longer trajectories. This limitation is particularly pronounced in Three Robots Stack Cube, where simultaneous grasping by multiple grippers is beneficial. Global Synchronization introduces unnecessary waiting. For example, an agent that has completed its current subtask must wait for all other agents before proceeding, even when all dependencies of its next subtask are satisfied. In contrast, Graph2Demo’s dependency-aware coordination allows agents to execute independent subtasks concurrently and advance asynchronously as soon as their dependencies are satisfied.
We will replace the “w/o Task Graph” ablation with these newly added ablations in the revised manuscript.