- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies 3D part assembly guided by multimodal instruction manuals, addressing step-to-part matching, final-pose prediction, and assembly trajectory generation conditioned on a given starting pose. The authors introduce AssemblyVerse, which unifies 3D assemblies from five sources and constructs assembly procedures by reversing geometrically verified disassembly trajectories, accompanied by generated visual and textual instructions.
The proposed AssemblyMind builds on Qwen3-VL-2B with a point-cloud encoder and represents trajectories as discrete pose tokens. It predicts the final pose before intermediate waypoints and uses staging-randomized supervised training to improve final-pose accuracy and trajectory generation across starting configurations. GV-GRPO subsequently optimizes trajectories using a reward that requires both a correct final pose and passage of the geometric validity check. The main contributions are a multi-domain dataset of assembly procedures and a multimodal framework integrating part matching, target-pose prediction, and trajectory generation.
Critical strengths
- The task formulation is valuable. Beyond predicting where parts belong, the paper considers how each part moves from a given starting pose to its target placement. Subsequent predictions use the previously predicted assembly state, allowing evaluation to reflect the effects of accumulated errors.
- The dataset brings together everyday objects, LEGO sets, CAD assemblies, and industrial assemblies, aligning part geometry with step-wise visual and textual instructions and geometrically verified trajectories. Validating candidate disassembly paths before allowing a VLM to select the next part combines semantic selection with geometric feasibility checks and provides a useful resource for further research.
- The method is well motivated, and the ablations in Table 4 support the effectiveness of the overall training approach. Predicting the final pose first gives intermediate trajectory generation an explicit goal, while staging randomization reduces reliance on fixed paths from reference starts. Adding these components substantially improves final-pose accuracy and trajectory validity under free-space starts, although the independent contributions of token reordering and final-pose loss weighting require further clarification.
- GV-GRPO rewards final-pose correctness and geometric validity without requiring agreement with a single reference path, which is appropriate for tasks with multiple feasible trajectories. Its benefits also appear in assembly-level metrics: in Table 2, E2E success increases from 14.78% to 17.06% under ground-truth order and free-space starts. The cross-dataset experiments provide additional encouraging evidence of transfer.
Critical weaknesses
- The benefits of direct trajectory generation over learned target poses followed by conventional motion planning remain unclear. The experiments show that AssemblyMind outperforms the evaluated learning-based baselines, but do not include a comparison in which the same model predicts the final pose and a planner such as RRT-Connect connects the start to that pose. Holding target-pose prediction quality fixed would allow a more direct assessment of the success-rate and inference-time benefits of learned trajectory generation, helping distinguish improvements in final-pose prediction from those in path generation.
- The coverage of starting-pose generalization and reinforcement learning optimization is limited. The free-space evaluation described in the appendix primarily changes part translations along the six coordinate-axis directions, while GV-GRPO sampling holds rotations fixed and explores only translations. The results therefore support adaptation within the specified starting-pose distribution, but provide limited direct evidence for arbitrary initial orientations or more general 6-DoF starting conditions. Testing randomized initial rotations and explaining the decision to hold rotations fixed would help establish the method's scope.
- The contributions of language inputs and pose-token reordering need clearer ablations. Difference images and point clouds already provide substantial part-identification and geometric information, but the paper does not evaluate removing or shuffling textual instructions, leaving the additional benefit of language unclear. Furthermore, final-pose tokens receive a fivefold loss weight, and Table 4 does not explicitly state whether this weight is held constant across serialization orders. Clarifying this setting is necessary to determine whether the improvement in final-pose accuracy can be independently attributed to token reordering.
Questions for authors
- Could the authors add a baseline that uses AssemblyMind to predict the final pose and RRT-Connect to generate the path, comparing success rates and inference time? This would help assess the necessity and practical benefits of directly generating trajectories.
- Could the authors evaluate randomized initial rotations? Why does GV-GRPO sampling hold rotations fixed and explore only translations, and could this restriction affect assembly tasks that require substantial rotational motion?
- Could the authors provide ablations that remove or shuffle textual instructions to assess their contribution beyond difference images and point clouds? In the pose-token reordering ablation, is the final-pose loss weight held constant across serialization orders?