- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes an agent-guided synthetic-data pipeline for dexterous manipulation in which an LLM agent configures scene sampling, grasp retrieval, and planning, while deterministic filters and a whole-task physics verifier gate every trajectory; the result is a 16k-hour physically verified corpus used to jointly pre-train a dual-DiT world-action model (video dynamics + arm–hand action generation); the shared checkpoint is adapted per task via post-training on ~100 simulation or real-robot trajectories, and evaluated on six unilateral/bimanual tasks in both domains under base, position-, and visual-generalization settings, with ablations on data scale, adaptation, and sim–real composition.
Critical strengths
- The paper shows considerable engineering effort in constructing the data synthesis pipeline.
- The paper presents a comprehensive evaluation across the 6-task benchmark, covering aspects such as spatial and visual generalizability.
Critical weaknesses
- The conclusions of this work are questionable due to the narrow task distribution in the pretraining data and inadequate task evaluation experiments. a. The data synthesis method lacks novelty. Prior work such as HumanoidGen employs a similar method to synthesize dexterous manipulation demonstrations, sharing almost the same task distribution as this paper and also supporting a post-rendering step. b. The task distribution presented in the appendix reveals a limited range of tasks within the pretraining dataset, which suggests low task diversity and raises concerns about the effectiveness of pretraining on such data. c. The number of distinct assets used in data synthesis is not reported in the paper, which may further exacerbate the lack of diversity in the pretraining data. d. The simulation evaluation appears to be an in-domain assessment, as the model is evaluated on simulation tasks from which its post-training data was synthesized. This experimental design lacks sufficient evidence to demonstrate the broad effectiveness of the proposed data synthesis method. e. The paper does not provide results on any open-sourced simulation benchmarks, making it difficult to systematically assess the model's generalization capability to unseen tasks or environments compared to other baselines.
- Flaws in experimental design and a lack of focus in the experiments make the results unreliable. a. The comparison is unfair: the baselines (PI05, LDA-1B, Fast-WAM) were pretrained on parallel-gripper data but only post-trained on dexterous manipulation, whereas DexSimWAM is pretrained and post-trained entirely on dexterous manipulation ("dexterous-hand-native"). This experimental design fails to provide adequate support for the authors' claim, casting doubt on its validity. b. Nearly half of the experiment section is devoted to reporting aspects unrelated to the claimed main contributions. For the relevant experiments, the authors spend most of the text describing the experimental setup but fail to provide sufficient analysis of the reported results. c. Only one task is used to evaluate pretraining scaling performance, which makes the conclusion prone to bias. The paper is not ready for publication due to the writing issues mentioned in the question section below and the concerns mentioned above.
Questions for authors
- Could you provide evaluation results and comparisons with other models on established simulation benchmarks (e.g., DexVerse, HumanoidGen, GR-1 Tabletop Robocasa)?
- Can the proposed data synthesis method support sim-to-real policy transfer using only synthesized demonstrations?
- Line 204: Could you please clarify what the "cuRobo constraints and budgets" generated by the LLM refer to? Providing concrete examples would be helpful.
- Line 206: Given that the framework provides preference context to allow agents to apply different sampling strategies, is there any quality control mechanism in place? Could this flexibility introduce bias into the synthesized demonstrations, potentially affecting pretraining effectiveness?
- Line 211: The "Drawer state coverage" does not appear to be shown in the figure. Could you clarify this? Additionally, is there a distinction between the state representation of an articulated object like a drawer and that of a rigid object like a bottle?
- In the data generation process, the feasibility of dexterous grasping is guaranteed, but its semantics are ignored. Could this neglect affect the semantic alignment of the collected demonstrations? (For example, "pour the water from a pot" implies grasping the handle rather than the rim.)
- The task categories in the pretraining dataset appear to be limited, primarily focusing on drawer opening/closing, transportation of specific objects, and some pick-and-place tasks. Could you report the number of assets used in synthesizing the pretraining dataset, as object diversity is crucial for diversifying instructions in general-purpose training? Furthermore, could you please explain what the "paired failure" task refers to in the pretraining dataset?
- Could you please explain your experiment design in response to the points mentioned in the "weakness" section above? Questions in this part can be asked and discussed only if we ensure the experiment design is complete and fair.
The following are some additional writing and formatting issues in the paper:
- Figure 1: The font size in the pipeline part is too small to read clearly.
- Line 215: There is excessive white space.
- Line 205: The phrase "fixed scheduler" is confusing.
- Lines 359 and 369: The bolded sentences are unrelated to their corresponding main text.
Thank you for the detailed rebuttal addressing the weaknesses and questions I had previously asked. After reviewing your responses, I believe my W2, W4, Q1, Q2, Q3, Q4, Q6 and Q7 have been addressed (though for W4 specifically, I would advise you to remove that additional section from the revised manuscript and explore the other work as a separate project). I have additional comments and clarifying questions for the other points based on the new findings are as follows:
Given the time constraints, please note that I do not expect additional experiments or results here for any questions. However, additional clarifications by you would be helpful in better understanding the contributions for a final rating.