- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SimWeaver, a complete sim-to-real pipeline that automatically generates manipulation demonstrations for deformable objects, renders RGB videos, and trains an end-to-end policy (pi0.5). SimWeaver consists of four core components: SimWeaver-Sim, SimWeaver-Syn, SimWeaver-Real, and SimWeaver-Asset. The resulting trained policy can be directly transferred to the real world. Both the methodology and experimental sections are fairly comprehensive.
Although the method proposed in SimWeaver-Sim is very promising, I still have concerns about the simulation baselines, data synthesis, and experimental evaluation. Taking these concerns into account, my initial recommendation is Weak Reject.
Critical strengths
-
I consider SimWeaver-Sim to be the most important contribution. Achieving penetration-free cloth grasping is an important problem in robotics. Through the active-collision-region scheme, the authors achieve stable and replayable grasping simulation.
-
SimWeaver-Asset is also well developed, particularly in providing a complete set of physical simulation parameters, which is largely absent in other work.
Critical weaknesses
-
For SimWeaver-Sim, the authors should compare their approach with currently prevalent attachment-based grasping methods to demonstrate why friction-based grasping is better than attachment-based grasping. Attachment-based methods select cloth vertices based on their distance to the gripper and attach nearby vertices to it. If necessary, collisions between the gripper and the garment can be disabled to ensure simulation stability. This raises two further questions:
a. For the tasks selected in this paper, attachment-based simulation is sufficient, and friction-based grasping simulation is not necessary. The authors could try tasks that can only be handled by friction-based grasping simulation, such as unfolding a sleeve that has already been folded, to better demonstrate the advantages of SimWeaver-Sim.
b. The authors should discuss the relationship between simulated and real-world grasping success rates. If a grasp slips in simulation, could this be caused by limitations of the simulator itself, such as insufficient numerical solver accuracy or an inadequate friction model, even though the same grasp would succeed on the real robot? Conversely, if a grasp succeeds in simulation, does it necessarily succeed in the real world?
-
I do not fully understand the SimWeaver-Syn component. The authors should explain it in more straightforward language and avoid presenting engineering implementations as more substantial methodological innovations than they are. Specifically, I have the following concerns:
a. Why does TAS reduce Lift fail in the ablation study in Table 2? My understanding is that TAS selects better grasp pairs through the predefined graph G and uses F to select more feasible grasp points, for example, avoiding vertices on the underside or in folded regions. These seem to have little direct connection to Lift fail.
b. Graph G is essentially a set of manually defined, task-specific and object-category-specific landmark pairs. Presenting G as "topological relations" seems to overstate what it represents.
-
Regarding the sim-to-real experiments, I have the following questions:
a. What exactly are the object randomization ranges for all tasks? A model trained on 200 demonstrations largely overfits to a particular class of manipulation trajectories, and I would like to know to what extent it can generalize to different object positions and initial states.
b. Generalization to different positions can be achieved through large-scale synthesis of manipulation demonstrations in simulation. I do not understand why the authors use only 200 training demonstrations even in simulation.
c. If possible, the authors could discuss how to train using a mixture of simulated and real-world data.
Questions for authors
Please refer to Critical Weaknesses. Please point out any inaccuracies in my comments. If the concerns above are clarified, I will raise my score.
5 comments
[Part 2/3: W1(b), W2]
W1(b): Simulated vs. real grasp outcomes. We thank the reviewer and add the consistency between simulated and real grasps. Experiment 1 already includes the real robot; here we add the non-attachment simulators of Table 1, replaying the same bimanual garment grasp-and-lift open-loop on the real robot:
| Real | SimWeaver-Sim | Isaac Sim (PhysX) | Newton VBD | |
|---|---|---|---|---|
| Garment lifted | 10/10 | ✓ 10/10 | ✗ 0/10 (slips) | ✗ 0/10 (penetration / explosion) |
In both comparisons, SimWeaver-Sim's grasp outcomes agree with the real robot, and policies trained only on its demonstrations reach 91.3% real-world success (Sec. 7.4).
W2: Clarity of SimWeaver-Syn. We thank the reviewer and clarify Syn below. Demonstration synthesis needs an invariant to decide where to grasp. Rigid objects provide one through object pose. Deformables have no stable pose under deformation, wrinkling and occlusion, so prior methods rely on teleoperation, learned sampling and post-hoc filtering. Our key idea is to use topology as the invariant for deformables: whether a bimanual grasp pair can complete a task is decided by how the two points are connected on the object, not by their current positions. In plain terms, Syn repeats three steps. It picks a grasp pair allowed by the topology graph , checks that the pair is visible and reachable in the current state (), and executes and verifies the grasp, reselecting if it fails.
W2(a): Why TAS reduces Lift fail. We thank the reviewer for pointing this out; Table 2 mislabeled these failures. As the reviewer notes, filters infeasible grasp points; it is retained in w/o TAS, which removes only . Grasping is therefore unaffected: the cloth is lifted, but the selected pair, e.g., a diagonal, cannot flatten it. We add a Config fail category (cloth lifted and moved as planned, but wrong final state), distinct from Lift fail. All w/o-TAS failures fall into it (Config fail 43.7%, Lift fail 0%), and we will revise Table 2, the main text and the appendix accordingly.
W2(b): Why is topological. encodes how landmarks are connected on the object, e.g., whether two corners are adjacent along the boundary. However the cloth is crumpled or folded, this connectivity does not change, which is what makes it a topological property. It also decides in principle whether a task can succeed. Grasping two boundary-adjacent corners flattens the cloth, while grasping a diagonal pair cannot. Geometry cannot replace this, since crumpling may bring diagonal points closer than adjacent ones. The ablation confirms it: removing drops the pass rate from 89.7% to 56.3%.
In practice, constrains boundary segments rather than fixed points: a landmark that is occluded or folded inward can be replaced by the nearest reachable point along the same boundary edge, which preserves the connectivity and hence the topological constraint; we will further clarify this in the appendix. Landmarks are labeled once on a category template and transferred automatically to every asset of that category. Edges follow a connectivity rule set by the task, e.g., boundary adjacency for flattening. Since topology is a property of the object category, defining it once per category is inherent to the problem, and the same definition holds for every instance under any deformation. We will add this definition to Sec. 4.
[Part 3/3: W3]
W3(a): Randomization ranges and generalization. The ranges are listed below (full list in Appendix A, Table 4). They are determined by the object size, the workspace of the two arms and IK solvability: the Piper arms have a limited workspace, while the garments and bags are large and already occupy most of it, so the pose offsets are kept within the region where both arms can reach the grasp points with feasible IK. Because the objects are large, these offsets still move the grasp points widely across the 200 demonstrations:
| Silk grasping | Silk unfolding | Garment folding | Snack packaging | Garment unfolding | |
|---|---|---|---|---|---|
| Pose range | ±10 cm, ±15° | ±5 cm, ±10° + fold | ±5 cm, ±15° | ±2 cm, ±5° | – (hang-and-drop) |
| Grasp-point spread (cm) | 27×35 | 42×28 | 26×24 | 10×10 | 19×33 |
Coverage can be extended beyond this region in two simple ways: objects farther away can first be pulled back into it by a repositioning motion, or arms with a larger and more dexterous workspace can be used.
The policies also generalize beyond the training distribution. On silk grasping, trained with ±15° rotations, we test on the real robot under out-of-distribution 30° rotations, textures and lighting: the policy trained on 200 simulated demonstrations succeeds in all trials, while the policy trained on 100 real demonstrations reaches 8.7%, 13.0% and 69.6% (Sec. 7.5).
W3(b): Why 200 demonstrations. 200 demonstrations suffice because they are high-quality and sample-efficient. Under the same pose range (±5 cm, ±15°), SIM1's generated data reaches 0% real-world success on T-shirt folding at 200 samples and needs 10k samples, plus 200 teleoperated seed demonstrations, to reach 90% (their Fig. 8a); we reach 91.3% on garment folding with 200 synthesized demonstrations and no teleoperation. Our scaling curve also saturates at this budget: on silk grasping, real-world success rises from 60.9% at 50 to 87.0% at 100 and 100% at 200 demonstrations (Fig. 4a). Within our range, 200 demonstrations are thus already sufficient, and the range itself already covers the main operating region of the two arms. Objects placed farther away are hard for both arms to reach regardless of the amount of data, and can instead be pulled back into this region by a simple repositioning motion. We agree that more data can cover more positions and environment conditions; SimWeaver-Syn is built for this, generating 2,824 trajectories per day on a single 8-GPU server (Appendix I).
W3(c): Mixing simulated and real data. The two sources can be mixed directly: they share the same robot, cameras and action space, and the photometric augmentation is fitted to the real cameras (Sec. 5), so their distributions are consistent. Since simulated data provides broader coverage and stronger out-of-distribution generalization (Sec. 7.5), we suggest a sim-dominant mixture, with a small amount of real data to close the residual sim-to-real gap from effects not modeled in simulation, such as real lighting and scene details. We will add this discussion to the paper.
We will add all of the above to the revision. We hope these new experiments and clarifications address the reviewer's concerns, and we would appreciate it if the reviewer would reconsider the score in light of them. We are happy to answer any further questions.
[Part 1/3: W1]
We thank the reviewer for recognizing SimWeaver-Sim and SimWeaver-Asset, and address the comments in three parts.
W1: Comparison with attachment-based grasping. We add the following discussion and experiments.
Limitations of attachment. Attachment decides whether the cloth is grasped by a distance rule rather than by contact, so it produces grasps that are not physically possible. As DexGarmentLab notes of GarmentLab's attach blocks, "even minimal contact … can establish attachment and lift the garment, which is clearly unreasonable." Prior work likewise adopts attachment or anchoring because a lasting physical grasp could not be simulated, and acknowledges that it cannot represent realistic finger–cloth contact (Matas et al., 2018; Nayak et al., 2026).
The root cause is that frictional grasping of thin cloth is hard to simulate: in Isaac Sim, a position-controlled gripper tends to penetrate PBD cloth or let it slip, and attachments to articulation links are not supported. Pipelines such as GarmentLab therefore bind the cloth vertices near the fingertip to a proxy body and filter gripper–cloth collision, so whether the cloth is grasped no longer depends on whether the fingers pinch it.
Experiment 1: grasp offset (W1(a)). To isolate the grasp model, we re-implement in our own simulator GarmentLab's attach block (a 3 cm cube) and the attachment block provided in DexGarmentLab's code (an 8 mm-radius sphere), with sizes and overlap offsets taken from their code. The cloth, physics parameters and trajectory are identical, and the real robot replays the same trajectory. Each setting is repeated 10 times.
Entries: trials in which the cloth is lifted. ✓ / ✗: the simulated outcome matches / contradicts the real robot.
When the fingers close above the cloth or beside its corner, the real robot and SimWeaver-Sim miss, while both attachment schemes still lift the cloth even though the fingers never pinch it, which is the failure DexGarmentLab describes. Since only the grasp model differs, these false positives come from attachment itself, not from a particular simulator; shrinking the block only moves this boundary. Such imperfect grasps are the norm rather than contrived cases: on crumpled, self-occluded cloth, grasp points are never hit exactly, and policies evaluated in simulation grasp imprecisely. Attachment records them as successes, so demonstrations and evaluations include grasps that fail on the real robot. This also answers the first part of W1(a): Experiment 1 uses the silk from our silk-grasping task, so attachment is not sufficient for our tasks either.
Experiment 2: unfolding a folded sleeve (W1(a)). We also run the task the reviewer suggests: unfolding a sleeve folded onto the garment body (in simulation). All methods start from the same folded state and use the same grasp point and trajectory; each is repeated 10 times.
SimWeaver-Sim pinches only the sleeve and folds it back, leaving the body in place. Both attachment schemes also bind the body layer beneath the sleeve, because they select vertices by distance and cannot tell the layers apart; the whole garment is dragged along and the sleeve is not unfolded.
SimWeaver-Sim simulates real grasping through contact, which is exactly what attachment avoids. SimWeaver-Sim instead establishes reliable force interaction and coupling between the rigid gripper and the cloth: contact forces from the jaws enter the cloth solver correctly, and when the jaws compress the cloth, the active-collision-region scheme (Sec. 3.2) keeps contact and friction enabled while suspending the self-intersection repair that would otherwise conflict with them. The cloth is therefore held, or missed, according to how the jaws actually close on it. By coupling rigid–cloth contact reliably, SimWeaver-Sim solves at its root the problem that attachment works around.
We will add these discussions and experiments to Sec. 3 and the appendix.