- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposed the FabricBench benchmark for bimanual fabric manipulation. Proposed simulator built on Newton combines redesigned cloth-rigid contact with bidirectional force coupling, enabling several manuplation without artificial attachment. Benchmark included six representative policies reveals substantial remaining challenges in bimanual fabric manipulation. Overall, the paper is clearly written, easy to follow, and provides informative empirical findings.
Critical strengths
-
This paper proposed a contact-driven cloth simulator that avoids artificial attachment mechanisms. This makes grasping, holding, and release emerge from physical contact, reducing evaluation artifacts caused by heuristic attach detach rules.
-
This paper explicitly split the generalization problem into configuration shift and appearance shift. The experiments show that current policies are substantially more sensitive to changes in fabric configuration than to appearance changes, providing a useful diagnosis of current policy limitations.
Critical weaknesses
- The physical fidelity of the simulator is not fully validated against real-world fabric dynamics. Current experiments in paper show that proposed benchmark is behaviorally suitable for manipulation evaluation, but quantitative sim-to-real comparisons of cloth deformation, trajectories, contact forces, or other physical quantities are limited. This makes it difficult to assess how accurately the simulator reproduces real fabric dynamics.
Questions for authors
- Are the 10k assets actually utilized? The main experiments mostly study appearance and configuration shifts, while geometry and material generalization appear relatively limited.
- Could the authors provide stronger quantitative validation of the simulator’s physical fidelity, for example by comparing simulated and real cloth trajectories, deformation, or interaction forces?
- How extensively were the baseline simulators tuned? Since the comparison mainly uses released/default configurations, could task-specific tuning substantially improve their grasp/hold/release performance?
- How representative are the three discrete configuration levels C0/C1/C2 of the broader fabric configuration space? Could the conclusions change under a denser or more continuous range of deformation levels?
3 comments
Thanks for authors' response, clarification and additional experimental results.
For the Q1(b), I would encourage the authors includes more comprehensive evaluation of generalization in paper or supp, even the effect was relatively small, which would affect the reliability of proposed simulator.
Considering the exsiting evidence and additional experiments, I would not decline my rating, nor raise my rating.
Thank you for the follow-up and for your constructive suggestion. We agree that reporting these results strengthens the reliability of the simulator, and we will include the generalization evaluation in the revision. We appreciate your positive assessment of our work.
Best regards, The Authors
W1 & Q2: Quantitative physical fidelity. We add two matched sim–real comparisons with the same robot actions and initial fabric state. (i) A dual-arm grasp–hold–release trajectory is executed on the real robot and replayed in each simulator; only FabricBench reproduces the real grasp, hold and release (videos: https://anonymous.4open.science/w/Fabric-Anonymous-23A0/website/index.html). (ii) On RGBench (Hu et al., AAAI 2026), a public sim-to-real benchmark that replays real bimanual fold and grasp-and-lift trajectories and measures the per-frame Chamfer distance to real point clouds (lower is better):
Both will be added to Sec. 5.1.
Q1 (a): Use of the 10k assets. All 10k assets pass the same stability check and have each been simulated (Sec. 4.2; App. A). In training, Carry bag uses 10 bag appearance variants (App. E). In evaluation, for consistency, the standard setting uses the canonical asset, and the appearance axis uses 50 variants sampled from the held-out assets, with only appearance changed (Table 4).
Q1 (b): Geometry and material generalization. We thank the reviewer for this interesting question. Beyond the randomized handle stiffness of the bag (App. E), we varied stretch stiffness, bending stiffness and mass over 0.5–2×; this check is not in the paper because the effect was small: grasp-and-place and folding were largely insensitive, while flinging was sensitive mainly to air damping rather than these parameters. Since each asset carries an explicit material profile (App. A), the material profile can represent fabrics as different as cotton and leather, and we will continue to study this question. For geometry, we thank the reviewer for the suggestion: morphology variants are included in the released assets, and we will explore them in future work.
Q3: Tuning of baseline simulators. The baselines start from tuned settings, e.g., the cloth parameters released by DexGarmentLab for Isaac Sim PBD, and we further tuned mass and friction for Isaac Sim and the step size, iterations and contact parameters for VBD and Style3D. All simulators use the same solver and contact settings across tasks, without task-specific tuning. In our sweeps, tuning did not remove these failures (App. B.7), because the failures come from how these simulators couple cloth and rigid bodies. Isaac Sim PBD resolves cloth–rigid contact only at particles padded by a rest offset, with dynamic friction only, so the closing jaws push the fabric's particles aside instead of pinching them; DexGarmentLab likewise reports that grippers "penetrate the garment without achieving effective lifting" and adds adhesion. VBD and Style3D detect gripper contact only at cloth vertices, so vertices sink into the fingers and the fabric is not released; coupled to an external rigid solver, they also apply contact forces to the cloth only, so a bag's contents are not carried. FabricBench instead fixes these failures in the contact mechanism itself, through triangle-level detection, anchored friction and bidirectional coupling (Sec. 3), so grasping, holding and releasing are ensured by the mechanism and are robust to its physical parameters within the tested ranges.
Q4: Representativeness of C0–C2. C0–C2 do not cover the full configuration space; they are three designed difficulty levels, each a seeded distribution of initial states (Fig. 3; App. C), spanning the range where policies remain distinguishable. On Flatten T-shirt, success falls from 70–92% (C0) to 16–62% (C1) and 6–14% (C2), with different policy rankings per level (Table 4). Harsher levels would not change this trend: success is already low at C2 (0–2% on Flatten silk), and more random initializations, such as drops from randomly chosen points, bring all policies to nearly zero. Sec. 6 already notes this limitation, and denser or continuous levels are future work.
We thank the reviewer again for the positive assessment and hope these additions address the remaining concerns.