acceptodds
Withdrawn from ICLR 2027

DoughMaster: Demonstrate Once, Then Calibrate and Scale Dough Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Robot demonstrations are expensive, and simulation is a popular way to multiply them. This works well for rigid objects, but not for dough. Every press permanently changes the dough, so a simulator with the wrong material parameters drifts further from reality at every step of a long task. Calibrating the simulator usually requires extra data, such as dedicated probing motions or random exploration. We show that this extra data is unnecessary: the task demonstrations already contain everything needed to identify the material. Our key idea is to use the same demonstrations twice. We first replay them in a differentiable simulator to identify the physical parameters of the dough, and then generate many new demonstrations in the simulator they calibrated. To make identification work with real RGB-D data, we compare the simulation only with the surface each camera can see, and we supervise how the shape changes rather than its absolute position. The calibrated simulator then creates physically consistent variations in dough placement, camera pose, and appearance. On a real dual-arm robot shaping dough into letters, our method identifies material parameters that reproduce the observed deformations more accurately than a DPSI-style baseline using the same data (4.78 vs 5.10,mm Chamfer distance). With only two teleoperated demonstrations per letter, our augmented data trains a VLA policy that reaches a 54.4% success rate, compared with 24.4% when trained on five times as many real demonstrations.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, deformable object manipulation, real-to-sim-to-real

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MRQ3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The core idea is: use the same real robot demonstrations twice: first to identify the dough’s physical properties, then to replay and augment those demonstrations in a calibrated simulator.

Their pipeline has three stages. First, they teleoperate a dual-arm robot to shape dough into letters. Second, they replay the recorded tool motions in a differentiable Material Point Method simulator and optimize the dough’s material parameters. Third, they use that calibrated simulator to generate many new physically consistent demonstrations with spatial, camera, and appearance variations.

Critical strengths

The main empirical results are strong. For system identification, their calibrated simulator achieves 4.78 mm Chamfer distance versus 5.10 mm for their DPSI-style baseline. The improvement is modest but consistent on the bulk surface metrics. The more impressive result is downstream policy learning. With only 2 real demonstrations per letter, simulation augmentation raises real-robot success.

The paper, overall, introduces a data-efficiency pipeline for deformable manipulation that turns scarce demonstrations into both system-identification data and simulation seeds.

Critical weaknesses
  • The central pipeline is established Real2Sim2Real [1] for deformable objects, with only an incremental change in how system-ID data are obtained. The paper repeatedly presents “demonstrate once, identify and augment” as its key insight. But from a systems perspective, the procedure remains:
\rightarrow \hat{\theta}_\text{sim} \rightarrow D_\text{sim} \rightarrow \pi \rightarrow \text{real deployment}.$$ The distinction is that $D_\text{real}$ serves simultaneously as task data and system-ID data. That is useful, but it does not obviously constitute a fundamentally new learning paradigm. - The motivation for robotic dough letter shaping is extremely weak. This is perhaps the biggest problem for me. The paper says deformable elasto-plastic manipulation is important, but then evaluates shaping ~40 g pieces of dough into the letters A, I, C, L, and R. What real robotic application requires a robot to form alphabet letters out of dough? It is not obvious that this benchmark represents: - food preparation, - industrial forming, - clay manufacturing, - surgical manipulation, - packing, - household manipulation, - or any other practically meaningful task. The letters appear to be chosen because they produce convenient different deformation trajectories, not because they represent an application. That is okay for a benchmark, but then the paper needs to argue why success on this benchmark teaches us something general. - There is almost no evidence that the method generalizes beyond one dough formulation. The paper makes claims about “elasto-plastic material identification” and deformable manipulation broadly, but apparently fits one set of material parameters for the dough and evaluates on that same material/task setup. [1] Real2Sim2Real: Self-Supervised Learning of Physical Single-Step Dynamic Actions for Planar Robot Casting by Lim et al.
Questions for authors
  • If the main novelty is “reuse task demonstrations for system identification,” why is this presented as a new framework rather than a data-reuse strategy within an existing Real2Sim2Real framework?
  • What scientific or practical problem is solved by shaping dough into letters? Please motivate why these tasks constitute a meaningful benchmark for deformable manipulation rather than an arbitrary visually appealing demonstration.
  • What properties of A/I/C/L/R test generalizable deformable-manipulation capabilities rather than memorization of five specific shape trajectories?
  • Why should the reader believe this is a general material-identification method rather than a simulator-tuning procedure for one batch of dough?
Reviewer_1o8jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DoughMaster reuses task demonstrations to calibrate a differentiable MPM dough simulator and generate augmented policy-training data. Visible-surface matching and deformation-increment supervision support calibration. Experiments evaluate simulation accuracy and real-robot performance on five letter-shaping tasks.

Critical strengths
  1. Reusing demonstrations for both calibration and augmentation offers a practical way to reduce data collection for plastic deformation.
  2. The real-robot gains are substantial: 54.4% success versus 24.4% using five times more real demonstrations, with 50 trials per letter per condition (Table 2).
Critical weaknesses
  1. Table 2 lacks policy comparisons using uncalibrated, DPSI-calibrated, and proposed simulator parameters with the same augmentation budget. The policy gains therefore do not establish the contribution of the proposed calibration.
  2. Table 1 evaluates the fitting frames, while Sec. 4.3 tests geometric recovery rather than material identification. Neither directly establishes that the proposed loss improves dynamics prediction on unseen demonstrations.
Questions for authors
  1. With augmentation and training budgets fixed, how does policy success change across uncalibrated, DPSI-calibrated, and proposed parameters?
  2. On held-out demonstrations, how accurately do the fitted dynamics predict deformation, with and without the deformation-increment loss?
Reviewer_8JYYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

DoughMaster uses task demonstrations to calibrate a simulator and then generate additional training data for dough manipulation. The method reconstructs an initial particle representation from RGB-D observations, replays the recorded tool motions in a differentiable MPM simulator, and fits five material parameters. The fitting objective combines robust surface matching with a deformation-increment loss. A camera-visible surface proxy accounts for the difference between the volumetric simulation and the surfaces observed by the cameras. The fitted simulator generates trajectories with different spatial layouts, camera poses, and appearances. These trajectories are used to fine-tune MolmoAct2. On five letter-shaping tasks, the paper reports 54.4% success using augmentation from two demonstrations per letter, compared with 24.4% using ten real demonstrations per letter. The identification method also reduces Chamfer distance from 5.10 mm for DPSI-on-Genesis to 4.78 mm.

Critical strengths
  1. Table 2 reports an increase from 24.4% to 54.4% success, with improvements on all five letters. The augmented approach uses two source demonstrations per letter, while the real-only baseline uses ten. This is a useful comparison given the effort required to collect long dough-shaping demonstrations. The evaluation uses 50 trials per letter for each of four conditions, giving 1,000 physical trials in total. Spatial augmentation produces the largest improvement, and camera and appearance augmentation further improve the aggregate success rate. These results provide meaningful evidence that the generated data help the policy.

  2. The same recorded interactions provide both the calibration data and the trajectories to augment. Keeping the MPM state across episodes is appropriate for dough, since earlier contacts affect how it responds later. The visible-surface proxy addresses a concrete mismatch between the internal simulation particles and the surfaces captured by RGB-D cameras. The deformation-increment loss also gives the fitting process information about shape changes relative to the initial observation. Section 4.3 provides a controlled test of these loss choices under observation bias and missing points. At 25% occlusion, the combined objective achieves 2.007 mm recovery error, compared with 2.617 mm without deformation supervision and 2.885 mm without robustification. The authors report variation across three seeds and clearly state that this experiment evaluates geometry recovery within a restricted candidate family.

Critical weaknesses
  1. All augmented conditions in Table 2 use the proposed calibrated simulator. There is no policy comparison using the initial material parameters or parameters fitted by DPSI-on-Genesis under the same augmentation settings. It is therefore unclear how much of the improvement comes from material calibration and how much comes from the additional spatial and visual diversity. Table 1 does not resolve this question. Section 4.2 evaluates the simulator on the supervised boundary frames used for fitting. The reduction from 5.10 mm to 4.78 mm shows better agreement with those observations, but the paper does not report whether the fitted parameters improve prediction on a held-out demonstration or an unseen action sequence. The P95 error is also essentially unchanged. The loss ablation in Section 4.3 is useful, but it optimizes three geometric coefficients rather than material parameters through MPM. It therefore does not establish that the proposed loss improves the actual identification problem. In Table 1, the two methods differ in their point sets, target downsampling, loss, and optimizer, so the source of the improvement remains unclear.

I would like to see a policy comparison using initial, DPSI-fitted, and proposed-fitted parameters with matched augmentation and training settings. A held-out trajectory evaluation would also help establish whether the calibration captures dynamics that transfer beyond the fitting data.

  1. The policy-training setup needs more detail. The paper does not report the learning rate, batch size, number of updates, trainable components, real/synthetic sampling mixture, or number of independently trained policies. Appendix C gives the number of augmentation candidates, but not how many pass screening or how many are used in each Table 2 condition. This makes it difficult to separate the effects of augmentation type, dataset size, and training exposure. The three seeds in Section 4.3 concern the geometry diagnostic and do not establish variation across policy-training runs.
Questions for authors
  1. How much does calibration affect policy success? Could you compare augmentation using the initial parameters, DPSI-on-Genesis fitted parameters, and your fitted parameters while keeping the source demonstrations, augmentation settings, accepted training-data amount, and policy-training budget matched? A focused comparison on a representative subset of tasks would already be informative.

  2. Do the fitted parameters improve prediction on unseen trajectories? Were any complete demonstrations or action sequences excluded from identification and used for evaluation? Holding out points in the geometry-recovery experiment does not test this. It would also help to evaluate the proposed loss components in the actual MPM identification problem.

  3. What determines the absolute scale of the material parameters? From Appendix A.1, jointly scaling the parameters as

\longmapsto (cE,\nu,c\rho,cY,c\eta)$$ appears to leave the plastic correction unchanged while scaling stress and mass equally. Under the stated prescribed-tool-motion setting, this suggests that geometry alone may not uniquely determine the absolute parameter values. Does measured dough mass or another independent constraint enter the fitting process? Finding parameters that predict deformation well can still be useful, but the interpretation of the recovered values should be clear.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.