acceptodds
Withdrawn from ICLR 2027

DROP-Op: Generalized Passive Response for Object Dynamics Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting object dynamics from sparse interactions requires a representation that captures how responses vary across space and actions. We introduce generalized passive response (GPR), an action-to-displacement response map defined for a given initial state and interaction interface. DROP-Op identifies a finite-dimensional GPR state by fitting spatially varying coefficients in a learned response basis. The fit combines geometry-based evidence sharing with response-dependent regularization, and the identified state is reused to predict responses to new actions under compatible conditions. We evaluate DROP-Op on a simulated grasping benchmark with individual and combined shifts in geometry, material properties, actions, and grippers. On the 46-panel confirmation set, DROP-Op reduces mean endpoint and translation-centered displacement errors by 31.6% and 30.9%, respectively, relative to ALPaCA-MSE, the baseline with the lowest mean endpoint error. Matched attention and pooled-identification controls also yield higher prediction errors. Additional backbone and external-task studies show task-dependent benefits of adapting the identification procedure. Reusing the identified state amortizes identification cost over repeated queries.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
object dynamics prediction, physical representation learning, neural operators, system identification, out of distribution generalization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YS9QOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper focuses on predicting object dynamics from sparse interactions. The authors argue for the need for a representation capable of capturing how responses vary across space and actions. To this end, they introduce Generalized Passive Response (GPR) and the DROP-Op method. Evaluated on a simulated grasping benchmark, DROP-Op achieves a 31.6% reduction in mean endpoint error and a 30.9% reduction in translation-centered displacement error compared to the baseline model, ALPaCA-MSE.

Critical strengths
  1. The experimental setup in the paper is described in detail, and the coverage is both extensive and thorough.
  2. The interactive visualizations provided via the link are commendable.
Critical weaknesses

The writing and presentation of this paper are weak, with several confusing aspects:

  1. The term "DROP-Op" appears rather abruptly in the abstract, and the motivation is not sufficiently laid out.
  2. The visual presentation throughout the paper is cluttered (especially in Figure 2); the boundaries between modules are unclear, and the directions of the arrows are ambiguous.
  3. The contributions of the paper are not effectively articulated; while a grasping benchmark is presented as a primary contribution, there is no intuitive comparison to demonstrate its advantages over other existing approaches.
Questions for authors
  1. Please elaborate on the significance of the proposed method for deployment in real-world robotic experiments, and discuss the "sim-to-real" gap.
  2. The “Introduction” mentions several relatively recent methods (specifically those from 2026); please explain why these recent approaches were not included in the baseline comparisons.
  3. The paper identifies the grasping benchmark as a primary contribution; please further elaborate on the advantages of this proposed benchmark compared to existing alternatives.
  4. The writing and the presentation of figures in the paper are limited and require further improvement.
3 comments
AuthorsOpenReview ↗

Response to Reviewer YS9Q (1/3)

We sincerely appreciate Reviewer YS9Q for the positive comments on the breadth of our experiments and the interactive visualizations. Your suggestions on motivation, presentation and comparisons have helped us clarify the contributions and strengthen the evaluation.

[Q1] What does DROP-Op mean, and what motivates it?

Critical Weakness 1: The term "DROP-Op" appears rather abruptly in the abstract, and the motivation is not sufficiently laid out.

DROP stands for Dynamics from Responses for Object Prediction; Op denotes the operator formulation. We now introduce the goal first: understanding an unfamiliar object's physical dynamics from a few interactions to predict motion and deformation under new actions.

GPR represents action-dependent responses without separately estimating every physical property. DROP-Op identifies a finite-dimensional GPR state through a shared network that produces a base displacement prediction and a response basis, and an identification step that fits spatial coefficients from sparse observations. The coefficients are reused for new actions under the same initial configuration and gripper and contact settings.

Changes in the revised manuscript: Motivation and name in the Abstract/Introduction, pp. 1–2; two components in the Sec. 3 opening, p. 3.

[Q2] How have the figures and module connections been clarified?

Critical Weakness 2: The visual presentation throughout the paper is cluttered (especially in Figure 2); the boundaries between modules are unclear, and the directions of the arrows are ambiguous.

Question 4: The writing and the presentation of figures in the paper are limited and require further improvement.

We have redrawn Figs. 1–4 and revised their captions. Fig. 2 uses distinct backgrounds and direct arrows to separate the shared network, coefficient identification and prediction. Its intermediate quantities follow the Methods prediction equation.

FigureWhat the revised figure explains
Fig. 1, p. 2Training on benchmark interactions, identifying a response state, predicting dynamics, and evaluating predictions on ID and OOD tasks
Fig. 2, p. 4(a) The shared network produces the base prediction and basis; (b) observed residuals determine the coefficients; (c) the new base prediction is combined with a correction formed from the new basis and fixed coefficients
Fig. 3, p. 6ID evaluation and individual or combined shifts in objects, actions and grippers
Fig. 4, p. 8Recorded interaction sequences, predicted deformation and displacement errors

Changes in the revised manuscript: Figs. 1–4 and captions, pp. 2, 4, 6 and 8; corresponding text in Secs. 3, 4.1 and 4.2.

[Q3] What are the main contributions?

Critical Weakness 3: The contributions of the paper are not effectively articulated; while a grasping benchmark is presented as a primary contribution, there is no intuitive comparison to demonstrate its advantages over other existing approaches.

We seek a reusable representation of an unfamiliar object's dynamics from a few interactions, enabling prediction under new actions. Each contribution addresses a difficulty in obtaining this capability.

GPR reduces reliance on explicit physical-parameter estimation. Selecting properties and estimating their spatial distributions requires modeling effort, and the appropriate parameterization may change across tasks. GPR instead describes the action-to-displacement response, capturing the combined effects of geometry, material and interaction conditions.

DROP-Op identifies and reuses the response state. Pooling sparse observations can lose spatial response information, and reprocessing them for each query adds computation. A shared network learns a base displacement prediction and response basis. Local coefficient fitting uses geometry-based evidence sharing and response-dependent regularization to retain spatial information and stabilize sparse fits. Reusing these coefficients avoids repeated identification for compatible actions.

The benchmark tests individual and combined factor shifts. Accuracy in one configuration does not establish generalization when several factors change together. We vary object, action and gripper factors separately and jointly under the same observation budget.

GPR defines what to represent, DROP-Op provides identification and reuse, and the benchmark evaluates the resulting predictive generalization.

Changes in the revised manuscript: Introduction, pp. 1–2; Secs. 3.1–3.3, pp. 3–5; Sec. 4.1 / Fig. 3, pp. 5–6.

AuthorsOpenReview ↗

Response to Reviewer YS9Q (2/3)

[Q4] What does the benchmark add to existing evaluation settings?

Critical Weakness 3: The contributions of the paper are not effectively articulated; while a grasping benchmark is presented as a primary contribution, there is no intuitive comparison to demonstrate its advantages over other existing approaches.

Question 3: The paper identifies the grasping benchmark as a primary contribution; please further elaborate on the advantages of this proposed benchmark compared to existing alternatives.

Deformable grasping provides reproducible actions while retaining the coupled effects of geometry, material layout, motion and deformation. The following comparison explains our evaluation focus:

Related benchmarkExisting evaluationWhat our protocol examines
DefGraspNetsQuasistatic stress and deformation prediction from meshes, supplied elastic modulus and grasp force; tests unseen grasps, moduli and objectsDisplacement trajectories predicted from four observed grasps, with physical properties inferred implicitly; also varies material layout and gripper configuration
ACIDDynamics prediction and manipulation from partial RGB-D observations across six plush-toy types and 78 variantsResponse prediction from the same sparse point observations while object, action and gripper factors vary separately and jointly
DensePhysNetPhysical representations learned from sliding and collision observations, evaluated through property decoding and manipulationFull 3D displacement trajectories and generalization to controlled geometry–material and object–gripper combinations

We vary objects (shape, size, topology, stiffness and material layout), actions (location, angle, closure and timing), and grippers (finger count, geometry and compliance), separately and together. This tests individual shifts and unseen geometry–material and object–gripper combinations, with the same four supports, four observation times and 32 anchors for each predictor.

A model may handle a new shape and a changed material separately yet fail when both change. Our protocol reveals such differences under a common prediction target and observation budget, complementing the perception and manipulation questions addressed by the benchmarks above.

Changes in the revised manuscript: Sec. 4.1 / Fig. 3, pp. 5–6; Appendix B.1 / Tables 6–8, pp. 21–22; related benchmarks in Appendix B.6 / Table 12, p. 30. The comparison table is provided in this response.

[Q5] What do the real-data results show about sim-to-real transfer?

Question 1: Please elaborate on the significance of the proposed method for deployment in real-world robotic experiments, and discuss the "sim-to-real" gap.

We added experiments on GAUGE, using real-world textile-bending measurements across six materials. Mean EPE (mm):

Window / trainingDROP-OpALPaCA-MSEResponse attention
Loading / real only2.25652.89984.5822
Loading / simulation + real2.14822.42493.1884
Full video / real only4.378910.767232.6302
Full video / simulation + real3.750110.88947.7225

DROP-Op's advantage persists on real measurements. Simulation pretraining followed by real-data adaptation further reduces EPE by 4.8% on loading sequences and 14.4% on full videos relative to real-only training. Both conditions use the same amount of real-data training; pretraining is additional.

The remaining sim-to-real gap concerns the learned dynamics and observations. Simulation-only transfer remains inaccurate: four supports cannot compensate for a basis that does not cover the target responses. Real-data training adapts both the shared predictor and basis. Deployment must also handle tracking errors, occlusion and changing tools or contacts. The material splits were used during development, so these marker-based results provide internal transfer validation.

We have not yet evaluated closed-loop robot manipulation. Encouraged by these results, we plan to further expand our research to real-world robotic experiments.

Changes in the revised manuscript: Sec. 4.5, p. 10; Appendix F.5, pp. 44–47, including Tables 23–27 and “Simulation-to-real gap,” p. 46; future directions in Sec. 5, p. 10.

AuthorsOpenReview ↗

Response to Reviewer YS9Q (3/3)

[Q6] Why were some recent methods absent, and what comparisons have been added?

Question 2: The “Introduction” mentions several relatively recent methods (specifically those from 2026); please explain why these recent approaches were not included in the baseline comparisons.

The original paper compares DISCO and EqCollide on their released tasks. Other recent methods require video, volumetric meshes or numerical solvers and must be adapted to our inputs: initial geometry, gripper commands and sparse responses.

We have completed NFF, LPWM and MAVEN grasp adaptations using the original training data, common sparse observations and the same 138-pass training budget. Mean errors on the original 46-panel confirmation set are shown below.

NFF (three seeds):

MethodEPE (mm)dEPE (mm)
NFF2.92792.4957
DROP-Op0.77560.7222

LPWM and MAVEN(one seed) :

MethodEPE (mm)dEPE (mm)
LPWM3.36602.7746
MAVEN2.58722.4475
DROP-Op0.77680.7262

DROP-Op lowers mean EPE by 76.9% and 70.0% relative to LPWM and MAVEN, respectively, and has lower EPE and dEPE in all 46 panels against both adaptations. Parameter counts are comparable.

Changes in the revised manuscript: Required inputs and retained components: Appendix C.5, p. 35. Added baseline results: Appendix C.4 / Tables 18–19, pp. 34–35. External comparisons: Appendix F.4, pp. 43–44. The NFF, LPWM and MAVEN results are provided above; their insertion into Table 1 and IoU evaluation remain pending.

[Q7] How has the overall writing and experimental narrative been revised?

Critical Weakness 1: The term "DROP-Op" appears rather abruptly in the abstract, and the motivation is not sufficiently laid out.

Critical Weakness 2: The visual presentation throughout the paper is cluttered (especially in Figure 2); the boundaries between modules are unclear, and the directions of the arrows are ambiguous.

Question 4: The writing and the presentation of figures in the paper are limited and require further improvement.

The Abstract and Introduction now connect the predictive goal to the effort of physical-property estimation, the difficulty of using sparse observations, and the cost of repeated processing. They then explain how GPR and DROP-Op address these challenges and how the benchmark evaluates generalization (Q3–Q4).

Methods explains the setup, coefficient identification and training of the shared network, with definitions before use and one connecting prediction equation. Figs. 1–4 explain the task, method, generalization and predictions (Q2). Fig. 5 compares accuracy and the cost of first and repeated queries; Fig. 6 tests identification across backbones.

Experiments retain their original order: generalization and efficiency, representations, identification versus conditioning, transfer across backbones and tasks, and reuse. Table 1 reports 31.6% lower EPE and 30.9% lower dEPE than ALPaCA-MSE; Tables 2–3 examine mechanisms; reuse tests measure repeated-query savings. Physical-setting, support-selection and GAUGE studies extend these results. The completed NFF, LPWM and MAVEN comparisons are reported in Q6.

Changes in the revised manuscript: Abstract/Introduction, pp. 1–2; Methods, pp. 3–5; Secs. 4.2–4.6, pp. 7–10; added studies in Secs. 4.1–4.3, pp. 6–8, and Sec. 4.5, p. 10; Appendices B.5–B.6, C.4–C.5, D.3, F.5 and H.4–H.5. Figs. 1–6: pp. 2, 4, 6, 8, 8 and 10.

Reviewer_5HHROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is incoherent. It talks very abstractly about understanding object responses to actions, but seems to focus entirely on understanding the change in shape of deformable objects when pinched with simple floating fingers acting as a simulation of a robot gripper.

The method is impossible to understand. The paper claims it's mapping from object state and action at time t to displacement. However, it never formalizes how the state is defined, what object representation is being used (I think it might be a tetrahedral mesh, but there's no way to know), or how actions are encoded. The method description goes on to say "the network learns", but this is the first mention of any sort of network in the paper, so I am not sure what is going on. Equation (2) factorizes the function from equation (1) to have different components, but again no details of these are given.

The method description then continues to discuss fitting "coefficients" but never says what these coefficients are. It then talk about each "anchor", without defining what an anchor is. Equations (3) and (4) then go on to define a minimization problem that are not clearly connected to the above parts of the approach.

The inputs to the method being proposed are never clearly defined and components are mentioned without definition or any direct connection to the previously discussed sections.

It's meaningless to examine the experiments and results in any detail given there's no clear statement of what was actually proposed in the paper.

Critical strengths

I have nothing positive to say about this submission.

Critical weaknesses

The paper is incoherent. The approach makes no sense and as such the experiments can't even be critiqued.

Questions for authors

I have no direct questions for the author. The paper needs to be entirely rewritten to provide any level of clarity as to what was actually done.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 5HHR (1/3)

We sincerely thank Reviewer 5HHR for the detailed comments on the clarity of our formulation and method. We apologize that the original presentation made the approach difficult to follow. We have rewritten Methods, clarified the definitions and redesigned the figures.

[Q1] Why use deformable grasping to study object responses?

Paper Summary: It talks very abstractly about understanding object responses to actions, but seems to focus entirely on understanding the change in shape of deformable objects when pinched with simple floating fingers acting as a simulation of a robot gripper.

Thank you for emphasizing the relationship between goals and experiment design. Our goal is to learn an object's physical dynamics from a few interactions and predict its motion and deformation under new actions. We choose deformable grasping as a controlled entry point: local contact produces spatial deformation and overall motion that jointly depend on shape, material properties and interaction conditions, and the results show that these settings pose substantial challenges for generalization. In future work, we will examine more complex and general situations.

The benchmark varies objects (shape, size, topology, stiffness and material layout), actions (location, angle, closure and timing), and grippers (finger count, geometry and compliance), separately and together. Original confirmation results illustrate the difficulty. Mean EPE (mm):

MethodV2-L1V2-L2
ICON0.94731.6622
DROP-Op0.61091.0843

For example, ICON's error increases on V2-L2, which adds contact, motion-rate, actuation, gripper and spatial-mechanics shifts. DROP-Op reduces L2 EPE by 34.8%. Thus, prescribed grasping actions still allow challenging tests of generalization. We have clarified this scope and plan to extend evaluation to rigid objects and broader interactions.

Changes in the revised manuscript: Introduction, pp. 1–2; Sec. 4.1 / Fig. 3, pp. 5–6. Results: Appendix C.1 / Table 13, pp. 30–31.

[Q2] How are the state, object representation and action defined?

Paper Summary: The paper claims it's mapping from object state and action at time t to displacement. However, it never formalizes how the state is defined, what object representation is being used (I think it might be a tetrahedral mesh, but there's no way to know), or how actions are encoded.

We predict the displacement trajectory from a fixed initial configuration and a commanded action trajectory. Geometry is go=xnn=196g_o=\\{x_n\\}_{n=1}^{96}, where xn∈R3x_n\in\mathbb R^3 are surface vertices selected by farthest-point sampling and tracked across grasps. These coordinates are the predictor's input; the tetrahedral mesh is used by the simulator.

An action u=(q0,…,q27)u=(q_0,\ldots,q_{27}) gives commanded positions qt∈RG×3q_t\in\mathbb R^{G\times3} of GG gripper points. The target is do(xn,t;u)=χo(xn,t;u)−xnd_o(x_n,t;u)=\chi_o(x_n,t;u)-x_n, where χo\chi_o is material point nn's position at time tt. Supports and queries share initial configuration, object properties, and gripper and contact conditions. Q5 defines the response state inferred from supports.

Changes in the revised manuscript: Definitions in Sec. 3.1, p. 3; sampling and encoding in Appendix A.4, pp. 18–19.

[Q3] What is the network and how is it learned?

Paper Summary: The method description goes on to say "the network learns", but this is the first mention of any sort of network in the paper, so I am not sure what is going on.

DROP-Op has two components: a shared network produces a base displacement prediction and response basis; identification fits coefficients that combine the basis values into a displacement correction.

The geometry encoder computes each point's features from its position and the object's local and global geometry. The action encoder represents gripper commands, and a temporal adapter incorporates their history. A decoder combines these features, with two heads producing base displacement μθ∈R3\mu_\theta\in\mathbb R^3 and basis values ϕθ∈R12\phi_\theta\in\mathbb R^{12}.

During training, coefficients fitted from supports predict a separate query interaction. The query loss trains the shared network through the differentiable solve. At inference, network weights are fixed and four observed grasps determine the coefficients.

Changes in the revised manuscript: Sec. 3 opening, p. 3; network and training in Sec. 3.3, p. 5; Fig. 2(a), p. 4.

AuthorsOpenReview ↗

Response to Reviewer 5HHR (2/3)

[Q4] What is an anchor?

Paper Summary: It then talk about each "anchor", without defining what an anchor is.

An anchor aia_i is one of the 32 observed surface points selected from the 96-point representation. Support displacement is measured and a coefficient matrix AiA_i is fitted there. Geometry-based interpolation provides coefficients at the other prediction points.

Changes in the revised manuscript: Sec. 3.1, p. 3; interpolation in Sec. 3.2, p. 5; Fig. 2(b), p. 4.

[Q5] What are the fitted coefficients?

Paper Summary: The method description then continues to discuss fitting "coefficients" but never says what these coefficients are.

Each anchor has a coefficient matrix Ai∈R12×3A_i\in\mathbb R^{12\times3}, with twelve weights per displacement coordinate. All four supports jointly determine one collection of 32 matrices, the GPR representation:

zo=(A1,…,A32).z_o=(A_1,\ldots,A_{32}).

For a new action, the network produces a new base prediction and basis values. The same coefficients combine those basis values into a correction, allowing reuse across compatible actions.

Changes in the revised manuscript: Sec. 3.1 / revised Eq. (1), p. 4; “Reusing the coefficients” in Sec. 3.2, p. 5; Fig. 2(b–c), p. 4.

[Q6] What are the complete inputs and outputs?

Paper Summary: The inputs to the method being proposed are never clearly defined and components are mentioned without definition or any direct connection to the previously discussed sections.

The revised setup specifies the complete interface:

QuantityInformation supplied or predicted
Initial geometryCoordinates of 96 surface points
Four support graspsGripper commands at four times; displacements at 32 anchors at those times
New actionCommanded gripper trajectory over 28 frames
OutputPredicted displacements of all 96 surface points over 28 frames

Support commands are interpolated between the four observed times. For a query, the model receives the commanded trajectory and predicts its unobserved response. Q2 defines the quantities, and Q8 explains their use.

Changes in the revised manuscript: Secs. 3.1–3.2, pp. 3–5; Appendix A.2, pp. 16–17.

[Q7] How do the original Equations (1)–(4) connect?

Paper Summary — Eq. (2): Equation (2) factorizes the function from equation (1) to have different components, but again no details of these are given.

Paper Summary — Eqs. (3)–(4): Equations (3) and (4) then go on to define a minimization problem that are not clearly connected to the above parts of the approach.

Using the original equation numbers, Eq. (1) defines Go:u↦do(⋅,⋅;u)\mathcal G_o:u\mapsto d_o(\cdot,\cdot;u). Eq. (2) predicts this response as

d^o(x,t;u)=μθ(x,t;go,u)+ϕθ(x,t;go,u)⊤ao(x).\widehat d_o(x,t;u)=\mu_\theta(x,t;g_o,u)+\phi_\theta(x,t;g_o,u)^\top a_o(x).

The network supplies the base displacement μθ\mu_\theta and basis values ϕθ\phi_\theta; ao(x)a_o(x) interpolates anchor coefficients AiA_i. Subtracting the base prediction from observed displacement gives the residual to fit.

At each anchor jj, stacking basis values and residuals from four grasps at four times gives Xj∈R16×12X_j\in\mathbb R^{16\times12} and Ej∈R16×3E_j\in\mathbb R^{16\times3}, normalized by object size. Stacking weighted rows from related anchors gives Qi,YiQ_i,Y_i. Eq. (3) fits the coefficients by regularized least squares:

Ai=arg⁡min⁡A∣QiA−Yi∣F2+λi∣A−Ci∣F2.A_i=\arg\min_A\\|Q_iA-Y_i\\|_F^2+\lambda_i\\|A-C_i\\|_F^2.

CiC_i is a reference computed from supports, and λi>0\lambda_i>0 controls regularization. With both fixed, setting the derivative to zero gives Qi⊤(QiAi−Yi)+λi(Ai−Ci)=0Q_i^\top(Q_iA_i-Y_i)+\lambda_i(A_i-C_i)=0. Defining Gi=Qi⊤QiG_i=Q_i^\top Q_i and Hi=Qi⊤YiH_i=Q_i^\top Y_i yields Eq. (4):

(Gi+λiI)Ai=Hi+λiCi.(G_i+\lambda_i I)A_i=H_i+\lambda_i C_i.

This solves Eq. (3), and the coefficients enter Eq. (2) for new-action prediction.

Changes in the revised manuscript: Secs. 3.1–3.2, pp. 3–5; derivation in Appendix A.3, pp. 17–18. Original Eq. (1) is retained in Appendix A.2; revised Eq. (1) defines the finite-dimensional GPR state, and revised Eq. (3) expands the weighted fit by anchor.

AuthorsOpenReview ↗

Response to Reviewer 5HHR (3/3)

[Q8] How are the components connected?

Paper Summary — components: The inputs to the method being proposed are never clearly defined and components are mentioned without definition or any direct connection to the previously discussed sections.

Paper Summary — factorization: Equation (2) factorizes the function from equation (1) to have different components, but again no details of these are given.

The components follow the prediction equation in Q7:

StageOperation and output
Shared networkMaps the initial geometry and an action trajectory to a base displacement prediction and response basis
IdentificationSubtracts the base prediction from observed support displacements, then fits coefficients that combine the fixed basis values to explain these residuals
Prediction for a new actionCombines the new base prediction and basis values with the previously identified coefficients to predict displacement

Within identification, geometry-based evidence sharing combines equations from related anchors, and response-dependent regularization stabilizes the local fits. Fig. 2 shows both operations, residual subtraction and the final prediction sum.

Changes in the revised manuscript: Secs. 3.1–3.3, pp. 3–5; Fig. 2(a–c), p. 4.

[Q9] How has the presentation been revised?

Paper Summary — overall assessment: This paper is incoherent.

Paper Summary — method clarity: The method is impossible to understand.

Paper Summary — assessing experiments: It's meaningless to examine the experiments and results in any detail given there's no clear statement of what was actually proposed in the paper.

Critical Weakness 1: The paper is incoherent. The approach makes no sense and as such the experiments can't even be critiqued.

Questions for Authors: I have no direct questions for the author. The paper needs to be entirely rewritten to provide any level of clarity as to what was actually done.

Thank you for the suggestions on presentation. Important definitions were previously left in the appendix, leaving the inputs and component connections unclear. Methods now defines terms before use and follows problem setup, identification from four grasps, and shared-network training, connected by the prediction equation (Q2–Q8).

The Abstract and Introduction explain the predictive goal and the challenges of physical-property estimation, sparse observations and repeated prediction before introducing GPR and DROP-Op. Figs. 1–4 clarify the task, method, generalization factors and predictions. We hope these revisions make the approach and its evidence easier to assess.

Changes in the revised manuscript: Abstract/Introduction, pp. 1–2; Methods, pp. 3–5; Figs. 1–4, pp. 2, 4, 6 and 8.

Reviewer_otLYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper first introduces a controlled grasping benchmark for evaluating dynamics prediction under individual and combined object and interaction shifts. It then proposes to directly define the mapping from actions to spatiotemporal displacement fields as GPR and identify it, rather than explicitly identifying physical parameters.

Based on this definition, the paper proposes DROP-Op, which decomposes the displacement field into a shared displacement prediction and a linear combination of a learned rank-12 response basis. The coefficients are fitted from the four observed grasps at 32 anchors via a closed-form ridge solve that combines geometry-based evidence sharing with response-dependent regularization, and the identified GPR state is reused to predict responses to new actions under compatible conditions.

In the experiments, each support grasp is observed at four frames and 32 sites, and the confirmation set is evaluated after model freezing. DROP-Op achieves the lowest mean endpoint error (EPE) among all evaluated methods. The paper further reports representation comparisons, mechanism controls (matched attention and pooled identification), compute-matched baselines, downstream task readouts, and external-task studies.

Critical strengths
  1. A structured and reusable identification mechanism. Instead of conditioning a network implicitly on context, DROP-Op turns the observed interactions into an explicit regularized least-squares problem at each spatial anchor, solved in closed form and differentiated end-to-end.
  2. Broad experimental study. Beyond the main results, where DROP-Op achieves the lowest mean EPE among all evaluated methods, the paper includes representation comparisons, mechanism controls, compute-matched baselines, sensitivity analyses, downstream tasks and external-task studies, with negative transfer reported openly. These show the value and fun of this work.
Critical weaknesses

Lack of clarity in the benchmark’s physical settings:

  1. The reported Young’s moduli are 32–54 Pa (soft) and 64–248.4 Pa (stiff), much softer than most realworld objects, and pad friction is about 0.01, which is also confused in real world.
  2. Only a normalized geometry scale is given. Absolute object size, whether gravity is enabled, and density are not clearly reported.
  3. How the four support interactions are selected is not described, although the choice of probes affects error.
Questions for authors

Questions:

  1. Basic physical settings of the benchmark. Please report the absolute object dimensions (in metres), density, whether gravity is enabled in simulation, whether any unit scaling is applied Currently only a normalized geometry scale is given, which makes it difficult to judge what physical regime the benchmark represents.
  2. Rationale for the physical parameters. The moduli in the benchmark are generally very small and the pad friction coefficient is approximately 0.01, both far below typical real-world objects and gripper contact. Please explain why these values were chosen.
  3. Selection of support interactions. How are the four support interactions selected: random sampling from the action distribution, a fixed design, or a diversity constraint? Requested experiments:
  4. Performance under realistic physical parameters. If the moduli and friction coefficient are set to ranges similar to real soft objects and gripper contact, does DROP-Op’s advantage over the baselines still hold?
  5. Effect and cost of the support selection strategy. Please report: performance when supports are selected fully at random; the cost of the current selection procedure, if it involves additional design or filtering; whether the same selection strategy remains effective across different objects and grippers.
  6. Physical-parameter identification baseline. Since the introduction contrasts GPR with explicit physical-parameter estimation, would it be possible to include a simple comparison against such an approach?
4 comments
AuthorsOpenReview ↗

Response to Reviewer otLY (1/3)

We sincerely appreciate Reviewer otLY for the positive comments on our contributions, especially the structured and reusable identification mechanism and the broad experimental study. The suggestions in both the weakness and question sections are very helpful and have guided our additional experiments on the benchmark’s physical settings, support selection, and physical-parameter identification.

[Q1] Choice of Young’s moduli, friction and other physical settings.

Critical Weakness 1: The reported Young’s moduli are 32–54 Pa (soft) and 64–248.4 Pa (stiff), much softer than most realworld objects, and pad friction is about 0.01, which is also confused in real world.

Question 2: Rationale for the physical parameters. The moduli in the benchmark are generally very small and the pad friction coefficient is approximately 0.01, both far below typical real-world objects and gripper contact. Please explain why these values were chosen.

Requested Experiment 1: Performance under realistic physical parameters. If the moduli and friction coefficient are set to ranges similar to real soft objects and gripper contact, does DROP-Op’s advantage over the baselines still hold?

Young's moduli: Thank you for emphasizing realistic physical settings. We agree that the original Young's moduli are lower than those of many real soft objects. We chose a highly compliant regime to make spatial deformation more pronounced and differences in predictive performance easier to distinguish under changes in geometry, material layout and interaction conditions.

Friction: Our original description omitted an important distinction: approximately 0.01 is the pad-surface parameter, while the object surface uses 1.1. At equal contact priority, MuJoCo takes their componentwise maximum, giving effective sliding friction 1.1 in the base configuration (MuJoCo documentation). We have clarified this distinction.

Other settings: Gravity is disabled to isolate contact-induced motion and deformation, and total mass is fixed across shapes. Poisson ratio and damping describe the assumed material response. Q2 gives the absolute values and units.

To alleviate concerns, we also tested soft moduli of 20–40 kPa, stiff moduli from twice the soft modulus to 80 kPa, density 1,000 kg/m³, and effective friction 0.3, 0.7 and 1.1. (EPE in mm, mean ± standard deviation):

Effective sliding frictionDROP-OpALPaCA-MSEResponse attention
0.30.4220 ± 0.01030.4895 ± 0.00240.6644 ± 0.0077
0.70.3286 ± 0.00270.3704 ± 0.00330.5220 ± 0.0028
1.10.3514 ± 0.00180.3925 ± 0.00280.5392 ± 0.0022

DROP-Op retains the lowest mean error at all three friction levels, improving over ALPaCA-MSE by 10.5–13.8%. Its advantage therefore extends beyond the original highly compliant regime.

Changes in the revised manuscript: Physical settings and friction: Appendix B.5, pp. 28–29. Higher-modulus tests: Sec. 4.2, p. 7; Appendix H.4 / Table 30, p. 50.

[Q2] Absolute dimensions, density, gravity and unit scaling.

Critical Weakness 2: Only a normalized geometry scale is given. Absolute object size, whether gravity is enabled, and density are not clearly reported.

Question 1: Basic physical settings of the benchmark. Please report the absolute object dimensions (in metres), density, whether gravity is enabled in simulation, whether any unit scaling is applied Currently only a normalized geometry scale is given, which makes it difficult to judge what physical regime the benchmark represents.

We have added the absolute quantities in our paper:

ParameterOriginal setting
Rest-mesh extent along x0.122926–0.180686 m
Rest-mesh extent along y0.082285–0.121011 m
Rest-mesh extent along z0.072550–0.135733 m
Total object mass0.110 kg, distributed equally among nodes
Bulk-average density83.168–126.293 kg/m³
Gravity(0, 0, 0) m/s²
Soft-region Young’s modulus32–54 Pa
Stiff-region Young’s modulus64–248.4 Pa
Poisson ratio0.25
Pad sliding-friction parameterApproximately 0.01
Object-surface / effective sliding friction1.1 / 1.1 in the base configuration
Simulation timestep0.000125 s
SolverConjugate gradient (CG)
Length / mass / time unitsm / kg / s

Bulk-average density is mass divided by mesh volume. Geometry is centered and normalized by its root-mean-square radius within the predictor.

Changes in the revised manuscript: Sec. 4.1, p. 6; Appendix B.5, pp. 28–29; Fit settings in Appendix B.2.3, p. 25; normalization in Appendix A.4, p. 18.

AuthorsOpenReview ↗

Response to Reviewer otLY (2/3)

[Q3] Performance under more realistic physical parameters.

Critical Weakness 1: The reported Young’s moduli are 32–54 Pa (soft) and 64–248.4 Pa (stiff), much softer than most realworld objects, and pad friction is about 0.01, which is also confused in real world.

Question 2: Rationale for the physical parameters. The moduli in the benchmark are generally very small and the pad friction coefficient is approximately 0.01, both far below typical real-world objects and gripper contact. Please explain why these values were chosen.

Requested Experiment 1: Performance under realistic physical parameters. If the moduli and friction coefficient are set to ranges similar to real soft objects and gripper contact, does DROP-Op’s advantage over the baselines still hold?

The table in Q1 shows that with soft moduli of 20–40 kPa, stiff moduli from twice the soft modulus to 80 kPa, density 1,000 kg/m³, and friction 0.3, 0.7 and 1.1, DROP-Op retains the lowest mean EPE. The reduction relative to ALPaCA-MSE is 10.5–13.8%.

Changes in the revised manuscript: Sec. 4.2, p. 7; Appendix H.4 / Table 30, p. 50, including the settings and numerical checks.

[Q4] Selection of the four supports and performance with random actions.

Critical Weakness 3: How the four support interactions are selected is not described, although the choice of probes affects error.

Question 3: Selection of support interactions. How are the four support interactions selected: random sampling from the action distribution, a fixed design, or a diversity constraint?

Requested Experiment 2: Effect and cost of the support selection strategy. Please report: performance when supports are selected fully at random; the cost of the current selection procedure, if it involves additional design or filtering; whether the same selection strategy remains effective across different objects and grippers.

The supports are four predefined grasps with normalized closure levels 0.35/0.50/0.65/0.50 and variations in timing, eccentricity and angle. The same action templates are used across objects and grippers, with closure scaled to the range calibrated for each geometry (Table 11). So it's mostly a fixed design.

We followed your advice and sampled grasps randomly from the allowed action ranges. All methods received the same random supports and were evaluated on the same valid development cases. Mean EPE (mm):

MethodSupport selectionEPE
DROP-OpOriginal fixed slots0.7485
DROP-OpRandom0.8650
ALPaCA-MSESame random supports1.4557
Response attentionSame random supports1.2461

Random supports increase DROP-Op's error relative to the predefined grasps, but its advantage over both baselines remains. Q5 addresses selection cost and generalization.

Changes in the revised manuscript: Templates and calibration: Appendix B.5 / Table 11, pp. 28–29. Random-support comparison and validity checks: Appendix H.5, pp. 50–51.

AuthorsOpenReview ↗

Response to Reviewer otLY (3/3)

[Q5] Selection effect, acquisition cost and transfer across objects and grippers.

Critical Weakness 3: How the four support interactions are selected is not described, although the choice of probes affects error.

Question 3: Selection of support interactions. How are the four support interactions selected: random sampling from the action distribution, a fixed design, or a diversity constraint?

Requested Experiment 2: Effect and cost of the support selection strategy. Please report: performance when supports are selected fully at random; the cost of the current selection procedure, if it involves additional design or filtering; whether the same selection strategy remains effective across different objects and grippers.

Original design. The four action templates are fixed in advance and require no online search or per-object selection optimization. The original results across objects and grippers use these same templates (Table 1; Appendix C.1), supporting this fixed protocol across the tested conditions. Four accepted supports average 97.79 s of accumulated simulation-process time.

Further study. Following your suggestion, we asked whether choosing the four support actions more carefully improves prediction with the same interaction budget. For each object–gripper pair, we generate candidate grasps with different closure depths, directions, contact offsets and timings, and remove commands outside the gripper's mechanical limits. The shared network provides a response basis for each candidate. These basis values form the equations used to identify the GPR coefficients, so we can score how much an additional action would constrain coefficient directions that the selected actions leave uncertain. The score uses a prediction-variance proxy derived from the regularized coefficient fit; it requires no measured response from the candidate being scored.

We compare the four strategies below. Mean EPE (mm):

StrategyHow actions are selectedAll 51 combinationsNew finger counts: 12 combinations
RandomSample four feasible candidates uniformly.0.69091.1140
Parameter diversitySelect four candidates spread out in normalized action-parameter space.0.71240.9835
Information-batchGreedily select four complementary actions using the basis-derived score, before executing any grasp.0.62841.0648
Response-sequentialUse each observed grasp to update response-dependent regularization, then score and select the next action.0.65891.1486

Changes in the revised manuscript: Original design and cost: Appendix B.5, pp. 28–29. Additional selection study, exclusions and failures: Appendix H.5 / Table 31, pp. 50–51.

[Q6] Comparison with explicit physical-parameter identification.

Requested Experiment 3: Physical-parameter identification baseline. Since the introduction contrasts GPR with explicit physical-parameter estimation, would it be possible to include a simple comparison against such an approach?

We compare GPR with spatial and pooled elastic fields, and with an added control that estimates two Young's moduli and a material-boundary offset. Field controls receive true properties; inferred scalars receive extra property supervision during training. Mean EPE (mm):

Representation / controlEPE (mm)
True spatial elastic fields (Table 2B)2.2609
True pooled elastic fields (Table 2B)2.2717
Inferred two moduli + boundary offset2.0394
True two moduli + boundary offset2.1513
DROP-Op0.9068

DROP-Op has lower error in these comparisons. We also adapted AdaptiGraph for explicit physical-parameter identification. It retains the original dynamics model and uses CMA-ES to fit the two moduli and boundary offset from supports, with extra property supervision during training. Similarly, DROP-Op achieves lower EPE and dEPE.

MethodEPE (mm)dEPE (mm)
AdaptiGraph, explicit physical identification1.34241.2381
DROP-Op0.77680.7262

Changes in the revised manuscript: Sec. 4.3, pp. 7–8; Table 2, p. 9; added scalar control in Appendix D.3, pp. 36–37; explicit AdaptiGraph comparison in Appendix C.4 / Table 19, p. 34. The comparison is reported in Appendix C.4 / Table 19; inclusion in Table 1 awaits completion of the IoU evaluation.

I do not intend to change my rating based on the rebuttal. While some of the missing details were provided, looking at the manuscript I still find it difficult to follow with extremely choppy text. Additionally the overall framing is still presented as much broader than what's actually being studied in the paper.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response [1/2]

We sincerely thank the reviewers for their comments on the physical settings and identification comparisons (otLY), task and method formulation (5HHR), and motivation, benchmark design and real-world relevance (YS9Q). These suggestions have helped us improve both the explanation and the experimental evidence.

→ Strengths: We appreciate the reviewers' recognition of the structured and reusable identification mechanism (otLY), the predictive improvements (otLY, YS9Q), and the breadth of the experimental evaluation (otLY, YS9Q).

[Structured and reusable identification — otLY] Reviewer otLY highlights that DROP-Op formulates identification as an explicit regularized least-squares problem at each spatial anchor, solved in closed form and differentiated end-to-end. The identified coefficients provide a reusable response state for predicting new actions under compatible conditions.

[Predictive performance — otLY, YS9Q] Reviewer otLY notes that DROP-Op achieves the lowest mean EPE among the evaluated methods. Reviewer YS9Q also notes the 31.6% reduction in EPE and 30.9% reduction in dEPE relative to ALPaCA-MSE. These results support the effectiveness of response identification for dynamics prediction from sparse interactions (Table 1).

[Comprehensive experimental evaluation — otLY, YS9Q] Both reviewers recognize the extensive experimental coverage. Beyond the main prediction results, the study includes representation comparisons, mechanism controls, compute-matched baselines, sensitivity analyses, downstream readouts and external-task studies. Reviewer otLY also appreciates the reporting of negative transfer, while Reviewer YS9Q commends the interactive visualizations.

→ Weakness: The reviews identified unclear definitions and component connections, insufficient explanation of the physical settings and benchmark contribution, and a need for additional real-data and recent-method comparisons. We have rewritten the formulation, redrawn the figures and extended the experiments. The individual replies quote the relevant comments and provide the corresponding revision locations.

[Goal, contributions and their connection — 5HHR, YS9Q] Our goal is to understand an unfamiliar object's physical behavior from a few interactions and predict its motion and deformation under new actions. This requires reducing the modeling effort of selecting physical properties, using sparse response evidence effectively, and avoiding repeated processing of the same observations.

GPR describes action-dependent responses directly, capturing combined physical effects through an identified response state. DROP-Op supplies the identification mechanism: a shared network learns a base displacement prediction and response basis; local coefficient fitting, geometry-based evidence sharing and response-dependent regularization use the sparse observations. The coefficients are then reused for compatible new actions. The benchmark tests this capability through individual and combined object, action and gripper shifts under a fixed observation budget. Thus, GPR defines what to identify, DROP-Op explains how to identify and reuse it, and the benchmark tests its generalization.

[Physical settings and higher-modulus evaluation — otLY] We have added absolute dimensions, mass/density, gravity and units, and clarified that the reported pad friction of about 0.01 is a surface parameter; the base configuration's effective sliding friction is 1.1. We also evaluated soft moduli of 20–40 kPa, stiff moduli from twice the soft modulus to 80 kPa, density 1,000 kg/m³ and friction 0.3–1.1. Three-seed mean EPE (mm):

Effective sliding frictionDROP-OpALPaCA-MSEResponse attention
0.30.42200.48950.6644
0.70.32860.37040.5220
1.10.35140.39250.5392

DROP-Op retains the lowest mean error at all three levels, improving over ALPaCA-MSE by 10.5–13.8%. This extends the evidence beyond the original highly compliant regime. Gravity remains disabled, and numerical convergence checks determine the included shape families. Settings, checks and seed variability are reported in Sec. 4.2 and Appendices B.5 and H.4 / Table 30.

AuthorsOpenReview ↗

General Response [2/2]

[Real measurements and sim-to-real adaptation — YS9Q] We added real textile-interaction experiments on GAUGE, covering six materials, three material-transfer folds and three seeds. Each prediction uses four sparsely observed interactions. Mean EPE (mm):

Window / trainingDROP-OpALPaCA-MSEResponse attention
Loading / real only2.25652.89984.5822
Loading / simulation + real2.14822.42493.1884
Full video / real only4.378910.767232.6302
Full video / simulation + real3.750110.88947.7225

Both conditions use 1,024 real-data updates; simulation pretraining is additional. DROP-Op's advantage persists on real measurements, and simulation pretraining improves its adapted predictions. These results support real-response prediction and adaptation. Simulation-only transfer remains inaccurate, showing that the learned basis must cover the target dynamics. The data use tracked markers and previously exposed development splits. We have not yet evaluated closed-loop robot hardware; encouraged by these results, we plan to connect visual observation, sparse probing and action selection in future robot experiments (YS9Q Q5).

[Support selection and additional comparisons — otLY, YS9Q] The fixed-support design, random-support comparison, selection cost and transfer analysis are detailed in otLY Q4–Q5. With random supports, DROP-Op remains more accurate than the compared predictors. A frozen information-based selector further improves aggregate EPE, with failures and weaker transfer to new finger counts reported in Appendix H.5. Physical-representation and explicit-parameter comparisons are discussed in otLY Q6.

The completed NFF grasp adaptation uses the original data, sparse observations and 138-pass training budget. Three-seed means on the same 46 panels:

MethodEPE (mm)dEPE (mm)
NFF2.92792.4957
DROP-Op0.77560.7222

We have also completed LPWM and MAVEN under the same 138-pass training budget and sparse-observation protocol. Single-seed results on the same 46 panels, using the same seed for all methods:

MethodEPE (mm)dEPE (mm)
LPWM3.36602.7746
MAVEN2.58722.4475
DROP-Op0.77680.7262

DROP-Op has lower EPE and dEPE in every panel than both adaptations. Further details are provided in YS9Q Q6. Table 1 insertion and IoU evaluation for these recent-method comparisons remain pending. DISCO is compared on its original task in Appendix F. Existing external-task comparisons and method interfaces are documented in Appendices F and C.5.

[Paper Revision Updates] The revised manuscript includes the following changes:

  • Motivation and contributions (5HHR, YS9Q): revised the Abstract and Introduction to connect the predictive goal, three challenges, GPR, DROP-Op and controlled benchmark evaluation (pp. 1–2).
  • Definitions and method flow (5HHR, YS9Q): defined geometry, actions, observations, anchors and coefficients; reorganized Methods as setup → identification → shared-network training (Secs. 3.1–3.3, pp. 3–5; Appendix A.3, pp. 17–18).
  • Figures (5HHR, YS9Q): redrew Figs. 1–6 and revised their captions to explain the task, module connections, ID/OOD tests, predictions, cost and backbone transfer (pp. 2, 4, 6, 8, 8 and 10).
  • Physical settings and supports (otLY): added absolute settings, friction clarification, higher-modulus tests, random-support results and selection cost/transfer analysis (Secs. 4.1–4.2, pp. 6–7; Appendix B.5 / Table 11, pp. 28–29; Appendix H.4–H.5 / Tables 30–31, pp. 50–51).
  • Physical identification (otLY): retained the representation comparisons and added a three-parameter identification control (Sec. 4.3, pp. 7–8; Table 2, p. 9; Appendix D.3, pp. 36–37; Appendix C.4 / Table 19, p. 34).
  • Real data and next steps (YS9Q): added GAUGE results and sim-to-real analysis, with markerless observation and closed-loop robotics as future directions (Secs. 4.5 and 5, p. 10; Appendix F.5, pp. 44–47; Table 23, p. 45).
  • Evaluation context (YS9Q): compared related evaluation settings and explained recent-method interfaces (Appendix B.6 / Table 12, p. 30; Appendix C.5, p. 35).

We sincerely appreciate the reviewers' suggestions and welcome further discussion of the revised formulation and additional evidence.

Dear Reviewers,

Thank you again for your reviews of this submission. The authors have now posted their responses, and the author–reviewer discussion period ends on November 18, 2026 (AoE).

Please read the authors' rebuttal and let them know whether it addresses your concerns. A short reply is enough, for example:

  • which of your concerns are now resolved;
  • which remain open, and what evidence or clarification would resolve them;
  • whether you plan to update your rating.

If the rebuttal changes your assessment, please update your review and rating before the discussion period ends. If you have already engaged, thank you, and no further action is needed unless the authors reply again.

Your participation is important for a fair and well-informed decision.

Dear Reviewers,

I hope this message finds you well. As the discussion period is nearing its end with less than three days remaining. I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.