acceptodds
Under review as a conference paper at ICLR 2027

Guiding without Prescribing: A Differentiable Trajectory Learning Framework for Robot-Assisted Dressing

Abstract

Robot-assisted dressing requires generating effective manipulation trajectories under complex cloth dynamics and sustained garment-body interactions. Learning a single trajectory solver across diverse arm configurations is further challenging because effective trajectories can vary substantially with pose. Differentiable physics provides first-order gradients for trajectory optimization, but obtaining these gradients requires complete garment states unavailable to the solver at inference. We propose Guiding without Prescribing (GwoP), a differentiable trajectory learning framework that integrates physical and body-relative guidance during training. GwoP separates observation-conditioned trajectory prediction from full-state physical evaluation within a differentiable computation graph, allowing physical gradients to train the solver without requiring complete garment states at inference. To support shared learning across arm poses, GwoP further predicts trajectories and auxiliary body-surface relations in parallel, providing structural guidance without prescribing trajectory geometry. Across unseen arm configurations, GwoP achieves high dressing progress with limited garment deformation. We further validate the learned trajectories through cross-simulator transfer and real-robot execution across diverse arm poses.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot-Assisted Dressing, Cloth Manipulation, Differentiable Physics, Trajectory Optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JSVCOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper propose a method to train dressing policy, which utilize a differentiable cloth simulation (DiffCloth) to get denser learning signals for model update. The simulator and real world experiments show that the policy success rate is clearly higher than baselines, while an ablation version without the learning gradient signals of differentiable simulation demonstrates the necessity of differentiable feedback from simulator clearly, solidifying the core argument of the authors.

Critical strengths
  1. clear evidence that demonstrates the significance and necessity of training a model for complex tasks like dressing with differentiable simulator feedback.
  2. strong performance across real-world and simulator environment.
  3. good and clear visualization of method and experiment process.
Critical weaknesses
  1. real-world execution is not robust and applicable, with too many annotations and demo-orientated specific designs. There are still far away from a practical dressing system.
Questions for authors
  1. Baseline methods are not documented enough. For example, how TD-MPC2 are employed in the task setting?
  2. The component of dataset is not clear, because some samples in dataset are interpolated, not independent novel poses.
  3. Can authors convert the real-world execution from open-looped one into close-loop system, and also get rid of the heavy annotation and designs? Will it still remain successful in real-world dressing task?
2 comments

We thank the reviewer for the encouraging assessment and for recognizing the benefits of differentiable physical feedback, simulation performance, and real-robot evaluation. We have expanded the implementation details and added experiments to clarify dataset composition and robustness.

[Q1] TD-MPC2 Implementation and Task Adaptation

Thank you for requesting further details. We have substantially expanded the TD-MPC2 implementation description in Appendix C.3.

We adapt TD-MPC2 to the DiffCloth dressing environment while retaining its implicit world model and model predictive control framework.

The main task-specific settings are as follows:

  • Observation and action: A 1,024 × 6 point cloud containing 3D coordinates and semantic labels for the arm, garment, and grasp geometry. The policy predicts a 6D action controlling two grasp points independently, matching GwoP's manipulation interface.
  • Action execution: Each action is applied over 20 DiffCloth simulation steps, with an episode limited to 800 simulation steps.
  • Reward: The task reward combines cuff advancement along the arm, garment-stretch penalties, gripper–arm contact penalties, distance-to-arm deviation terms, and simulation-failure penalties. The deformation term penalizes edge-length and triangle-area growth beyond a 5% tolerance.
  • Model and planning: A PointNet++ encoder produces a 512D latent state. The world model retains latent dynamics, reward and termination predictors, a learned policy prior, and five Q-functions. MPPI uses a horizon of three actions, six optimization iterations, 512 candidate trajectories per iteration, and 64 elites.
  • Training: The evaluated checkpoint is obtained after approximately 118,000 environment steps, with 174.76 hours of training.

With the aligned two-gripper interface, TD-MPC2 achieves 56.2% elbow-passage success and 47.9% dressing progress. The updated comparison is reported in Sec. 5.2 (Table 1, Fig. 2), and the observation, reward, planning, and training details are documented in Appendix C.3.

[Q2] Dataset Composition and Interpolation/Extrapolation

Thank you for pointing out the need to distinguish interpolated test configurations from extrapolated ones. We have clarified the dataset construction and provided a visualization of the complete pose space in Appendix B.3 (Fig. 6).

Arm poses are parameterized by shoulder angle ϕs\phi_s and elbow angle ϕe\phi_e, with:

\phi_e \in {0^\circ,2^\circ,\ldots,80^\circ}.$$ This produces 1,066 distinct arm configurations. We train on 25 poses selected from a sparse 5 × 5 grid and reserve the remaining 1,041 configurations for evaluation. The held-out set is divided into: | Subset | Elbow Pass ↑ | Progress ↑ | Edge Viol. ↓ | Area Viol. ↓ | | ---------------- | ------------ | ---------- | ------------ | ---------- | | Interpolation | 100.0% | 75.8% | 0.0107 | 0.0233 | | Extrapolation | 87.5% | 71.2% | 0.0139 | 0.0276 | | **Total unseen** | **97.1%** | **74.7%** | **0.0115** | **0.0243** | These are the results of the model used in the main comparison (Seed 2). The interpolation subset consists of unseen configurations within the angular bounds of the training grid, whereas the extrapolation subset lies outside those bounds. Thus, **all 1,041 test configurations are held out, but they represent two different generalization regimes**. We additionally evaluated three independent training seeds. The average extrapolation success rate is 89.9 ± 7.5%, compared with 100.0 ± 0.0% for interpolation. The split definitions have been added to Sec. 5.1 and Appendix B.3, with the complete multi-seed results in Appendix D.4 (Table 5, Fig. 9).

[W1/Q3] Closed-Loop Execution and Automated Perception

We thank the reviewer for highlighting two important directions toward practical deployment: closed-loop control and automated perception.

For closed-loop execution, GwoP predicts trajectories directly from task observations, with a network forward pass and single-trajectory decoding taking only 2.60 ms on average. This provides a basis for incorporating repeated observations and trajectory updates during execution. A possible extension is to update garment and body observations online, regenerate the trajectory, and execute the updated motion. As the current solver is trained to predict complete trajectories from initial states, reliable replanning from intermediate dressing states requires further development. We therefore share the reviewer's view that closed-loop feedback is an important next step, which we also identified as a future direction in Sec. 6 of our original submission.

For automated perception, the current manual keypoint annotations and calibration procedure provide controlled, repeatable inputs for evaluating trajectory generation. These could potentially be replaced by automatic human-pose estimation, body-model fitting, and coordinate registration.

To investigate sensitivity to uncertainties introduced by perception and initialization, we have conducted two additional perturbation experiments (Appendix D.5, Tables 6–7, Figs. 10–11). The results are summarized below:

PerturbationMagnitudeElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
Initial Garment/Gripper Position≤ 4 cm100.0%65.1%0.01100.0232
≤ 8 cm93.2%57.6%0.01490.0303
Shoulder-Angle Estimation8°97.7%74.4%0.01220.0254
24°96.3%66.2%0.01430.0277
Elbow-Angle Estimation8°98.8%73.8%0.01210.0251
24°90.8%64.5%0.01470.0290

Position perturbations are grouped by cumulative radius; angular perturbations are grouped by exact absolute error, averaged over the evaluated perturbations of the other joint.

The initialization experiment evaluates sensitivity to variations in initial garment placement and grasp-point positions, which are relevant to uncertainties in garment localization and initialization. The angular perturbations examine inaccuracies in human arm-pose estimation. The results show that GwoP maintains high elbow-passage success under moderate perturbations, while larger errors generally lead to reduced dressing performance. These findings provide preliminary evidence of tolerance to perception and initialization uncertainties.

We appreciate the reviewer's constructive suggestions, which help clarify the next steps toward a more autonomous and practical robot-assisted dressing system.

Reviewer_TzfpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Guiding without Prescribing (GwoP), a framework for learning robot-assisted dressing trajectories across different arm configurations. The solver receives partial garment observations, body and arm-configuration information, and initial gripper positions, and predicts two gripper trajectories parameterized by piecewise cubic Bézier curves.

The framework combines two training signals. First, predicted trajectories are executed in a differentiable cloth simulator, allowing objectives for garment alignment, dressing progress, deformation, and body clearance to update the trajectory predictor through physical gradients. Complete garment states are used for training-time evaluation but are not required as inputs to the deployed solver. Second, an auxiliary branch predicts distributions over body-surface regions, supervised using reference paths derived from body-surface geodesics. The trajectory and relation outputs share an internal representation, while the trajectory is not required to pass through the predicted relation representation.

Critical strengths
  1. The model predicts body-surface relations and gripper trajectories in parallel, so the geometric supervision can shape the shared representation without forcing the grippers to follow a predefined path. This makes sense for dressing, where different arm poses can require quite different motions despite sharing the same task structure. The comparisons with direct trajectory supervision and Query-to-Trajectory are relevant and support this design choice.

  2. Adding body-relative guidance improves dressing progress from 61.2% to 74.7% and elbow passage from 87.3% to 97.1%, while reducing both deformation measures. These are meaningful gains within the proposed framework. The transfer examples in CLO and the 46 successful elbow passages in 60 robot trials provide additional evidence that the predicted trajectories remain useful outside the training simulator. I find this combination of controlled ablations and physical execution a strong part of the paper.

Critical weaknesses
  1. My main concern is the control interface described in the appendix. GwoP independently controls two garment attachment points, while OnePolicy and TD-MPC2 each predict a single 3D displacement applied to a fixed set of 30 vertices. This changes both the available gripper motion and the physical constraints on the cloth. As a result, the improvement in Table 1 could partly come from the manipulation setup. The authors should clarify whether this description is accurate. If it is, a comparison using the same attachment constraints and independently controllable grasps is needed.
  2. Removing body-relative guidance also removes its ten epochs of pretraining, so this ablation combines the effects of initialization and continued auxiliary supervision. The trajectory-supervision baseline matches the pretraining duration, which is useful, but it does not separate these two effects. I would like to see relation pretraining followed by physics-only training compared with the full method. The gradient analysis is also limited to nine poses and a two-dimensional parameter subspace. Together with the absence of results across independent training seeds, this leaves the claim of more stable shared optimization less convincing than the performance improvement itself.
Questions for authors
  1. How much of the improvement comes from relation pretraining versus the auxiliary loss during joint training? What happens if the model receives the same relation pretraining but then trains using only the physical objective? This comparison would help separate the benefit of initialization from continued body-relative guidance. Do the main improvements also hold across independent training seeds?

  2. How well does the method perform outside the training pose range and under small initialization errors? Could you report dressing progress and elbow-passage success separately for the interpolation and extrapolation subsets? Have you evaluated sensitivity to small changes in initial garment placement or errors in the estimated arm configuration? These results would help assess how reliably the solver handles variations within its intended deployment setting.

5 comments

We thank the reviewer for the thoughtful assessment and for recognizing the motivation and experimental evidence for body-relative guidance. We have conducted additional baseline, ablation, multi-seed, and robustness experiments to address the remaining concerns.

[W1] Baseline Comparisons with Independently Controllable Grippers

Thank you for highlighting the manipulation-interface difference. We have updated both OnePolicy and TD-MPC2 to use two independently controllable garment grasp points, matching the attachment constraints and control interface of GwoP. The original-style Single-Gripper OnePolicy implementation is retained in the appendix for completeness.

The revised main comparison is:

MethodElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
Heuristic98.0%59.3%0.01490.0348
OnePolicy92.9%70.1%0.01580.0300
TD-MPC256.2%47.9%0.01950.0373
GwoP97.1%74.7%0.01150.0243

Under the aligned manipulation interface, GwoP retains the highest dressing progress and lowest garment deformation violations. The original-style Single-Gripper OnePolicy achieves 92.7% elbow passage and 48.3% progress, compared with 92.9% and 70.1% for the two-gripper implementation.

The main comparison has been updated in Sec. 5.2 (Table 1, Figs. 2–3). The two-gripper adaptations are documented in Appendix C.2–C.3, with the supplementary single-gripper comparison in Appendix D.2 (Table 3, Fig. 7). Additional execution examples are included in the supplementary video.

[W2.1/Q1.1] Separating Relation Pretraining from Continued Auxiliary Supervision

We appreciate this suggestion. We have added w/o Relation Pretraining and w/o Auxiliary Loss to independently examine the effects of the initialization stage and continued auxiliary supervision.

All variants use the same 22-epoch physical optimization stage. When enabled, relation pretraining precedes this stage for 10 epochs.

ConfigurationRelation PretrainingAuxiliary LossElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
w/o Body-Relative GuidanceNoNo87.3%61.2%0.01430.0272
w/o Relation PretrainingNoYes88.2%72.6%0.01300.0253
w/o Auxiliary LossYesNo93.9%63.2%0.01180.0247
FullYesYes97.1%74.7%0.01150.0243

The results reveal complementary effects.

Relation pretraining improves elbow-passage reliability. With auxiliary supervision retained, adding pretraining increases elbow passage from 88.2% to 97.1%, while progress increases from 72.6% to 74.7%.

Continued auxiliary supervision contributes more strongly to dressing progress. With pretraining retained, adding the auxiliary loss increases progress from 63.2% to 74.7%, while elbow passage increases from 93.9% to 97.1%.

These trends are also supported by the additional local optimization analysis. Adding continued auxiliary supervision improves the fraction of positively aligned pose pairs and reduces projected-gradient magnitude, both with and without relation pretraining. With pretraining retained, these metrics change from 47.2% to 66.7% and from 7.97 to 2.86, respectively.

We have expanded Sec. 5.3 (Table 2) and Appendix D.3.2 (Table 4).

[W2.2/Q1.2/Q2.1] Independent Training Seeds and Pose Generalization

Thank you for requesting this important evaluation. We have trained GwoP using three independent random seeds (1, 2, and 3). Seed 2 corresponds to the model used in the main comparison.

We additionally partition the 1,041 unseen test configurations into 800 interpolation poses within the training grid's angular bounds and 241 extrapolation poses outside those bounds.

The results are:

Evaluation SetSeedElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
InterpolationSeed 1100.0%84.1%0.01090.0235
Seed 2 (Main)100.0%75.8%0.01070.0233
Seed 3100.0%79.4%0.01070.0231
Mean ± Std100.0 ± 0.0%79.8 ± 4.2%0.0108 ± 0.00010.0233 ± 0.0002
ExtrapolationSeed 184.0%73.6%0.01650.0318
Seed 2 (Main)87.5%71.2%0.01390.0276
Seed 398.3%84.2%0.01260.0260
Mean ± Std89.9 ± 7.5%76.3 ± 6.9%0.0143 ± 0.00200.0285 ± 0.0030
Full Test SetSeed 196.3%81.6%0.01220.0254
Seed 2 (Main)97.1%74.7%0.01150.0243
Seed 399.6%80.5%0.01110.0237
Mean ± Std97.7 ± 1.7%78.9 ± 3.7%0.0116 ± 0.00060.0245 ± 0.0009

Seed 2 corresponds to the model used in the main-paper comparisons. Mean and sample standard deviation are calculated across three independent training seeds.

GwoP maintains high overall performance across the three training seeds. All three achieve 100% elbow-passage success on interpolated poses. Extrapolated poses show greater variability, but retain an average elbow-passage success rate of 89.9%.

This difference reflects both extrapolation beyond the training range and the geometric difficulty of some extrapolated poses, including configurations with a more flexed elbow or reduced clearance between the upper arm and torso.

The dataset composition and interpolation/extrapolation definitions are clarified in Sec. 5.1 and Appendix B.3 (Fig. 6). The independent-seed results are presented in Appendix D.4 (Table 5, Fig. 9), including pose-wise visualizations with shared color scales.

[Q2.2] Robustness to Initialization and Arm-Pose Estimation Errors

We appreciate the emphasis on robustness under deployment uncertainty. We have added two perturbation experiments across 25 reference arm configurations.

Initial garment and gripper position errors. This experiment examines uncertainty in garment-position observations and initial grasp-point locations. We evaluate 1,000 trials by applying 3D translational perturbations of up to 12 cm to the initial garment and gripper positions. The simulated body remains unchanged. Perturbations causing initial gripper penetration are rejected and resampled.

Perturbation RadiusTrialsElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
≤ 4 cm109100.0%65.1%0.01100.0232
≤ 8 cm73093.2%57.6%0.01490.0303
≤ 12 cm1,00088.1%54.3%0.01550.0314

All trials within 4 cm successfully pass the elbow, and the cumulative success rate remains 93.2% within 8 cm. Performance gradually decreases as the error magnitude increases. The directional analysis further shows greater sensitivity to downward initialization errors.

Arm-Joint Angle Estimation Errors. This experiment examines robustness to inaccuracies in the estimated human arm pose. For each reference configuration, we keep the simulated body geometry and initial garment state fixed while perturbing the estimated shoulder and elbow angles supplied to the solver. The resulting trajectory is then executed in the unchanged reference scene. We evaluate signed angular offsets ranging from 0° to ±24° for both joints.

The results are:

ErrorElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
Shoulder Angle Error |Δφs|
0°96.6%72.4%0.01250.0259
2°96.4%72.5%0.01250.0260
4°96.0%72.8%0.01240.0258
6°97.1%73.7%0.01230.0256
8°97.7%74.4%0.01220.0254
12°98.2%73.3%0.01210.0251
16°97.1%70.4%0.01260.0258
20°96.8%67.0%0.01380.0271
24°96.3%66.2%0.01430.0277
Elbow Angle Error |Δφe|
0°98.6%73.9%0.01200.0249
2°99.2%74.0%0.01200.0249
4°99.0%74.0%0.01200.0249
6°98.8%74.1%0.01210.0250
8°98.8%73.8%0.01210.0251
12°97.4%72.6%0.01260.0257
16°95.0%70.3%0.01320.0267
20°93.3%68.1%0.01380.0276
24°90.8%64.5%0.01470.0290
Both Joints Unperturbed
Δφs = Δφe = 0100.0%76.3%0.01120.0242

GwoP maintains high elbow-passage success under small arm-pose estimation errors. When both joints are unperturbed, all 25 reference configurations successfully pass the elbow, with 76.3% mean dressing progress. For elbow-angle errors up to 8°, the aggregated success rates remain above 98%. At an absolute elbow-angle error of 24°, success decreases to 90.8%, with dressing progress of 64.5%.

The results also reveal different sensitivities to the two joint angles. Elbow-angle errors have a more pronounced effect on elbow-passage success, whereas large shoulder-angle errors primarily reduce further garment advancement.

Together, the two experiments demonstrate that GwoP maintains substantial dressing capability under moderate initial-placement and arm-pose estimation errors, while performance decreases under larger perturbations. Both experiments include unsuccessful executions in their statistics.

We have added the experimental protocols, complete quantitative results, and visualizations in Appendix D.5 (Tables 6–7, Figs. 10–11).

Reviewer_U6GAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

this paper presents GwoP, a framework for learning robot-assisted dressing trajectories across different arm poses. it trains a policy to predict gripper trajectories from observations via trajectory optimization with differentiable simulator and task loss. on top of this, the paper introduces an auxiliary loss based on shortest paths along the human body surface, motivated by helping the network learn across different arm poses without directly constraining the predicted trajectories to follow these paths. experiments in sim and real show better dressing progress than several baselines, along with ablations justifying design choices

Critical strengths

the paper studies a challenging task and evaluates the method on a large number of unseen arm poses (in sim). the improvements in dressing progress over the baselines are substantial, and it's good to see real-robot experiments in addition to sim

Critical weaknesses
  1. limited methodological contribution: training control policies by differentiating through simulators is very well-established, including for deformable manipulation. the main new component seems to be the body relative auxiliary supervision, but this objective relies on hand-designed, task-specific geometric supervision and is kind of limited to this specific application. the paper also does not convincingly clarify what is fundamentally new relative to prior differentiable policy learning methods, or why this particular auxiliary objective is especially suitable for the problem or if there are general principles on how to design auxiliary objective for wider range of tasks
  2. potentially unfair baseline comparisons due to different action spaces: the proposed method predicts independent trajectories for two grippers, whereas the baselines (namely onepolicy and tdmpc2) predict a single 3D displacement applied to a fixed set of 30 attachment vertices. i wonder if the different parameterization could confound the results
  3. evidence for the claimed advantage of body-relative guidance is not very convincing: the ablations show that the proposed auxiliary supervision improves performance, but do not establish why. moreover, the Query-to-Trajectory ablation differs from the full method mostly by decoding trajectories from the predicted queries rather than from shared features? however, the large performance difference is somewhat surprising, given that the queries are just learned features rather than explicitly constrained geometric paths. the explanation that this representation unnecessarily restricts trajectory generation is not very convincing to me. more analysis would be appreciated
  4. limited real-world validation: the experiments mostly vary two arm joint angles while keeping the human arm static during execution and using a single garment structure. the system does not handle moving human arms or closed-loop adjustment. the real robot experiment is done on a manikin only
Questions for authors
  1. could the authors compare against baselines while aligning action sapce?
  2. what exactly is the architecture of Query-to-Trajectory, and why should decoding trajectories from the 32-dimensional queries restrict the space of possible trajectories? could the authors provide stronger evidence explaining the big performance gap between this variant and the full method?
  3. could the authors better isolate the contribution of body-relative supervision from pretraining and auxiliary-task regularization?
5 comments

We thank the reviewer for the detailed feedback and for recognizing the challenging task, extensive pose evaluation, and real-robot validation. We have revised the manuscript and conducted additional experiments to address the concerns.

[W1] Methodological Contribution and Generality

We appreciate the reviewer's observation that learning control policies through differentiable simulation is already well established.

Our contribution builds on this foundation and focuses on learning a shared trajectory solver across configurations that require substantially different trajectory geometries. GwoP combines physical gradients for trajectory optimization with auxiliary body-relative supervision to capture shared task structure. Predicting relations and trajectories in parallel allows structural guidance without prescribing trajectory geometry.

Regarding the broader applicability of this design, we believe it offers a useful principle for manipulation tasks involving diverse trajectories and interactions with structured geometry: identify task-relevant structural relations shared across configurations, use them as auxiliary supervision, and leave configuration-specific trajectory geometry to physical optimization. Body-surface relations instantiate this principle for dressing. Although our experiments focus on this task, we hope this formulation provides a useful reference for designing auxiliary objectives in other deformable manipulation problems.

We have clarified this distinction from prior differentiable policy learning in Secs. 2.2 and 4.2.

[W2/Q1] Fair Comparison with Aligned Action Spaces

Thank you for identifying the difference in manipulation interfaces. The original comparison followed the single-gripper control configuration of OnePolicy, but we agree that this introduced a confounding factor when comparing it with GwoP's independently controlled grippers.

We have therefore replaced the main comparisons with two-gripper implementations of both OnePolicy and TD-MPC2. Both now independently control the same two garment grasp points used by GwoP. We additionally retain the original-style Single-Gripper OnePolicy configuration as a supplementary comparison.

The updated results are:

MethodElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
Heuristic98.0%59.3%0.01490.0348
OnePolicy (two grippers)92.9%70.1%0.01580.0300
Single-Gripper OnePolicy92.7%48.3%0.02280.0464
TD-MPC2 (two grippers)56.2%47.9%0.01950.0373
GwoP97.1%74.7%0.01150.0243

Aligning the gripper interface substantially improves OnePolicy's dressing progress, confirming the importance of this experimental factor. GwoP nevertheless achieves the highest progress and lowest deformation violations among the compared methods under the aligned manipulation interface.

We have updated the main comparison in Sec. 5.2 (Table 1, Figs. 2–3). Implementation details are provided in Appendix C.2–C.3, and the single-gripper results are retained in Appendix D.2 (Table 3, Fig. 7). Additional execution examples are included in the supplementary video.

[W3.2/Q2] Query-to-Trajectory Architecture and Performance Gap

We thank the reviewer for questioning the interpretation of the original Query-to-Trajectory (Q2T) ablation. The original variant used a Shared Decoder, which independently mapped each 32-dimensional relation query to its corresponding 3D control-point displacement. We have added a Joint Decoder that uses all 24 queries together, allowing coordinated prediction of the full trajectory.

The architectures are:

  • Q2T (Shared Decoder): Each 32D query is mapped to a 3D displacement through a shared two-layer MLP, Linear(32,128) → SiLU → Linear(128,3).
  • Q2T (Joint Decoder): All 24 queries are concatenated into a 768D vector and jointly mapped to 72 trajectory parameters through Linear(768,72).
  • Full (GwoP): The 72D trajectory parameters and 24 × 32D relation queries are predicted in parallel from the same 128D shared representation.

All three use the same relation-pretraining and joint-training objectives.

ArchitectureElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
Q2T (Shared Decoder)75.0%54.4%0.01280.0288
Q2T (Joint Decoder)95.8%64.1%0.01050.0222
Full (Parallel Prediction)97.1%74.7%0.01150.0243

The Joint Decoder improves elbow passage by 20.8 percentage points and dressing progress by 9.7 points over the Shared Decoder. This substantial improvement shows that independent per-query decoding and the lack of joint control-point coordination account for an important part of the original performance gap.

Joint decoding brings elbow-passage success close to Full, although a 10.6-point progress gap remains. Qualitative results show that both Q2T variants generally recover similar routing patterns, while inaccuracies in the locations and magnitudes of trajectory turns can limit further garment advancement, especially in geometrically challenging poses.

We agree that the original explanation in terms of an inherently restricted trajectory solution space was too strong. The results support an empirical advantage of predicting trajectories and relations in parallel, without establishing that relation queries necessarily reduce representational capacity. We have revised the discussion accordingly.

The updated configurations are in Appendix C.4, quantitative results in Sec. 5.3 (Table 2), and qualitative comparisons in Appendix D.3.1 (Fig. 8).

[W3.1/Q3] Isolating Relation Pretraining and Continued Auxiliary Supervision

Thank you for this suggestion. We have added two ablations, w/o Relation Pretraining and w/o Auxiliary Loss, forming a 2 × 2 comparison that independently varies relation pretraining and auxiliary supervision during physical optimization.

ConfigurationRelation PretrainingAuxiliary LossElbow Pass ↑Progress ↑Edge Viol. ↓Area Viol. ↓
w/o Body-Relative GuidanceNoNo87.3%61.2%0.01430.0272
w/o Relation PretrainingNoYes88.2%72.6%0.01300.0253
w/o Auxiliary LossYesNo93.9%63.2%0.01180.0247
FullYesYes97.1%74.7%0.01150.0243

The distinction is informative. With continued auxiliary supervision retained, removing relation pretraining reduces elbow-passage success from 97.1% to 88.2%, while progress changes from 74.7% to 72.6%. Conversely, with relation pretraining retained, removing continued auxiliary supervision reduces progress from 74.7% to 63.2%.

These results indicate complementary contributions: relation pretraining provides a useful initialization for reliable elbow passage, while continued auxiliary supervision plays a larger role in further garment advancement.

We also extended the cross-pose gradient analysis. Adding continued auxiliary supervision increases the fraction of positively aligned pose pairs from 47.2% to 66.7% and reduces mean projected-gradient magnitude from 7.97 to 2.86 when relation pretraining is retained.

Results are reported in Sec. 5.3 (Table 2, Fig. 4) and Appendix D.3.2 (Table 4).

[W4] Real-World Validation and Current Scope

We appreciate the reviewer's concern about practical deployment. Our present real-robot evaluation uses static mannequin arm configurations and a short-sleeved open-front garment. We use a mannequin with marked joint configurations to obtain controlled, repeatable experiments across 12 arm poses, with five executions per pose. The resulting 60 trials include 46 successful elbow passages.

To further investigate sensitivity within this setup, we have added simulations with initial garment/gripper displacement errors and arm-pose estimation errors (Appendix D.5, Tables 6–7, Figs. 10–11). These experiments evaluate robustness to initialization and perception uncertainties, while also suggesting some tolerance to minor arm pose fluctuations during execution.

The current system predicts an open-loop trajectory from the initial observation. Closed-loop replanning, moving human arms, and generalization to substantially different garment structures remain important future directions, as discussed in Sec. 6.

Comments on the paper

1 comment

Summary of Revisions and Additional Experiments

We thank all reviewers for their constructive feedback. We have revised the manuscript and conducted several additional experiments to address the main concerns:

  1. Fair baseline comparisons. We implemented independently controllable two-gripper versions of OnePolicy and TD-MPC2. Under the aligned manipulation interface, GwoP achieves the highest dressing progress and lowest deformation violations. Results are in Sec. 5.2 (Table 1, Figs. 2–3) and Appendix C.2–C.3, D.2.

  2. Body-relative guidance analysis. We added ablations separating relation pretraining from continued auxiliary supervision, along with a joint Query-to-Trajectory decoder. The results clarify their complementary contributions and support the empirical advantage of parallel trajectory and relation prediction. See Sec. 5.3 (Table 2, Fig. 4) and Appendix C.4, D.3.

  3. Generalization and training consistency. We evaluated three independent training seeds and separately reported interpolation and extrapolation performance. The results demonstrate consistent performance across seeds, with the main improvements in dressing progress and deformation control maintained. See Sec. 5.1 and Appendix B.3 (Fig. 6), D.4 (Table 5, Fig. 9).

  4. Robustness to uncertainty. We conducted 1,000 initial garment/gripper position perturbation trials and 5,621 arm-pose estimation perturbation trials. The results show substantial tolerance to moderate errors, with performance degradation under larger perturbations. See Appendix D.5 (Tables 6–7, Figs. 10–11).

We have also updated the supplementary video with additional execution examples. Together, these revisions provide stronger evidence for the effectiveness, generalization, and robustness of GwoP, while clarifying its current limitations.

We sincerely appreciate the reviewers' time and thoughtful feedback, and hope that these revisions and additional results adequately address their concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.