- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ReSynC to improve the robustness of action-chunking generative policies using offline demonstrations. During training, a backward diffusion policy and backward dynamics model generate short recovery segments, which are stitched onto expert action chunks. At inference, forward and backward dynamics models score candidate chunks using cycle-consistency error, followed by contrastive selection. The method is evaluated on five simulated manipulation tasks and two physical robot tasks.
Critical strengths
-
Relevant problem and useful setting. Improving recovery without additional expert supervision or environment interaction is valuable. Aligning corrective data with the action sequences used to train the policy is a reasonable idea.
-
Consistent improvements over the base policy. ReSynC improves over Diffusion Policy on all five simulation tasks, with gains of 11.3 percentage points on Transport and 12.7 on Kitchen. The full-demonstration experiments also show that the improvements are not limited to a particularly weak, low-data baseline.
-
Several useful ablations. The paper examines the augmentation strategy, backward-policy architecture, inference selection, and key hyperparameters. These experiments help identify where the gains come from, and the limitations section is candid about the current evaluation scope.
Critical weaknesses
-
The synthesized recoveries are not directly validated. The main claim is that backward generation produces corrective behavior that returns the policy to expert trajectories. However, the paper does not measure whether executing a generated action sequence from its synthesized predecessor state actually reaches the intended anchor under the true dynamics. Appendix C acknowledges that recovery is only evaluated indirectly through task performance and injected action noise. These results show that the augmentation is useful, but do not establish whether it works by generating valid recoveries or through a more general regularization effect. Held-out forward/backward prediction errors and simulator validation of generated segments would directly address this central claim. Table 13 evaluates CCE as an OOD score and does not resolve this question.
-
The treatment of ambiguous predecessor states needs stronger justification. The backward policy can represent multiple action sequences, but the backward dynamics model predicts predecessor states deterministically using a squared-error objective. Multiple predecessor states may remain possible even after conditioning on the endpoint and action sequence, particularly with partial observations or uncertain contact modes. In such cases, the predictor may average incompatible states. The paper discusses stochastic and irreversible tasks as limitations, but does not establish when this conditional ambiguity is small enough for its model or show how ambiguity affects generated recoveries. This leaves the applicability conditions of the approach unclear.
-
The contrastive ablation does not isolate the contribution of negative candidates. Table 3 compares uniform sampling from the low-CCE set against a rule that selects a representative positive candidate while separating it from negative candidates. This changes both the selection rule and the use of negatives. Selecting the positive-set medoid, using only the first term of Eq. (3), would isolate the negative term. Without this control, the conclusion that the contrastive penalty provides the additional benefit is not established. The small gains in the limited-demonstration setting make this distinction especially relevant.
-
Practical validation is limited to low-dimensional state observations. All experiments use structured state inputs. The physical experiments use AprilTag-derived positions and gripper status for box pushing and banana lifting, with ten trials per task. Appendix C acknowledges the absence of visual evaluation and suggests using a pretrained latent space, but this extension is not implemented or tested. Since both recovery synthesis and action selection depend on state prediction, visual observations introduce a substantial additional challenge rather than a straightforward input replacement. The current results therefore establish a narrower scope than general visual manipulation.
Questions for authors
-
Can the authors report held-out forward/backward dynamics errors over the horizons actually used for synthesis and selection? More importantly, when generated predecessor states can be instantiated in simulation, what fraction are physically valid, and how closely do the generated actions reach their intended expert anchors? Please compare Gaussian and diffusion generation.
-
Under what task and observation conditions is the predecessor-state distribution sufficiently concentrated for a deterministic backward dynamics model? Can the authors provide a controlled evaluation or failure analysis showing how predecessor ambiguity affects recovery validity and downstream performance?
-
How does the proposed selector compare with selecting the medoid of the low-CCE set, using the same candidates and removing only the negative-distance term from Eq. (3)?
-
Can the authors evaluate ReSynC on at least one real-world task with an image-conditioned policy? Please explain how synthetic recovery generation and forward/backward dynamics are implemented in the visual or latent representation, and whether the improvement over the corresponding base policy persists.
My main concern is that the paper evaluates the final policy much more thoroughly than the learned models responsible for generating its recovery data. Direct validation of those recoveries, clearer applicability conditions, and stronger isolation of the proposed design choices would make the contribution more convincing.
7 comments
Response to the Reviewer ypYo (2)
Q2. Under what task and observation conditions is the predecessor-state distribution sufficiently concentrated for a deterministic backward dynamics model? Can the authors provide a controlled evaluation or failure analysis showing how predecessor ambiguity affects recovery validity and downstream performance?
A2. Regarding the conditions, a deterministic backward dynamics model suffices when, once the actions are given, the predecessor state is nearly determined. ReSynC factorizes synthesis as . Predecessor multimodality arises mainly from which actions led to , and is modeled by the diffusion backward policy (first factor). The deterministic backward dynamics model only approximates the second factor, which is concentrated when (i) the observation is approximately Markov, (ii) the transition is locally invertible given the action, and (iii) the backward horizon is short (two steps in all main experiments). In general, concentration breaks at contact-mode boundaries where different states lead to the same outcome (e.g., an object pushed against a wall), and under strong partial observation such as images or occlusion. Our benchmarks involve only mild partial observation. When this ambiguity is large, a stochastic or ensemble backward dynamics model is a natural extension. We have also clarified in Sec. 4.1 that the "several predecessor states" sentence refers to the backward policy.
Regarding recovery validity, on Push-T, conditioning on the actions removes most of the predecessor ambiguity, and starting from the predicted predecessor state brings the system within a few pixels of the demonstrated state. We measure the variance of the two-step change among the -nearest neighbors, with and without conditioning on the actions . Conditioning on the actions reduces the spread from 0.81 to 0.30, and the residual lies mostly in the T block, which moves only through contact. We then measured recovery validity by simulator replay on held-out episodes. We start from a predecessor state, execute the recorded actions , and compare the reached state with .
Table ypYo-3. Push-T replay error at held-out steps. px and ° are mean absolute errors.
| Start state | MSE (all) | MSE (contact) | MSE (no contact) | Agent (px) | Block (px / °) |
|---|---|---|---|---|---|
| True | 0.012 | 0.017 | 0.006 | 1.8 | 0.4 / 0.2 |
| Predicted | 0.548 | 0.328 | 0.781 | 5.3 | 4.4 / 1.5 |
Starting from the predicted , the agent and the block end about 5 px from in the 512 × 512 workspace, compared with under 2 px from the true . The error is larger in non-contact steps, where the agent moves faster and the backward model must predict a larger two-step displacement. Contact steps have a smaller error, but this is where physically invalid states occur (Table ypYo-2), because overlaps arise only in contact. Replay from the true gives a small error, so the velocities that Push-T omits from the state have a negligible effect.
Regarding downstream performance, after a state perturbation, ReSynC returns closer to the demonstrations than DP on Square and Transport, but not on Push-T. We perturbed demonstration states (robot joint angles on Square and Transport; agent and block positions on Push-T) and measured the distance to the demonstration distribution one action chunk later. Compared with DP, ReSynC returns significantly closer on Square at 6 of 6 noise levels and on Transport at 3 of 6, but on Push-T at none (Table 19). This one-chunk distance measure differs from the task-level recovery in Table YgCM-1, where RS does help on Push-T. We have added these conditions to Sec. 4.1 and these analyses to Appendix D.4 (Tables 18 and 19).
Response to the Reviewer ypYo (3)
Q3. How does the proposed selector compare with selecting the medoid of the low-CCE set, using the same candidates and removing only the negative-distance term from Eq. (3)?
A3. Removing the negative term leaves no gain over uniform sampling, so the additional benefit is associated with that term. With identical candidates, applied to the same DP trained without Reciprocal Synthesis, the positive-set medoid (P-Medoid) performs about the same as uniform sampling from the positive set (PS), while our contrastive selection (CS) reaches 50.4 and outperforms P-Medoid on four tasks, with Push-T on par. The margin is modest, and we have described it as such in Section 5.2; per-level results are in Table 21.
Table ypYo-4. Comparison of candidate selection strategies (performance in %, averaged over the five noise levels of Figure 3a; mean ± std over 3 seeds, 50 initial conditions each).
| Task | PS | P-Medoid | CS (ours) |
|---|---|---|---|
| Push-T | 56.5 ± 1.9 | 56.0 ± 2.3 | 56.3 ± 2.4 |
| Square | 59.7 ± 4.2 | 59.1 ± 3.8 | 62.3 ± 3.0 |
| Transport | 29.3 ± 2.7 | 27.9 ± 0.5 | 30.9 ± 3.7 |
| Tool-Hang | 23.2 ± 2.5 | 22.7 ± 1.5 | 25.1 ± 2.8 |
| Kitchen | 74.1 ± 3.0 | 75.6 ± 1.7 | 77.6 ± 1.7 |
| Average | 48.6 | 48.3 | 50.4 |
Q4. Can the authors evaluate ReSynC on at least one real-world task with an image-conditioned policy? Please explain how synthetic recovery generation and forward/backward dynamics are implemented in the visual or latent representation, and whether the improvement over the corresponding base policy persists.
A4. The improvement over the base policy persists with an image-conditioned policy. On a real-world UR5 task, Reciprocal Synthesis raises success from 60% to 80%.
The robot must push a cube from a random initial position into a fixed target region marked as a small square on the table, and a trial succeeds if the cube ends inside this region. We use 97 teleoperated demonstrations. Observations are two RGB cameras plus the end-effector position; the action is the 3D end-effector position.
RS raises success from 60% to 80%, while CS alone leaves success unchanged. Because this is a single task with 10 trials, we present it as evidence that ReSynC carries over to image-conditioned policies, not as a precise estimate of the gain. We have added this experiment to Appendix F and its scope to the Limitations.
Regarding the implementation, all components operate in the latent space of the base policy's image encoder, so synthesized recoveries are used for training without rendering images. We first train the base image diffusion policy (DP) end-to-end, then freeze its observation encoder and use its output as the latent state. All four variants share this encoder, so they are compared on the same representation.
- Dynamics: forward and backward models are trained offline on encoded demonstrations to predict or from and the action.
- Reciprocal Synthesis (RS): at each anchor frame, a backward diffusion policy conditioned on the anchor images proposes preceding actions, and the backward latent dynamics rolls them out into off-demonstration predecessor latents. These are stitched to the demonstrated suffix as in the state-based version. Because the policy head takes as input, these latents are used for training directly.
- Contrastive selection (CS): each candidate action chunk is scored by its forward–backward round-trip error in latent space, i.e., how closely predicting forward and then backward with the two dynamics models returns to the current latent.
Table ypYo-5. Real-world image-conditioned UR5 pushing task (10 trials per method). Time to success is averaged over successful trials.
| Metric | DP | DP + CS | DP + RS | ReSynC (RS+CS) |
|---|---|---|---|---|
| Success (%) | 60.0 | 60.0 | 80.0 | 80.0 |
| Time to success (s) | 42.2 ± 13.7 | 37.1 ± 10.8 | 40.1 ± 9.2 | 39.4 ± 8.1 |
Response to the Reviewer ypYo
Q1. Can the authors report held-out forward/backward dynamics errors over the horizons actually used for synthesis and selection? More importantly, when generated predecessor states can be instantiated in simulation, what fraction are physically valid, and how closely do the generated actions reach their intended expert anchors? Please compare Gaussian and diffusion generation.
A1. We agree that the learned models that generate the recovery data should be evaluated directly, and we do so with all three measurements below.
Regarding held-out dynamics errors, the backward model is most accurate over the two steps used for synthesis, and errors grow over the longer horizon used for selection (Table 16, Appendix D.2). We evaluate both dynamics models on demonstrations held out from their training. 'Synthesis' uses the backward model over two steps. 'Selection' uses both models over the executed horizon (8 steps; 15 for Tool-Hang). Errors are L2 distances over the full state vector, with states normalized to [−1, 1]. Because the error is taken over the full vector, values are not directly comparable across tasks with different state dimensions. Kitchen has clearly larger errors than the other tasks: it is the task with the fewest training demonstrations (5% of the dataset).
Table ypYo-1. Held-out prediction error of the forward and backward dynamics models.
Regarding physical validity, our generator produces valid states more often than the Gaussian generator (83% vs. 72% on Push-T, 75% vs. 21% on Square), though less often than expert states. For 1000 held-out expert states per task, used as anchors (the states a recovery should return to), we generate a segment exactly as in training, reset the simulator to the synthesized predecessor state, execute the synthesized actions, and compare the reached state with the anchor. A state counts as physically valid if objects do not overlap beyond what occurs in expert data. We compare our generator (diffusion backward policy with the multi-step backward dynamics) with the BMIL-style Gaussian generator (Gaussian backward policy with its one-step backward model) on the same anchors. The two generators therefore differ in both the backward policy and the backward dynamics model.
Table ypYo-2. Simulator validation of synthesized segments (M = 2, 1000 anchors per task).
The largest gap is on Square, where most Gaussian-generated states place the gripper or the nut in collision. Synthesized states are still less valid than expert states, which are over 99% valid under the same test; the failures occur mostly when the robot is in contact with the object.
Regarding how closely the generated actions reach their intended anchors, on Push-T our synthesized segments move toward the anchor far more often (86% vs. 46%), but on Square neither generator does (53% vs. 51%). On Square, the backward dynamics error is larger than the distance the system moves in two steps, so the synthesized state is not a precise predecessor of the anchor. On Square, our evidence supports physical validity but not recovery at the level of single segments. Recovery of the trained policy is measured directly in the perturbation experiment on Push-T in our response to Reviewer YgCM (Q2), where training with Reciprocal Synthesis (RS) makes the policy recover more often than DP. We added these results and their scope to Appendix D.3.