acceptodds
Under review as a conference paper at ICLR 2027

Chunk-Aligned Recovery Synthesis for Robust Generative Control Policies

Abstract

Generative control policies, such as diffusion and flow-matching policies, have demonstrated strong performance in behavior cloning from multimodal demonstra- tions, but remain susceptible to covariate shift: small action errors accumulate over time, driving the policy into states outside the demonstration distribution, where its predictions become unreliable. Existing corrective-augmentation methods address this issue by synthesizing recovery data as isolated transitions. Such transitions are poorly suited to action-chunking policies, which are trained to produce temporally coherent sequences rather than individual corrective steps. We propose ReSynC (Reciprocal Synthesis and Cycle Consistency), which synthesizes recovery behav- ior in a chunk-aligned form: a truncated backward diffusion policy and a backward dynamics model generate a plausible deviation together with its recovery, and this segment replaces the prefix of an expert action chunk, so that the resulting training sample starts from an off-distribution state and ends in expert behavior. Chunk-aligned placement and an expressive backward generator interact: neither provides a consistent benefit without the other. At inference time, ReSynC reuses these dynamics models to score candidate action chunks by their cycle-consistency error and contrastively selects among the low-error candidates. We find that this score reflects whether a candidate remains within the data distribution used to fit the dynamics, rather than how accurate those dynamics are, and that this signal remains reliable under limited demonstrations. Across five simulated manipulation tasks and a physical robot arm, ReSynC improves average robustness under limited demonstrations without additional human supervision or environment interaction.

Primary area:
reinforcement learning
Keywords:
Imitation learning, Generative policy, robotics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ypYoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReSynC to improve the robustness of action-chunking generative policies using offline demonstrations. During training, a backward diffusion policy and backward dynamics model generate short recovery segments, which are stitched onto expert action chunks. At inference, forward and backward dynamics models score candidate chunks using cycle-consistency error, followed by contrastive selection. The method is evaluated on five simulated manipulation tasks and two physical robot tasks.

Critical strengths
  1. Relevant problem and useful setting. Improving recovery without additional expert supervision or environment interaction is valuable. Aligning corrective data with the action sequences used to train the policy is a reasonable idea.

  2. Consistent improvements over the base policy. ReSynC improves over Diffusion Policy on all five simulation tasks, with gains of 11.3 percentage points on Transport and 12.7 on Kitchen. The full-demonstration experiments also show that the improvements are not limited to a particularly weak, low-data baseline.

  3. Several useful ablations. The paper examines the augmentation strategy, backward-policy architecture, inference selection, and key hyperparameters. These experiments help identify where the gains come from, and the limitations section is candid about the current evaluation scope.

Critical weaknesses
  1. The synthesized recoveries are not directly validated. The main claim is that backward generation produces corrective behavior that returns the policy to expert trajectories. However, the paper does not measure whether executing a generated action sequence from its synthesized predecessor state actually reaches the intended anchor under the true dynamics. Appendix C acknowledges that recovery is only evaluated indirectly through task performance and injected action noise. These results show that the augmentation is useful, but do not establish whether it works by generating valid recoveries or through a more general regularization effect. Held-out forward/backward prediction errors and simulator validation of generated segments would directly address this central claim. Table 13 evaluates CCE as an OOD score and does not resolve this question.

  2. The treatment of ambiguous predecessor states needs stronger justification. The backward policy can represent multiple action sequences, but the backward dynamics model predicts predecessor states deterministically using a squared-error objective. Multiple predecessor states may remain possible even after conditioning on the endpoint and action sequence, particularly with partial observations or uncertain contact modes. In such cases, the predictor may average incompatible states. The paper discusses stochastic and irreversible tasks as limitations, but does not establish when this conditional ambiguity is small enough for its model or show how ambiguity affects generated recoveries. This leaves the applicability conditions of the approach unclear.

  3. The contrastive ablation does not isolate the contribution of negative candidates. Table 3 compares uniform sampling from the low-CCE set against a rule that selects a representative positive candidate while separating it from negative candidates. This changes both the selection rule and the use of negatives. Selecting the positive-set medoid, using only the first term of Eq. (3), would isolate the negative term. Without this control, the conclusion that the contrastive penalty provides the additional benefit is not established. The small gains in the limited-demonstration setting make this distinction especially relevant.

  4. Practical validation is limited to low-dimensional state observations. All experiments use structured state inputs. The physical experiments use AprilTag-derived positions and gripper status for box pushing and banana lifting, with ten trials per task. Appendix C acknowledges the absence of visual evaluation and suggests using a pretrained latent space, but this extension is not implemented or tested. Since both recovery synthesis and action selection depend on state prediction, visual observations introduce a substantial additional challenge rather than a straightforward input replacement. The current results therefore establish a narrower scope than general visual manipulation.

Questions for authors
  1. Can the authors report held-out forward/backward dynamics errors over the horizons actually used for synthesis and selection? More importantly, when generated predecessor states can be instantiated in simulation, what fraction are physically valid, and how closely do the generated actions reach their intended expert anchors? Please compare Gaussian and diffusion generation.

  2. Under what task and observation conditions is the predecessor-state distribution sufficiently concentrated for a deterministic backward dynamics model? Can the authors provide a controlled evaluation or failure analysis showing how predecessor ambiguity affects recovery validity and downstream performance?

  3. How does the proposed selector compare with selecting the medoid of the low-CCE set, using the same candidates and removing only the negative-distance term from Eq. (3)?

  4. Can the authors evaluate ReSynC on at least one real-world task with an image-conditioned policy? Please explain how synthetic recovery generation and forward/backward dynamics are implemented in the visual or latent representation, and whether the improvement over the corresponding base policy persists.

My main concern is that the paper evaluates the final policy much more thoroughly than the learned models responsible for generating its recovery data. Direct validation of those recoveries, clearer applicability conditions, and stronger isolation of the proposed design choices would make the contribution more convincing.

7 comments
AuthorsOpenReview ↗

Response to the Reviewer ypYo

Q1. Can the authors report held-out forward/backward dynamics errors over the horizons actually used for synthesis and selection? More importantly, when generated predecessor states can be instantiated in simulation, what fraction are physically valid, and how closely do the generated actions reach their intended expert anchors? Please compare Gaussian and diffusion generation.

A1. We agree that the learned models that generate the recovery data should be evaluated directly, and we do so with all three measurements below.

Regarding held-out dynamics errors, the backward model is most accurate over the two steps used for synthesis, and errors grow over the longer horizon used for selection (Table 16, Appendix D.2). We evaluate both dynamics models on demonstrations held out from their training. 'Synthesis' uses the backward model over two steps. 'Selection' uses both models over the executed horizon (8 steps; 15 for Tool-Hang). Errors are L2 distances over the full state vector, with states normalized to [−1, 1]. Because the error is taken over the full vector, values are not directly comparable across tasks with different state dimensions. Kitchen has clearly larger errors than the other tasks: it is the task with the fewest training demonstrations (5% of the dataset).

Table ypYo-1. Held-out prediction error of the forward and backward dynamics models.

Task (state dim.)Backward, 2 steps (synthesis)Forward, full horizon (selection)Backward, full horizon (selection)
Push-T (20)0.1280.1720.194
Square (23)0.1110.1390.140
Transport (59)0.0930.0990.109
Tool-Hang (53)0.1290.1730.196
Kitchen (60)0.5110.5330.540

Regarding physical validity, our generator produces valid states more often than the Gaussian generator (83% vs. 72% on Push-T, 75% vs. 21% on Square), though less often than expert states. For 1000 held-out expert states per task, used as anchors (the states a recovery should return to), we generate a segment exactly as in training, reset the simulator to the synthesized predecessor state, execute the synthesized actions, and compare the reached state with the anchor. A state counts as physically valid if objects do not overlap beyond what occurs in expert data. We compare our generator (diffusion backward policy with the multi-step backward dynamics) with the BMIL-style Gaussian generator (Gaussian backward policy with its one-step backward model) on the same anchors. The two generators therefore differ in both the backward policy and the backward dynamics model.

Table ypYo-2. Simulator validation of synthesized segments (M = 2, 1000 anchors per task).

TaskGeneratorPhysically valid states (%)Segments ending closer to the anchor (%)
Push-TGaussian7246
Push-TDiffusion (ours)8386
SquareGaussian2151
SquareDiffusion (ours)7553

The largest gap is on Square, where most Gaussian-generated states place the gripper or the nut in collision. Synthesized states are still less valid than expert states, which are over 99% valid under the same test; the failures occur mostly when the robot is in contact with the object.

Regarding how closely the generated actions reach their intended anchors, on Push-T our synthesized segments move toward the anchor far more often (86% vs. 46%), but on Square neither generator does (53% vs. 51%). On Square, the backward dynamics error is larger than the distance the system moves in two steps, so the synthesized state is not a precise predecessor of the anchor. On Square, our evidence supports physical validity but not recovery at the level of single segments. Recovery of the trained policy is measured directly in the perturbation experiment on Push-T in our response to Reviewer YgCM (Q2), where training with Reciprocal Synthesis (RS) makes the policy recover more often than DP. We added these results and their scope to Appendix D.3.

AuthorsOpenReview ↗

Response to the Reviewer ypYo (2)

Q2. Under what task and observation conditions is the predecessor-state distribution sufficiently concentrated for a deterministic backward dynamics model? Can the authors provide a controlled evaluation or failure analysis showing how predecessor ambiguity affects recovery validity and downstream performance?

A2. Regarding the conditions, a deterministic backward dynamics model suffices when, once the actions are given, the predecessor state is nearly determined. ReSynC factorizes synthesis as p(A^past∣s∗)⋅p(S^past∣s∗,A^past)p(\hat{A}_{\text{past}} \mid s^*) \cdot p(\hat{S}_{\text{past}} \mid s^*, \hat{A}_{\text{past}}). Predecessor multimodality arises mainly from which actions led to sts_t, and is modeled by the diffusion backward policy (first factor). The deterministic backward dynamics model only approximates the second factor, which is concentrated when (i) the observation is approximately Markov, (ii) the transition is locally invertible given the action, and (iii) the backward horizon is short (two steps in all main experiments). In general, concentration breaks at contact-mode boundaries where different states lead to the same outcome (e.g., an object pushed against a wall), and under strong partial observation such as images or occlusion. Our benchmarks involve only mild partial observation. When this ambiguity is large, a stochastic or ensemble backward dynamics model is a natural extension. We have also clarified in Sec. 4.1 that the "several predecessor states" sentence refers to the backward policy.

Regarding recovery validity, on Push-T, conditioning on the actions removes most of the predecessor ambiguity, and starting from the predicted predecessor state brings the system within a few pixels of the demonstrated state. We measure the variance of the two-step change st−2−sts_{t-2} - s_t among the kk-nearest neighbors, with and without conditioning on the actions At−2A_{t-2}. Conditioning on the actions reduces the spread from 0.81 to 0.30, and the residual lies mostly in the T block, which moves only through contact. We then measured recovery validity by simulator replay on held-out episodes. We start from a predecessor state, execute the recorded actions at−2,at−1a_{t-2}, a_{t-1}, and compare the reached state with sts_t.

Table ypYo-3. Push-T replay error at held-out steps. px and ° are mean absolute errors.

Start stateMSE (all)MSE (contact)MSE (no contact)Agent (px)Block (px / °)
True st−2s_{t-2}0.0120.0170.0061.80.4 / 0.2
Predicted st−2s_{t-2}0.5480.3280.7815.34.4 / 1.5

Starting from the predicted st−2s_{t-2}, the agent and the block end about 5 px from sts_t in the 512 × 512 workspace, compared with under 2 px from the true st−2s_{t-2}. The error is larger in non-contact steps, where the agent moves faster and the backward model must predict a larger two-step displacement. Contact steps have a smaller error, but this is where physically invalid states occur (Table ypYo-2), because overlaps arise only in contact. Replay from the true st−2s_{t-2} gives a small error, so the velocities that Push-T omits from the state have a negligible effect.

Regarding downstream performance, after a state perturbation, ReSynC returns closer to the demonstrations than DP on Square and Transport, but not on Push-T. We perturbed demonstration states (robot joint angles on Square and Transport; agent and block positions on Push-T) and measured the distance to the demonstration distribution one action chunk later. Compared with DP, ReSynC returns significantly closer on Square at 6 of 6 noise levels and on Transport at 3 of 6, but on Push-T at none (Table 19). This one-chunk distance measure differs from the task-level recovery in Table YgCM-1, where RS does help on Push-T. We have added these conditions to Sec. 4.1 and these analyses to Appendix D.4 (Tables 18 and 19).

AuthorsOpenReview ↗

Response to the Reviewer ypYo (3)

Q3. How does the proposed selector compare with selecting the medoid of the low-CCE set, using the same candidates and removing only the negative-distance term from Eq. (3)?

A3. Removing the negative term leaves no gain over uniform sampling, so the additional benefit is associated with that term. With identical candidates, applied to the same DP trained without Reciprocal Synthesis, the positive-set medoid (P-Medoid) performs about the same as uniform sampling from the positive set (PS), while our contrastive selection (CS) reaches 50.4 and outperforms P-Medoid on four tasks, with Push-T on par. The margin is modest, and we have described it as such in Section 5.2; per-level results are in Table 21.

Table ypYo-4. Comparison of candidate selection strategies (performance in %, averaged over the five noise levels of Figure 3a; mean ± std over 3 seeds, 50 initial conditions each).

TaskPSP-MedoidCS (ours)
Push-T56.5 ± 1.956.0 ± 2.356.3 ± 2.4
Square59.7 ± 4.259.1 ± 3.862.3 ± 3.0
Transport29.3 ± 2.727.9 ± 0.530.9 ± 3.7
Tool-Hang23.2 ± 2.522.7 ± 1.525.1 ± 2.8
Kitchen74.1 ± 3.075.6 ± 1.777.6 ± 1.7
Average48.648.350.4

Q4. Can the authors evaluate ReSynC on at least one real-world task with an image-conditioned policy? Please explain how synthetic recovery generation and forward/backward dynamics are implemented in the visual or latent representation, and whether the improvement over the corresponding base policy persists.

A4. The improvement over the base policy persists with an image-conditioned policy. On a real-world UR5 task, Reciprocal Synthesis raises success from 60% to 80%.

The robot must push a cube from a random initial position into a fixed target region marked as a small square on the table, and a trial succeeds if the cube ends inside this region. We use 97 teleoperated demonstrations. Observations are two RGB cameras plus the end-effector position; the action is the 3D end-effector position.

RS raises success from 60% to 80%, while CS alone leaves success unchanged. Because this is a single task with 10 trials, we present it as evidence that ReSynC carries over to image-conditioned policies, not as a precise estimate of the gain. We have added this experiment to Appendix F and its scope to the Limitations.

Regarding the implementation, all components operate in the latent space of the base policy's image encoder, so synthesized recoveries are used for training without rendering images. We first train the base image diffusion policy (DP) end-to-end, then freeze its observation encoder and use its output ztz_t as the latent state. All four variants share this encoder, so they are compared on the same representation.

  • Dynamics: forward and backward models are trained offline on encoded demonstrations to predict zt+1z_{t+1} or zt−1z_{t-1} from ztz_t and the action.
  • Reciprocal Synthesis (RS): at each anchor frame, a backward diffusion policy conditioned on the anchor images proposes preceding actions, and the backward latent dynamics rolls them out into off-demonstration predecessor latents. These are stitched to the demonstrated suffix as in the state-based version. Because the policy head takes ztz_t as input, these latents are used for training directly.
  • Contrastive selection (CS): each candidate action chunk is scored by its forward–backward round-trip error in latent space, i.e., how closely predicting forward and then backward with the two dynamics models returns to the current latent.

Table ypYo-5. Real-world image-conditioned UR5 pushing task (10 trials per method). Time to success is averaged over successful trials.

MetricDPDP + CSDP + RSReSynC (RS+CS)
Success (%)60.060.080.080.0
Time to success (s)42.2 ± 13.737.1 ± 10.840.1 ± 9.239.4 ± 8.1
Reviewer_pSP7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses compounding errors in generative imitation learning through recovery-data synthesis and inference-time action selection. During training, a backward diffusion policy and backward dynamics model generate corrective trajectories, which replace the prefix of expert action chunks. This aligns the synthesized recovery behavior with the temporal structure consumed by chunk-based policies. At inference, forward and backward dynamics models compute a cycle-consistency error for candidate action chunks, followed by contrastive selection within the low-error subset. Experiments cover five simulated manipulation tasks and two physical-robot tasks, with additional analyses of perturbation robustness, component contributions, and hyperparameter sensitivity.

Critical strengths
  1. A well-motivated contribution to chunk-based imitation learning. The method connects the structure of data augmentation to the policy’s prediction format, offering a concrete explanation for why existing augmentation strategies may transfer poorly to action-chunking policies.
  2. Useful analysis of chunk alignment. The factorial experiment separates the backward-policy family from the stitching strategy, providing evidence that their combination matters, particularly on Square.
Critical weaknesses

The proposed mechanisms are only indirectly validated. Task success and robustness to action noise do not directly establish that synthesized trajectories teach recovery to the expert distribution; Appendix C explicitly acknowledges this. Likewise, Table 13 evaluates CCE against a nearest-neighbor OOD proxy on Push-T, without directly separating the effects of dynamics accuracy and distribution coverage.

Questions for authors
  1. Does contrastive selection outperform a positive-only medoid selector, using identical candidate pools?
  2. Can you quantify recovery success after controlled deviations and validate CCE against actual rollout outcomes?
  3. What are the training cost, inference hardware, and physical-robot demonstration budgets?
4 comments
AuthorsOpenReview ↗

Response to the Reviewer pSP7

We appreciate the reviewer's encouraging assessment and thoughtful questions. We have conducted additional experiments to address the feedback, as detailed below.

Q1. Does contrastive selection outperform a positive-only medoid selector, using identical candidate pools?

A1. Yes, contrastive selection (CS) outperforms the positive-only medoid selector (P-Medoid). We evaluated P-Medoid, which uses only the first term of Eq. (3), using identical candidate pools across five tasks and five noise levels. P-Medoid achieves performance comparable to uniform sampling from the positive set (PS, 48.3 vs. 48.6 on average), whereas CS achieves 50.4 and outperforms the medoid on four of five tasks and is on par on Push-T. These results suggest that selecting a representative candidate from the positive set alone provides little benefit, while the additional improvement from CS is associated with the second term of Eq. (3), which pushes the selection away from the negative set. Per-level results are in Table 21.

Table pSP7-1. Comparison of candidate selection strategies (performance in %, averaged over the five noise levels of Figure 3a; mean ± std over 3 seeds, 50 initial conditions each).

TaskPSP-MedoidCS (ours)
Push-T56.5 ± 1.956.0 ± 2.356.3 ± 2.4
Square59.7 ± 4.259.1 ± 3.862.3 ± 3.0
Transport29.3 ± 2.727.9 ± 0.530.9 ± 3.7
Tool-Hang23.2 ± 2.522.7 ± 1.525.1 ± 2.8
Kitchen74.1 ± 3.075.6 ± 1.777.6 ± 1.7
Average48.648.350.4

Q2. Can you quantify recovery success after controlled deviations and validate CCE against actual rollout outcomes?

A2. Regarding recovery after controlled deviations, training with Reciprocal Synthesis (RS) makes the policy recover from a displaced object more often. We designed a perturbation experiment on Push-T: when the block first reaches 50% goal coverage, we instantly move it away from the goal by half the length of the T's top bar and let the policy continue. All methods start from the same initial positions, and we run 300 episodes per method, excluding those that never reach 50% coverage before the perturbation. We report the maximum coverage after the perturbation, the fraction of episodes that regain 50% coverage, the steps needed to regain it, and the fraction of contact steps in which the pusher overlaps the block (violations).

Table pSP7-2. Perturbation recovery on Push-T for DP, DP + RS, and ReSynC (mean ± std over 6 evaluation seeds).

MethodMax coverage (%)Regain 50% (%)Steps to regainViolations (%)
DP64.9 ± 6.363.4 ± 7.773 ± 72.7 ± 0.5
DP + RS73.4 ± 6.877.1 ± 10.872 ± 74.9 ± 1.5
ReSynC (RS + CS)68.2 ± 6.369.7 ± 7.472 ± 63.9 ± 0.8

With RS, the policy regains 50% coverage 13.7 points more often than DP and reaches 8.5 points higher maximum coverage. As Appendix C of the original submission acknowledged, the recovery had previously been evaluated only indirectly; this now measures it directly. ReSynC lies between DP and DP + RS and is within one standard deviation of DP + RS, so we do not claim that CS improves recovery from this type of deviation. We have reported the experiment in Table 4 (Section 5.2) and Appendix D.1, and our response to Reviewer YgCM (A2) discusses the results further.

Regarding CCE and actual rollout outcomes, episodes with lower CCE recover more often, so CCE correlates with actual recovery outcomes. Whereas Table 15 compares CCE only with a nearest-neighbor OOD measure, here we relate it directly to rollout outcomes using the ReSynC episodes from the recovery experiment above. For each episode, we compute the CCE of the chunk selected at the first decision after the perturbation, before the outcome is observed, and divide the episodes into three equal-sized groups by this value.

Table pSP7-3. Recovery outcome by CCE at the first decision after the perturbation on Push-T (mean ± standard error over the episodes in each group).

CCE groupRegain 50% (%)Max coverage (%)
Lowest third80 ± 576 ± 4
Middle third72 ± 670 ± 4
Highest third58 ± 660 ± 4

In the lowest-CCE third, 80% of episodes regain 50% coverage, compared with 72% and 58% in the middle and highest thirds, and maximum coverage follows the same trend. The trend is gradual, however, and many high-CCE episodes still recover, so CCE is informative but cannot guarantee that an individual action is safe. Finally, this analysis compares episodes rather than alternative candidates from the same state, so it does not establish how much CCE-based selection improves recovery. We have added this analysis to Appendix E.1.

AuthorsOpenReview ↗

Response to the Reviewer pSP7 (2)

Q3. What are the training cost, inference hardware, and physical-robot demonstration budgets?

A3. Training settings are listed in Tables 7–10, and training times are in Table 11. The synthetic recoveries are generated during base-policy training with two denoising steps of the backward policy. On a single NVIDIA RTX 5090, the full training pipeline takes about 4.2 h for Transport, the most demanding task in Table 11 (3.4 h for the base policy, 0.4 h for the backward policy, 0.4 h for the dynamics models), compared with 1.5 h for DP; the other tasks in Table 11 are cheaper. At inference, measured on a single NVIDIA RTX 4090, using 16 candidates increases the latency per control step by approximately 6% compared with DP (609.86 ms vs. 574.83 ms, Table 12e). For the real robot, we collected 50 demonstrations per task using keyboard teleoperation on a Sawyer arm, and each task was evaluated over 10 trials with a single training seed. For the image-conditioned experiment on a UR5 arm (Appendix F), we used 97 teleoperated demonstrations and 10 trials per method. We have added these details to Appendix A.

AuthorsOpenReview ↗

Follow-up on your concerns

Dear Reviewer,

As the discussion period draws to a close, we would like to check whether we have addressed all of your concerns satisfactorily.

We appreciate your insights and would be glad to follow up on any remaining issues. If there is any additional feedback you would like us to consider, please let us know.

Thank you for your time and effort in reviewing our paper.

Thank you for the additional experiments. The medoid comparison and controlled perturbation study strengthen the evidence for contrastive selection and recovery synthesis. The cost and hardware clarifications are also helpful. Recovery validation remains limited to one setting, and CCE’s correlation with recovery does not establish its ability to rank candidate actions. Overall, the rebuttal addresses my main concerns, and I maintain my rating of weak acceptance.

Reviewer_YgCMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReSynC to mitigate covariate shift in chunk-based diffusion policies. During training, a truncated backward diffusion policy and backward dynamics model synthesize deviation–recovery segments that replace the prefix of expert action chunks. At inference, forward–backward cycle-consistency error (CCE) ranks sampled action chunks, followed by contrastive selection. Experiments on five simulated manipulation tasks and two real-robot tasks report improvements over diffusion-policy and several corrective-augmentation or test-time baselines. This paper tends to address a critical problem in long-horizon robotic manipulation, compounding errors. While the paper addresses this problem with promising results, its methodological novelty over prior work remains unclear, and the claimed recovery mechanism lacks direct experimental validation.

Critical strengths
  1. A well-motivated, policy-aligned formulation of corrective augmentation.

Rather than inserting isolated corrective transitions, the method builds hybrid chunks that retain the sequence structure expected by an action-chunking policy. The Gaussian/diffusion × isolated/stitched factorial study offers useful evidence that an expressive backward generator and chunk-aligned placement work together; this is a substantive methodological insight beyond generic data augmentation. 2. Meaningful performance gains under limited demonstrations, with some physical validation.

ReSynC improves upon the base diffusion policy on all five simulated tasks in Table 1 (e.g., Transport 28.7→40.0; Kitchen 78.0→90.7), and reports a 60% mean success rate versus 20% for the base policy on two Sawyer tasks (Table 4). Perturbation tests and component ablations further support the relevance of the overall framework, although the real-robot sample size is modest.

Critical weaknesses
  1. Limited methodological novelty over existing recovery synthesis and trajectory stitching methods.

The main components of ReSynC have close precedents in prior work. BMIL already uses backward models to synthesize corrective demonstrations; RTDiff and DiffStitch explore diffusion-based trajectory synthesis and stitching; and CABI employs bidirectional dynamics consistency to assess generated transitions. ReSynC primarily combines these ideas and adapts them to chunk-based diffusion policies. While this integration is practically motivated, the paper does not sufficiently establish what fundamentally new capability is enabled by chunk alignment beyond existing corrective augmentation techniques.

  1. Insufficient direct validation of the proposed recovery mechanism and the necessity of contrastive selection.

The experiments primarily measure task success and robustness under injected action noise, without directly evaluating whether the synthesized recovery trajectories are physically plausible or whether the learned policy can reliably return from off-distribution states to the expert distribution. This limits the evidence supporting the paper's central recovery claim. Furthermore, Table 3 shows that contrastive selection provides only marginal improvements over uniform sampling from low-CCE candidates in the limited-demonstration regime (1.3 points on Push-T and 0.6 points on Square). Although CCE correlates with a k-NN-based OOD measure, this does not establish that low-CCE actions are dynamically valid or more likely to recover successfully. These gaps make it difficult to determine whether the additional modeling and inference complexity is justified by the proposed mechanisms.

Questions for authors
  1. What capability is uniquely enabled by chunk-aligned stitching beyond prior backward-model corrective augmentation and diffusion-based trajectory stitching? Please isolate the contribution of temporal placement and sequence coherence with controlled comparisons (e.g., identical synthetic data volume and backward generator, varying prefix placement and recovery length) and explain why the two-step default prefix suffices.
  2. Can the authors directly measure recovery from controlled deviations (recovery success, time or distance to the expert manifold, physical constraint violations), and compare CS with positive-set uniform sampling or minimum-CCE selection across all tasks and disturbance levels under equal candidate and inference budgets?
  3. Table 3 reports Push-T/Square results for the sampling strategies (e.g., CS 62.0/67.3 at 20% demonstrations) that differ from the main ReSynC results in Table 1 (69.3/77.3). Are these different trained policies, evaluation protocols, or random seeds? Please clarify the experimental setup so the incremental contribution of CS can be interpreted consistently.
4 comments
AuthorsOpenReview ↗

Response to the Reviewer YgCM

We thank the reviewer for the detailed and thoughtful review, and for recognizing the motivation and empirical strengths of our approach. We address the comments below.

Q1. What capability is uniquely enabled by chunk-aligned stitching beyond prior backward-model corrective augmentation and diffusion-based trajectory stitching? Please isolate the contribution of temporal placement and sequence coherence with controlled comparisons (e.g., identical synthetic data volume and backward generator, varying prefix placement and recovery length) and explain why the two-step default prefix suffices.

A1. Regarding the new capability, chunk-aligned stitching produces a single training chunk that contains the deviation, the recovery, and the continuing expert actions. A chunk-based policy outputs a sequence of actions and executes part of them before replanning. To learn recovery, it needs training chunks that start away from the demonstration, return to it, and then continue with the expert actions. Prior methods do not produce such chunks. BMIL, ROMI, and CABI insert backward-generated transitions at arbitrary dataset states, and RTDiff stores its generated multi-step trajectories into the dataset as individual transitions. In both cases, corrective steps are used in isolation from the actions that follow them. DiffStitch generates coherent sequences, but they connect two existing trajectories rather than starting from a deviated state. Generating trajectories backward with diffusion, or checking forward–backward consistency, is not new in itself. Our contribution lies in the form of the training sample, and we have clarified this in Section 2 in the revised manuscript.

For the controlled comparison of placement and coherence, our factorial study in Table 2 shows that stitching helps only when the generated recovery is coherent. All four variants across two backward generators and two placements use the same amount of synthetic data, which we have clarified in the caption of Table 2. With the Gaussian generator, stitching leaves the average unchanged and lowers performance on Square. With the diffusion generator, in contrast, stitching raises the average from 58.5 to 62.2. Because stitching places the generated steps inside the chunk that the policy is trained to reproduce, a coherent recovery teaches the policy how to recover.

As for the recovery length, two steps perform best on every task we tested. Table 12a varies only the number of recovery steps from one to four. We attribute this to the fact that a single step stays too close to the demonstration to teach recovery, while longer recoveries require predicting further back from the demonstrated state, where the learned backward model becomes less accurate. Consistent with this, the held-out error of the backward dynamics model is larger over the full 8-step horizon than over two steps on every task (Table 16, Appendix D.2).

AuthorsOpenReview ↗

Response to the Reviewer YgCM (2)

Q2. Can the authors directly measure recovery from controlled deviations (recovery success, time or distance to the expert manifold, physical constraint violations), and compare CS with positive-set uniform sampling or minimum-CCE selection across all tasks and disturbance levels under equal candidate and inference budgets?

A2. Regarding direct measurement of recovery, training with Reciprocal Synthesis (RS) makes the policy recover from a displaced object more often. We designed a perturbation experiment on Push-T, where the object state can be displaced and reset exactly. When the block first reaches 50% goal coverage, we instantly move it away from the goal by half the length of the T's top bar (60 pixels in the 512 ×\times 512 workspace) and let the policy continue. All methods start from the same initial positions, and we run 300 episodes per method, excluding those that never reach 50% coverage before the perturbation (58% of episodes remain for DP, 64% for DP + RS, and 66% for ReSynC). We measure recovery success (the maximum coverage after the perturbation and the fraction of episodes that regain 50% coverage), recovery time (the steps needed to regain 50% coverage), and constraint violations (the fraction of contact steps in which the pusher overlaps the block by more than 2 pixels).

Table YgCM-1. Perturbation recovery on Push-T for DP, DP + RS, and ReSynC (mean ± std over 6 evaluation seeds).

MethodMax coverage (%)Regain 50% (%)Steps to regainViolations (%)
DP64.9 ± 6.363.4 ± 7.773 ± 72.7 ± 0.5
DP + RS73.4 ± 6.877.1 ± 10.872 ± 74.9 ± 1.5
ReSynC (RS + CS)68.2 ± 6.369.7 ± 7.472 ± 63.9 ± 0.8

With RS, the policy regains 50% coverage 13.7 points more often than DP and reaches 8.5 points higher maximum coverage. Among the episodes that recover, all methods take about the same number of steps, so RS mainly changes whether the policy recovers rather than how fast. For distance to the expert distribution, we also perturbed demonstration states on three tasks and measured the distance one action chunk later: ReSynC returns closer than DP on Square and Transport, but not on Push-T (Table 19, Appendix D.4). Violations remain rare for all methods (under 5% of contact steps). This matches our simulator check of the synthesized data, in which a fraction of the generated states contain pusher–block overlap (Table 17, Appendix D.3), and we have added this discussion in Appendix C.

Under this sudden displacement, adding CS does not further improve recovery: ReSynC and DP + RS are within one standard deviation of each other, so we do not claim a recovery benefit from CS here. After the object is displaced, every candidate action chunk starts from the same unfamiliar state, so all of them receive similarly poor consistency scores and choosing among them changes little. The two components therefore play different roles: RS teaches the policy to recover from displaced states, while CS filters out poor actions under execution noise, as shown below. We clarified this division of roles in Section 5.2.

Regarding the selector comparison, CS is the best selector on four of the five tasks and on par on Push-T, while selecting the single minimum-CCE candidate is the weakest on every task. We extended the comparison to all five tasks and the five noise levels of Figure 3a, and added the positive-set medoid (P-Medoid) as a selector suggested by Reviewers pSP7 and ypYo. P-Medoid picks the most representative low-error candidate but, unlike CS, ignores the high-error candidates. As in Table 3, all selectors are applied to the same base policy trained without RS, and they share the same 16 candidates per step and the same dynamics-model computation, so candidate and inference budgets are identical. The table below averages over noise levels, which is why values are lower than the noise-free results in Table 3. Per-level results are presented in Table 21.

Table YgCM-2. Comparison of candidate selection strategies (performance in %, mean ± std over 3 seeds).

TaskMinPSP-MedoidCS (ours)
Push-T55.3 ± 1.756.5 ± 1.956.0 ± 2.356.3 ± 2.4
Square56.0 ± 1.859.7 ± 4.259.1 ± 3.862.3 ± 3.0
Transport26.0 ± 3.729.3 ± 2.727.9 ± 0.530.9 ± 3.7
Tool-Hang22.5 ± 2.423.2 ± 2.522.7 ± 1.525.1 ± 2.8
Kitchen73.9 ± 1.374.1 ± 3.075.6 ± 1.777.6 ± 1.7
Average46.748.648.350.4

P-Medoid performs about the same as uniform positive sampling, whereas CS exceeds P-Medoid on Square, Transport, Tool-Hang, and Kitchen. Since the two differ only in whether high-error candidates are taken into account, we attribute this gain to the contrastive term in Eq. (3). The gain is modest but consistent in direction, and we have revised Section 5.2 and Appendix E.2 to describe it as such.

AuthorsOpenReview ↗

Response to the Reviewer YgCM (3)

Q3. Table 3 reports Push-T/Square results for the sampling strategies (e.g., CS 62.0/67.3 at 20% demonstrations) that differ from the main ReSynC results in Table 1 (69.3/77.3). Are these different trained policies, evaluation protocols, or random seeds? Please clarify the experimental setup so the incremental contribution of CS can be interpreted consistently.

A3. The two tables evaluate different trained policies under the same evaluation protocol. In Table 3, all selection strategies are applied to the base DP trained without Reciprocal Synthesis, so that only the inference-time selection varies, whereas Table 1 reports the policy trained with Reciprocal Synthesis. Within Table 3, all strategies share the same seeds, initial conditions, and pool of 16 candidates, so the columns differ only in how a chunk is selected. The incremental contribution of CS on top of Reciprocal Synthesis is shown in the component-wise analysis of Figure 3b, where adding CS raises the average success rate. We have added this setup explicitly in Section 5.2.

AuthorsOpenReview ↗

Follow-up on your concerns

Dear Reviewer,

As the discussion period draws to a close, we would like to check whether we have addressed all of your concerns satisfactorily.

We appreciate your insights and would be glad to follow up on any remaining issues. If there is any additional feedback you would like us to consider, please let us know.

Thank you for your time and effort in reviewing our paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chair,

We sincerely appreciate your time and effort in reviewing our manuscript.

As the reviewers commented, ReSynC addresses an important problem of generative control policies: compounding errors in generative imitation learning (Reviewers YgCM, ypYo). Our framework synthesizes recovery data with a backward diffusion policy and a backward dynamics model, and reuses these dynamics models at inference time for contrastive cycle-consistency selection. It was recognized as a well-motivated formulation that aligns corrective augmentation with the policy’s prediction format (All Reviewers). The analysis showing that an expressive backward policy and chunk-aligned placement work together is useful (Reviewers YgCM, pSP7). ReSynC consistently improves over the base policy on all five simulated tasks and on a real robot (Reviewers YgCM, ypYo), and ablations help identify where the gains come from (Reviewers pSP7, ypYo).

In the rebuttal, we have carefully considered the reviewers' suggestions and addressed them in our responses. As a result, we have revised the manuscript with the following analysis and experiments:

New experiments and analyses

  • Recovery after a controlled perturbation (Table 4 in Section 5.2 and Appendix D.1)
  • Held-out forward/backward dynamics errors (Table 16 in Appendix D.2)
  • Simulator validation of synthesized segments (Table 17 in Appendix D.3)
  • Predecessor-ambiguity analysis (Tables 18 and 19 in Appendix D.4)
  • CCE against actual rollout outcomes (Table 20 in Appendix E.1)
  • Evaluation selector on all tasks, including the extra positive-set medoid baseline (Section 5.2 and Table 21 in Appendix E.2)
  • Image-conditioned real-robot experiment (Table 22 in Appendix F)

Clarifications and revisions

  • Positioning relative to prior backward-model corrective augmentation and diffusion-based trajectory stitching (Section 2)
  • Role of the deterministic backward dynamics model in synthesis (Section 4.1)
  • Interpretation of the factorial study and the choice of a two-step prefix (Section 5.2)
  • Experimental setup of the sampling-strategy comparison (Table 3) and a more measured claim on the contrastive term (Section 5.2)
  • Training cost, hardware, and real-robot demonstration budgets (Table 11; Appendix A.1)
  • Expanded limitations on the validity of synthesized states and the evaluation scope (Appendix C)

In the revised manuscript, these updates are temporarily highlighted in blue\textbf{\color{blue}blue} for your convenience.

We hope that our responses and revisions address all the reviewers’ concerns. Thank you once again for your valuable contributions.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.