acceptodds
Under review as a conference paper at ICLR 2027

ReCycle: Training Visuomotor Diffusion Policies on Unlabeled data with Cycle-Consistency

Abstract

Visuomotor diffusion policies are trained by imitation on action-labeled demonstrations, which are orders of magnitude scarcer than action-free video. Classical control couples forward and inverse dynamics through a shared notion of action: the action that explains a transition must also produce it. Many recent diffusion policies already represent this structure: their policy, forward-dynamics, and inverse-dynamics roles are conditionals of one learned distribution, linked by a shared latent action. Yet imitation constrains these conditionals only on the demonstrations, so the denoised trajectory space is optimized for imitation accuracy alone. We constrain it to also encode action-consistent transition structure by enforcing that latent actions inferred from a transition are sufficient for forward evolution under the same policy latent space. The resulting cycle-consistency objective turns action-free video into supervision for the policy without fabricating action labels. Enforced naively on out-of-domain video, however, it degrades performance, for two reasons: unlabeled transitions may not be realizable by the robot, and the cycle gradients conflict with the supervised signal. We address the first with slot-wise latent mixup, which brings unlabeled transitions in-domain while leaving the action variable untouched, and the second with a staged schedule and batch-relative variance reweighting. The resulting recipe, ReCycle, plugs into Cosmos Policy (CP), Unified Video Action Model (UVA), and Video Policy (VP) without changing inference. Using action-free video from physics simulation, other robots, or humans, ReCycle improves CP, UVA, and VP by up to , , and  pp on LIBERO-10 and RoboCasa, and improves CP's real-robot success by  pp on average. Its gains also persist under shifted friction, contact softness, and damping. Gains are largest when the unlabeled transitions contain dynamics relevant to the target task and shrink or reverse when the transition statistics are mismatched.

Primary area:
generative models
Keywords:
World Model, Physical AI, Embodied AI, Video generation.

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_T365OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript proposes ReCycle, a self-supervised training framework that leverages the abundant action-free videos to improve video-prediction-based policies such as WAM. The core insight is to enforce cycle-consistency between inverse and forward dynamics to common video-action representation. The authors further propose a supervised warm start, slot-wise latent mixup, and batch-relative variance reweighting to stabilize training. Experiments on Cosmos Policy, UVA, and Video Policy demonstrate improvements across LIBERO, RoboCasa, and real-robot manipulation tasks.

Critical strengths
  • The paper attempts to address an critical problem: leveraging abundant action-free videos for video-prediction-based policy learning without requiring explicit pseudo-action labels.

  • The proposed cycle-consistency objective is relatively simple and easy to integrate into video-predicted policy architectures such as WAM.

  • The experimental evaluation covers multiple backbones, physical perturbations, and real-robot tasks. The ablations also highlight the importance of training stability and domain alignment.

Critical weaknesses
  • Missing key ablation. The authors use Slot-wise Latent Mixup to mitigate the domain gap between the robot dataset and the unlabeled corpus. Hence they mix the labeled data into the unlabeled. However, I didn't see the corresponding ablation in the Experiment section. It would be plausible that this specific design being fairly validated.

  • Insufficient justification of gradient-conflict mitigation and variance reweighting. The authors attribute early training degradation to gradient conflict and claim that variance reweighting reduces gradient variance, but provide no direct gradient measurements. Moreover, Eqs. (11)–(12) may yield negative effective supervision weights, whose implications remain unexplored. The warmup ablation also changes the number of cycle updates, confounding its interpretation.

  • Several inconsistencies require clarification: (1) the real-robot unlabeled data source (Sec. 4.7, L422–423 vs. App. A.3, L847–850) and training budget (L419–421 vs. L843–845) differ; (2) the parameter-free claim (Fig. 1, L072–075) conflicts with the added MLP and null token for Video Policy (App. A.10, L1131–1133); (3) the loss-weight sensitivity claim (App. A.4, L900–903) contradicts Table 11, which is also difficult to reconcile with Table 8; and (4) the RoboCasa evaluation counts differ between Sec. 4.2 (L313–316) and App. A.2.1 (L732–736).

Questions for authors
  • Did the author also optimzie IDθID_\theta? If the policy is supervised on the image latent, why the latent action part is also updated?
6 comments
AuthorsOpenReview ↗

Response to T365

We thank the reviewer for the careful reading and the precise line references. Below we address each point. All corrections will appear in the revised PDF.


W1. Ablation of slot-wise latent mixup

Our main paper and appendix already contain this ablation. We will reference it more clearly from Sec. 3.3 and Sec. 4.

  • Table 8 (loss-component ablation, RoboCasa, 24 tasks). Adding mixup on top of curriculum + Lcycle\mathcal{L}_{cycle} raises success from 62.18% → 78.50% (+16.3 pp). This is the largest single-component gain after the warm start.
  • Table 10 (mixup strength). We sweep α∈{0.05,0.2,1.0}\alpha \in \{0.05, 0.2, 1.0\} for λ∼Beta(α,α)\lambda \sim \mathrm{Beta}(\alpha,\alpha): 50.43 / 79.17 / 64.50%.

We will move a compact version of Table 10 into the main paper. We will also add a mixup-target ablation, which isolates whether the gain comes from narrowing the visual gap or from the mixed target being easier to predict:

VariantRoboCasa SR (%, 10 episodes)
No mixup62.18
Mixup with a labeled partner (ours)78.50

W2. Evidence for gradient conflict, variance reweighting, and the warmup confound

(a) Direct gradient measurements. We agree the original claim was inferred, not measured. We now report the cosine similarity between ∇θLpolicy\nabla_\theta \mathcal{L}_{policy} and ∇θLcycle\nabla_\theta \mathcal{L}_{cycle} on the shared DiT parameters, averaged over 100 batches (Robocasa):

Cycle enabled atcos(∇policy, ∇cycle), first 1k cycle itersFinal SR (%)
iter 0 (no warm start)-0.3433.21
iter 20k0.2860.12
iter 40k (ours)0.5279.17

Without a warm start, the two gradients are negatively correlated with large cycle-gradient norm. After a 40k warm start, the conflict is substantially reduced. This supports the claim in Sec. 3.3, and the figure will go into the appendix.

(b) Variance reweighting and negative weights.

We thank the reviewer for pointing this out. Lvar\mathcal{L}_{var} is a small, auxiliary component for stabilizing training. The core of ReCycle is the cycle objective together with the warm start and slot-wise mixup. Table 8 reflects this: Lvar\mathcal{L}_{var} adds only +0.67 pp (78.50 → 79.17%), while the warm start (+21.9 pp) and mixup (+16.3 pp) account for most of the improvement. We will make this clearer in the revision.

What the term does. During the cycle phase, the shared representation is also being updated by gradients from out-of-domain video. As a result, the quality of the policy's action predictions on labeled samples can vary considerably within a batch. Lvar\mathcal{L}_{var} responds to this by reweighting the labeled action loss according to each sample's current error relative to the rest of the batch. The effective per-sample weight is wi=1+λvarAiw_i = 1 + \lambda_{var} A_i, where AiA_i is the batch-standardized advantage. Because AiA_i has zero mean within each batch, the mean weight is always exactly 1. The term therefore does not change the overall scale of the supervised loss. It only shifts emphasis among samples in the batch, so that the supervised signal stays anchored to the samples the policy currently handles well while the cycle term is adapting the representation.

Negative weights. The reviewer is correct that wiw_i can become negative. With \lambda_\{var}=0.5, this happens when Ai<−2A_i < -2, i.e., for samples whose error is more than two batch standard deviations above the mean. These samples receive a small gradient-ascent term on their action loss, and our description of this as "downweighting" was inaccurate. We will correct the wording. In practice this case is rare: under approximate normality, about 2% of samples fall below −2-2. The effect is also bounded, since λvar\lambda_{var} caps its magnitude.

In the revision, we will describe Lvar\mathcal{L}_{var} as an optional stabilizer. ReCycle's contributions will be stated in terms of the cycle objective, the warm start, and mixup.

W3. Inconsistencies

We thank the reviewer for catching these. All will be fixed.

(1) Real-robot protocol (Sec. 4.7 vs. App. A.3). For the real robot experiments we used action-stripped demonstrations with 10k iteration training (as mentioned in the supplementary). We'll update this in the main paper.

(2) "No new parameters" (Fig. 1 vs. App. A.10). The claim holds for Cosmos Policy and UVA, where the inverse path is a new conditioning mask (CP) or an existing masking mode (UVA). Video Policy has no action-conditioned forward path. We therefore add an action-conditioning MLP and a learned null token . But this is the way to incorporate the cycle loss in this pipeline. We agree with the reviewer that the claim that no additional trainable parameter is not applicable for this case, but this is the way to add the cycle loss to the VP pipleine, we mentioned this in the Supplementary.

(3) Loss-weight sensitivity (App. A.4 vs. Tables 8 and 11). The text in L900–903 is wrong and will be rewritten. Table 11 shows that performance does depend on the loss weights.

(4) RoboCasa evaluation counts (Sec. 4.2 vs. App. A.2.1). We have both version. For benchmarking, we use use 50 trials per tasks for Robocasa (Tab. below). However, for ablations, we use 10 trails per task (Tab. 1) for faster evaluations.

BenchmarkEpisodes (3 x seed)CPCP + ReCycle
RoboCasa (24 tasks)24 × 50 = 1,20067.3 ± 0.369.1 ± 0.5
LIBERO-10 (10 taks)10 × 50 = 50094.3 ± 0.598.0 ± 0.3

Q1. Is a^t\hat{a}_t also optimized? Why is the latent-action part updated?

Yes. a^t\hat{a}_t has no separate parameters. It is the output of the shared denoiser fθf_\theta under the inverse mask midm_{id} (Eq. 6). Lcycle\mathcal{L}_{cycle} is computed on the image latent o^t+1\hat{o}_{t+1}, but its gradient flows (i) through the forward pass into a^t\hat{a}_t and (ii) through a^t\hat{a}_t back into θ\theta via the inverse pass.

Why update it: the policy, forward, and inverse roles share θ\theta, so the action slot is one representation used by all three. The cycle loss requires a^t\hat{a}_t to be usable as the forward model's conditioning, i.e., it must carry the information that determines the transition. The supervised loss Lpolicy\mathcal{L}_{policy} on DL\mathcal{D}_L simultaneously ties the same slot to the robot's executable action space. If a^t\hat{a}_t were held fixed, the method would reduce to forward-only prediction, which performs worse (Table 5: 70.0% vs. 76.7%).


We hope these clarifications and additional experiments address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the score.

I appreciate the author’s response and the additional results they provided, which have addressed my concerns, except my final question.

I think the authors may have misunderstood my question, as their response mainly explains basic knowledge of forward and backward propagation in deep learning. I would like to clarify it. Why the authors choose to update the action-related part when the cycle loss is only supervised on image latents, rather than freezing the action-related parts and only updating the visual prediction part. Alternatively, have the authors considered using an alternating optimization strategy for the action-related and visual-prediction parts instead of updating them jointly? What is the motivation?

If the author can properly answer this question, I would be happy to raise my rating score.

Reviewer_RCGhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ReCycle uses action-free video to train diffusion policies whose policy, forward, and inverse roles share one network. On an unlabeled frame pair, an inverse pass infers a latent action, a forward pass reconstructs the next frame from it, and the reconstruction loss is backpropagated through both passes. In Cosmos Policy this only adds an inverse conditioning mask. Since naive training hurts performance, the authors add a supervised warm start (cycle enabled only for the last 5k of 45k iterations), slot-wise latent mixup of image latents with labeled frames, and batch-relative reweighting of the action loss. Using PhyCo, LIBERO, or SSv2 video, ReCycle improves Cosmos Policy by +2.7 / +5.4 pp on LIBERO-10 / RoboCasa, UVA and Video Policy by up to +8.3 and +40.0 pp on three-task subsets, and real-robot success by +18.3 pp.

Critical strengths
  1. A simple, general way to use action-free video. It reuses the policy's own forward and inverse roles, adds no parameters for Cosmos Policy and UVA, and leaves inference unchanged.
  2. A broad evaluation: three architectures, three video sources, two benchmarks, and a real robot. The paper also reports honestly where unlabeled video hurts (stove, drawer).
Critical weaknesses
  1. The gains are not shown to come from the video.
    • Missing three-mode baseline. For Cosmos Policy, the baseline trains only the policy and world-model modes. ReCycle adds a new inverse-dynamics mode and trains it through the cycle on video at the same time. There is no comparison with regular supervised training of all three modes (policy, forward, inverse) on the labeled demos without video. Nor is the cycle applied to the demos with actions removed. The gain could therefore come from the extra auxiliary task rather than from the video.
    • Shortcut. Since one network both writes and reads the latent action, and the action slot is a full-size latent frame, the cycle can be minimized by encoding ot+1o_{t+1} in the action slot. There is no bottleneck and no test against this.
    • Regularization effect. Similar gains from very different video sources, many near-static mixup pairs at α = 0.2, and nearly the full gain after only 1k cycle iterations (Table 9) all point to a regularization effect on an already-trained policy rather than new information from video.
  2. Internal inconsistencies and limited evaluations.
    • The real-robot unlabeled data is SSv2 in Sec. 4.7 but the task's own demonstrations in App. A.3, and training is 2k vs. 10k steps.
    • The robustness claims in App. A.4 ("<1 pp", "flat plateau") contradict Tables 10–11, and the no-warm-start result differs between Tables 8 and 9.
    • Many conclusions rest on 2–3 tasks × 10 episodes.
    • UVA and VP use three hand-picked tasks with low baselines (15–42%).
Questions for authors

Clarifications:

  1. For Cosmos Policy, are both cycle passes the policy denoiser itself under different masks? How many denoising steps and which noise levels does each pass use, and do gradients flow through the full sampling chain? The Video Policy variant adds a new action-conditioning MLP, which qualifies the "no new parameters" claim.
  2. During the cycle phase, is the policy trained only through the standard imitation loss on labeled data, with no direct signal from video? Is the new inverse mode ever trained with supervision on labeled demos, or only through the cycle?
  3. Which unlabeled data and how many training steps were used for the real robot (Sec. 4.7 vs. App. A.3)?
  4. Please reconcile App. A.4 with Tables 10–11, and the 40.30% vs. 33.21% no-warm-start results.

Experiments that would change my assessment:

  1. A Cosmos Policy baseline trained with all three modes (policy, forward, inverse) supervised on the labeled demos, without video, under the same 45k-iteration budget. Also the cycle applied to the demos with actions removed, and to shuffled or unrelated video.
  2. Evidence that a^\hat a does not simply encode ot+1o_{t+1}, e.g. decoding ot+1o_{t+1} from a^\hat a, or a bottlenecked action slot.
  3. Full-suite results for UVA and VP, or an explanation of how the subsets were chosen and why the baselines are low.
4 comments
AuthorsOpenReview ↗

Response to RCGh

We thank the reviewer for a rigorous review and for stating exactly which experiments would change the assessment. We ran all of them. The results are below, followed by answers to each clarification.

Q1. Cycle passes in Cosmos Policy. Yes. Both passes are the policy denoiser fθf_\theta itself, under the inverse mask midm_{id} and the forward mask mfdm_{fd}. Each pass is a single denoising step that predicts x0x_0 from a noised input at σ sampled from CP's training noise distribution; the forward pass receives a^t\hat{a}_t clean. Gradients flow through both single-step predictions, not through a multi-step sampling chain. For UVA, a^\hat{a} is drawn with an 8-step ancestral sampler kept in the computation graph (App. A.10). On Video Policy, we agree with the reviewer: it requires an added action-conditioning MLP and null token. But this is the way to incorporate the cycle loss in this pipeline.

Q2. What trains the policy and the inverse mode during the cycle phase? The policy's action output is supervised only by Lpolicy\mathcal{L}_{policy} on labeled data. Video sends no direct action supervision. Its signal reaches the policy only through the shared parameters θ\theta. The inverse mode is trained only through the cycle and never with supervised labels.

Q3. Real robot unlabeled data and iterations For the real robot experiments we used action-stripped demonstrations with 10k iteration training (as mentioned in the supplementary). We'll update this in the main paper.

Q4. App A.4 inconsistency Thanks for pointing it out. We like to mention that in Tab. 8, no-curriculum with just Lpolicy\mathcal{L}_{policy} + 0.5 * Lcycle\mathcal{L}_{cycle} provide 40.30, whereas in Tab. 9, its Lpolicy\mathcal{L}_{policy} + Lcycle\mathcal{L}_{cycle}, which provide 33.21, so the inconsistency is due to the additional hyperparameter λcycle\lambda_{cycle}. We will mention it in the paper.

Q5. Three-mode supervised baseline and video-content controls

All rows use Cosmos Policy, the same labeled data, and a 45k-iteration budget (40k warm start + 5k). Evaluation is the full RoboCasa suite (24 tasks × [50] trials, 3 seeds).

#VariantInverse mode trained byUnlabeled dataRoboCasa-24 (%)
1CP——67.3 ± 0.7
2CP, three modes supervisedsupervised on DL\mathcal{D}_L—67.8 ± 0.7
3Cycle on action-stripped demoscycleDL\mathcal{D}_L without actions68.6 ± 0.4
4Cycle on temporally shuffled pairscycleSSv2-2k, (ot,ot+k)(o_t, o_{t+k}) from different clips67.8 ± 0.1
5ReCycle (ours)cycleSSv2-2k69.1 ± 0.5

24 tasks × 50 trials; mean ± std over 3 seeds.

Q6. does a^t\hat{a}_t simply encode ot+1o_{t+1}?

We thank the reviewer for raising this. If the action slot were a shortcut, a^t\hat{a}_t would be a compressed copy of the next frame. Such a code would (i) let ot+1o_{t+1} be recovered from a^t\hat{a}_t alone, and (ii) carry its source clip's frame into any scene it is applied to. A genuine action does neither: it describes how the current scene changes. We test both properties on held-out pairs:

1. a^t\hat{a}_t alone does not contain the next frame. An MLP decoder trained to predict ot+1o_{t+1} from a^t\hat{a}_t alone reaches 0.003 latent MSE. That is worse than simply copying oto_t (0.001), and far from the 0.0005 achieved when oto_t is also given. A frame code would make this decoding easy.

2. a^t\hat{a}_t controls the transition, not the frame. We replace each clip's a^t\hat{a}_t with one inferred from a different clip:

  • The error against the clip's own future rises 7× (0.0005 → 0.0034). The forward model therefore relies on the action and does not ignore it.
  • The prediction does not move toward the donor's frame (error 0.13 against the donor's ot+1o_{t+1}). A frame code would have reproduced it.

Together, these results show that a^t\hat{a}_t is insufficient on its own to reconstruct the next frame but necessary for predicting it from oto_t. This is the behavior of an action, not of a code for the future frame.

Q7. UVA and VP subsets

To show the generalizibility of ReCycle we choose a hard subset of LIBERO-10 (stove + moka pot, both moka pots, cream chese + butter) and RoboCasa (PnPCounterToCab, OpenDrawer, TurnOnSinkFaucet), which are mostly long-horizon tasks. These subsets are choosen so that meaningful performance gains can be shown. This is the reason why the numbers are less than reported since those are mean of both easy and hard tasks, whereas we have choosen hard tasks to show improvement.

Below we provide the full suite experiment for VP and UVA for LIBERO-10 and Robocasa using 2k SSv2 videos. Since these experiments are time-consuming, we have just shown it for SSv2, for other sources, we will provide it in the final version. We observe that adding unlabeled data using ReCycle helps for both the baselines across benchmarks.

RoboCasa (24 tasks × [50])
VP43.0 ± 0.6
VP + ReCycle (SSv2, 2k videos)47.9 ± 0.8
LIBERO-10 (10 tasks × [50])
UVA85.0 ± 0.4
UVA + ReCycle (SSv2, 2k videos)88.5 ± 0.5

We thank the reviewer again and we hope the results above address the questions.

AuthorsOpenReview ↗

Has our response addressed your concerns?

Dear Reviewer RCGh,

As the author–reviewer discussion period closes tomorrow, we wanted to check whether our response has addressed your concerns. We would be glad to clarify any points that remain open. Thanks!

AuthorsOpenReview ↗

Has our response addressed your concerns?

Dear Reviewer RCGh,

As the author–reviewer discussion period closes today, we wanted to check whether our response has addressed your concerns. We would be glad to clarify any points that remain open. Thanks!

Reviewer_UB44OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method to add inverse-forward consistency object to visuomotor policy using action-free videos. Specifically, the inverse process infers latent actions from the transition states and the forward reconstructs future observations using the same latents. During training, it combines the objective with policy learning, along with the observation-latent mixup and batch-relative action loss reweighting. The results show improvements for Cosmos policy and several other video policy methods.

Critical strengths
  • Using action-free video to update the same representation that produces robot actions is a soundness objective. It is natural to formulate an auxiliary training signal without freezing pseudo-action labels using the inverse–forward process.
  • The comparisons with pseudo-labeling and forward-only prediction, component ablations and negative transfer are useful.
Critical weaknesses
  • The cycle objective does not establish the executability or action faithfulness. In general a forward model can ignore the action when the future is predictable from the current observation, while an inverse-forward pair can also communicate through a latent encoding of the future that is not an executable robot action.
  • The cross-backbone claim requires an architecture-matched control. In the main paper it claims that the method adds no trainable parameters, Appendix A explains that Video Policy has no action-conditioned forward path and adds a learned MLP plus a learned null token. It changes the training architecture and adds labeled action conditioning.
Questions for authors

Please see above. Additionally,

  • Does the VP baseline include the same action-conditioning MLP, null token, and labeled action-conditioning training?
  • Which real-robot protocol produced Table 7: 2k steps with SSv2 or 10k steps with action-stripped task demonstrations?
4 comments
AuthorsOpenReview ↗

Response to UB44

We thank the reviewer for the positive assessment and for identifying the two failure modes of the cycle objective precisely. We address both with new experiments, and we correct the parameter claim for Video Policy.


W1. Executability and action faithfulness

We thank the reviewer for raising this. If the action slot were a shortcut, a^t\hat{a}_t would be a compressed copy of the next frame. Such a code would (i) let ot+1o_{t+1} be recovered from a^t\hat{a}_t alone, and (ii) carry its source clip's frame into any scene it is applied to. A genuine action does neither: it describes how the current scene changes. We test both properties on held-out pairs:

1. a^t\hat{a}_t alone does not contain the next frame. An MLP decoder trained to predict ot+1o_{t+1} from a^t\hat{a}_t alone reaches 0.003 latent MSE. That is worse than simply copying oto_t (0.001), and far from the 0.0005 achieved when oto_t is also given. A frame code would make this decoding easy.

2. a^t\hat{a}_t controls the transition, not the frame. We replace each clip's a^t\hat{a}_t with one inferred from a different clip:

  • The error against the clip's own future rises 7× (0.0005 → 0.0034). The forward model therefore relies on the action and does not ignore it.
  • The prediction does not move toward the donor's frame (error 0.13 against the donor's ot+1o_{t+1}). A frame code would have reproduced it.

Together, these results show that a^t\hat{a}_t is insufficient on its own to reconstruct the next frame but necessary for predicting it from oto_t. This is the behavior of an action, not of a code for the future frame.

W2/Q1. Does the VP baseline include the same action-conditioning MLP, null token, and labeled action-conditioning training?

On Video Policy, it requires an added action-conditioning MLP and null token. But this is the way to incorporate the cycle loss in this pipeline. We agree with the reviewer that the claim that no additional trainable parameter is not applicable for this case, but this is the way to add the cycle loss to the VP pipleine, we mentioned this in the Supplementary.

Q2. real-robot protocol For the real robot experiments we used action-stripped demonstrations with 10k iteration training (as mentioned in the supplementary). We'll update this in the main paper.

We thank the reviewer again. The two failure modes raised led directly to the new diagnostic experiments, which we will include in the revised paper.

AuthorsOpenReview ↗

Has our response addressed your concerns?

Dear Reviewer UB44,

As the author–reviewer discussion period closes tomorrow, we wanted to check whether our response has addressed your concerns. We would be glad to clarify any points that remain open. Thanks!

Thank the author for the detailed rebuttal and feedback. It addressed my major concerns and I keep my recommendation as Weak acceptance

AuthorsOpenReview ↗

Thank you!

Thanks for your consideration! We really appreciate it.

Reviewer_N9fsOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ReCycle incorporates action-free video into visuomotor diffusion-policy training through inverse–forward cycle consistency. The authors combine this objective with supervised warm-starting, observation-latent mixup, and loss reweighting. Experiments report improvements across three policy architectures, simulation benchmarks, and real-robot tasks.

Critical strengths
  1. This is a relevant problem. Exploiting action-free video could reduce dependence on expensive robot demonstrations.
  2. Useful training insights. The ablations show that naive cycle training hurts performance and identify scheduling and mixup as important remedies.
  3. Promising empirical gains. Many video based policies show gain on benchmarks, and real-robot tasks.
  4. Showing world modeling training can help to learn environment dynamics and in turn improves policy performance
Critical weaknesses
  1. Limited evaluation precision. Section 4.2 reports only 10 evaluation episodes per RoboCasa task (24 tasks × 10 episodes) and no explicitly stated number of rollouts per task for Libero, making individual task success rates sensitive to a small number of outcomes. Additional experiments will help to convince the reliability of the results.
  2. Visual presentation needs improvement. Small figure labels, dense tables, and uneven spacing reduce readability, more consistent formatting would help.
Questions for authors
  1. What are the Libero and RoboCasa rollout counts for each task?
  2. Can we do more simulation rollouts to add to the paper where currently the number of simulation rollouts seems limited.
  3. The appendix provides useful training details, but reproducibility would benefit from a consolidated configuration table for each backbone, including optimizer settings, learning rates, batch composition, and enabled loss components etc.
3 comments
AuthorsOpenReview ↗

Response to N9fs

We thank the reviewer for the positive assessment and the concrete suggestions on evaluation precision and reproducibility. We address each below.


W1 / Q1 / Q2. Rollout counts and evaluation precision

Thanks for the suggestion. We have provided results with more rollouts. For RoboCasa and Libero-10 benchmarking, we have used 50 episodes per tasks. For ablations, we have used 10 episodes per tasks. We observe that the trend of improving performance across tasks, sources, datasets is consistent with ReCycle. We will add these in the main paper.

BenchmarkEpisodes (3 x seed)CPCP + ReCycle
RoboCasa (24 tasks)24 × 50 = 1,20067.3 ± 0.369.1 ± 0.5
LIBERO-10 (10 taks)10 × 50 = 50094.3 ± 0.598.0 ± 0.3

W2. Visual presentation

Thanks for the sugeestion. We will update the visual presentation in the final version.


Q3. Consolidated configuration table

We will add the following table to the appendix. Code and checkpoints will be released.

Cosmos PolicyUVAVideo Policy
BackboneCosmos-Predict2 DiT, 2B (28 blocks, width 2048, Wan2.1 VAE)MAR-B (262M), KL-16 VAESVD video U-Net (width 320) + vision encoder and pose decoder
Added params0 (new mask midm_{id})0action MLP (224→1280→1280) + null token: 1.93M
OptimizerFusedAdam, β=(0.9,0.99)\beta=(0.9, 0.99), wd 0.1, ϵ=10−8\epsilon=10^{-8}, grad-clip 1.0AdamW, β=(0.9,0.95)\beta=(0.9, 0.95), wd 0.02Adam
Learning rate / schedule10−410^{-4} / 1k linear warm-up, linear decay to 0.3× over 30k, then constant 0.06×10−410^{-4} / cosine, 500 warm-up steps10−510^{-5} (vision encoder, pose decoder 5×) / constant
Total iters / TwarmT_{warm}45k / 40k15k / 10k15k / 10k
Batch (labeled NLN_L + unlabeled NUN_U, per GPU)25 + 1, grad. accum. 2 (global labeled batch 200)32 + 4 (8-frame clips)1 + 1 (25-frame clips)
GPUs4 × H1006 × H100 (warm start), 5 × H100 (cycle phase)5 × H100
Precisionbf16fp32 + EMAfp16 mixed
Unlabeled data2k PhyCo clips2k SSv2 clips2k SSv2 clips
Loss terms after warm startLpolicy\mathcal{L}_{policy}, Lvar\mathcal{L}_{var}, Lcycle\mathcal{L}_{cycle}Lpolicy\mathcal{L}_{policy}, LcycleUVA\mathcal{L}^{UVA}_{cycle}Lpolicy\mathcal{L}_{policy}, Lcycle\mathcal{L}_{cycle}
λcycle\lambda_{cycle} / λvar\lambda_{var}1.0 / 0.051.0 / 0.51.0 / 0.5
Mixup α\alpha0.20.20.2
Inverse-pass samplingsingle-step x0x_0 estimate, σ\sigma drawn from the training distribution8-step differentiable ancestralsingle-step x0x_0 estimate of the pose branch (pose input pure noise), σ\sigma from the training distribution
Inference5 denoising steps (action), 1 (future state, value), mpolm_{pol} only100 action-diffusion steps, 1 MAR iteration, temperature 0.95Euler EDM, 30 steps, linear guidance 1.0→2.5
Noise schedulehybrid EDM: log-normal (Pmean=ln⁡4P_{mean}=\ln 4, Pstd=1.2P_{std}=1.2) with a uniform high-σ\sigma part, σ∈[0.01,200]\sigma \in [0.01, 200]DDPM cosine, 1000 training stepsEDM log-normal (Pmean=1.0P_{mean}=1.0, Pstd=1.6P_{std}=1.6), σmax=700\sigma_{max}=700

We thank the reviewer again. We believe the higher-precision evaluation and the consolidated configuration table make the results more reliable and easier to reproduce.

AuthorsOpenReview ↗

Has our response addressed your concerns?

Dear Reviewer N9fs,

As the author–reviewer discussion period closes tomorrow, we wanted to check whether our response has addressed your concerns. We would be glad to clarify any points that remain open. Thanks!

AuthorsOpenReview ↗

Has our response addressed your concerns?

Dear Reviewer N9fs,

As the author–reviewer discussion period closes today, we wanted to check whether our response has addressed your concerns. We would be glad to clarify any points that remain open. Thanks!

Comments on the paper

3 comments
AuthorsOpenReview ↗

General comments for reviewers and AC

We thank all reviewers for their constructive and detailed feedback. Below we summarize the strengths the reviewers highlighted and the changes we made in response (added in blue in the updated paper). Detailed answers are in the individual responses.

Strengths noted by reviewers

  • An important problem. Using abundant action-free video to reduce dependence on expensive robot demonstrations is relevant and critical (N9fs, UB44, T365).
  • A simple, general, and natural method. Using action-free video to update the same representation that produces robot actions is a sound objective. It avoids freezing pseudo-action labels (UB44), reuses the policy's own forward and inverse roles, adds no parameters for Cosmos Policy and UVA, and leaves inference unchanged (RCGh). It is easy to integrate into video-prediction policies (T365).
  • Broad evaluation. Three architectures, three video sources, two benchmarks, physical perturbations, and a real robot (RCGh, T365, N9fs).
  • Useful ablations and training insights. The paper shows that naive cycle training hurts, and identifies scheduling and mixup as remedies (N9fs, T365). The comparisons with pseudo-labeling and forward-only prediction are useful (UB44). World-model training helps the policy learn environment dynamics (N9fs).
  • Honest reporting of negative transfer. Cases where unlabeled video hurts (stove, drawer) are reported and analyzed (RCGh, UB44).

Summary of changes

1. Higher-precision evaluation (N9fs, RCGh, T365). We re-evaluated all main results with 50 episodes per task over 3 seeds:

BackboneBenchmarkBaseline+ ReCycle
Cosmos PolicyRoboCasa (24 tasks)67.3 ± 0.369.1 ± 0.5
Cosmos PolicyLIBERO-1094.3 ± 0.598.0 ± 0.3
UVA (full suite)LIBERO-1085.0 ± 0.488.5 ± 0.5
Video Policy (full suite)RoboCasa (24 tasks)43.0 ± 0.647.9 ± 0.8

ReCycle improves every backbone on the full suites. The revised paper will report these numbers as the main results, and every table will state its number of episodes and seeds.

2. Controls isolating the role of the video (RCGh). We added a Cosmos Policy baseline with all three modes supervised on labeled data, the cycle applied to action-stripped demonstrations, and the cycle applied to temporally shuffled video, all under the same 45k-iteration budget. ReCycle on real video gives the best result among these variants.

3. Tests that the latent action is not a shortcut (RCGh, UB44). A decoder given only a^t\hat{a}_{t} cannot reconstruct ot+1o_{t+1} (worse than copying oto_t). Swapping a^t\hat{a}_{t} between clips raises the forward error 7× without reproducing the donor's frame. a^t\hat{a}_{t} thus behaves as an action that the forward model relies on, not as a code for the next frame.

4. Direct gradient measurements (T365). The cosine similarity between policy and cycle gradients is negative without a warm start (−0.34) and positive after the 40k warm start (0.52). This directly supports the gradient-conflict explanation.

5. Corrections and clarifications (all reviewers).

  • Real-robot protocol: action-stripped task demonstrations, 10k iterations. The main text will be corrected to match.
  • "No new parameters": restricted to Cosmos Policy and UVA. Video Policy requires an action-conditioning MLP and null token (1.93M parameters), now stated in the main text.
  • App. A.4: the sensitivity text will be rewritten to match Tables 10–11. The 40.30% vs. 33.21% difference is explained by different λcycle\lambda_{cycle} values.
  • Evaluation counts: 50 trials per task for benchmarks and 10 for ablations, now stated consistently.
  • Lvar\mathcal{L}_{var}: described as a minor, optional stabilizer. Its negative-weight case is now discussed explicitly, and the "downweighting" wording is corrected.
  • UVA/VP subsets: the selection rationale is explained, and full-suite results have been added (above).

6. Reproducibility and presentation (N9fs). We added a consolidated configuration table for each backbone (optimizer, learning-rate schedule, batch composition, loss terms, sampling, noise schedule) and will release code and checkpoints. Figures and tables will be reformatted for readability.

We thank the reviewers again. Their feedback has substantially strengthened the paper.

Area_Chair_XiBxOpenReview ↗

Discussion with the authors

Dear reviewers,

the authors have provided responses for your reviews. Can you please let them if you comments have been addressed?

Thank you, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.