- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This manuscript proposes ReCycle, a self-supervised training framework that leverages the abundant action-free videos to improve video-prediction-based policies such as WAM. The core insight is to enforce cycle-consistency between inverse and forward dynamics to common video-action representation. The authors further propose a supervised warm start, slot-wise latent mixup, and batch-relative variance reweighting to stabilize training. Experiments on Cosmos Policy, UVA, and Video Policy demonstrate improvements across LIBERO, RoboCasa, and real-robot manipulation tasks.
Critical strengths
-
The paper attempts to address an critical problem: leveraging abundant action-free videos for video-prediction-based policy learning without requiring explicit pseudo-action labels.
-
The proposed cycle-consistency objective is relatively simple and easy to integrate into video-predicted policy architectures such as WAM.
-
The experimental evaluation covers multiple backbones, physical perturbations, and real-robot tasks. The ablations also highlight the importance of training stability and domain alignment.
Critical weaknesses
-
Missing key ablation. The authors use Slot-wise Latent Mixup to mitigate the domain gap between the robot dataset and the unlabeled corpus. Hence they mix the labeled data into the unlabeled. However, I didn't see the corresponding ablation in the Experiment section. It would be plausible that this specific design being fairly validated.
-
Insufficient justification of gradient-conflict mitigation and variance reweighting. The authors attribute early training degradation to gradient conflict and claim that variance reweighting reduces gradient variance, but provide no direct gradient measurements. Moreover, Eqs. (11)–(12) may yield negative effective supervision weights, whose implications remain unexplored. The warmup ablation also changes the number of cycle updates, confounding its interpretation.
-
Several inconsistencies require clarification: (1) the real-robot unlabeled data source (Sec. 4.7, L422–423 vs. App. A.3, L847–850) and training budget (L419–421 vs. L843–845) differ; (2) the parameter-free claim (Fig. 1, L072–075) conflicts with the added MLP and null token for Video Policy (App. A.10, L1131–1133); (3) the loss-weight sensitivity claim (App. A.4, L900–903) contradicts Table 11, which is also difficult to reconcile with Table 8; and (4) the RoboCasa evaluation counts differ between Sec. 4.2 (L313–316) and App. A.2.1 (L732–736).
Questions for authors
- Did the author also optimzie ? If the policy is supervised on the image latent, why the latent action part is also updated?
6 comments
part 2
W3. Inconsistencies
We thank the reviewer for catching these. All will be fixed.
(1) Real-robot protocol (Sec. 4.7 vs. App. A.3). For the real robot experiments we used action-stripped demonstrations with 10k iteration training (as mentioned in the supplementary). We'll update this in the main paper.
(2) "No new parameters" (Fig. 1 vs. App. A.10). The claim holds for Cosmos Policy and UVA, where the inverse path is a new conditioning mask (CP) or an existing masking mode (UVA). Video Policy has no action-conditioned forward path. We therefore add an action-conditioning MLP and a learned null token . But this is the way to incorporate the cycle loss in this pipeline. We agree with the reviewer that the claim that no additional trainable parameter is not applicable for this case, but this is the way to add the cycle loss to the VP pipleine, we mentioned this in the Supplementary.
(3) Loss-weight sensitivity (App. A.4 vs. Tables 8 and 11). The text in L900–903 is wrong and will be rewritten. Table 11 shows that performance does depend on the loss weights.
(4) RoboCasa evaluation counts (Sec. 4.2 vs. App. A.2.1). We have both version. For benchmarking, we use use 50 trials per tasks for Robocasa (Tab. below). However, for ablations, we use 10 trails per task (Tab. 1) for faster evaluations.
| Benchmark | Episodes (3 x seed) | CP | CP + ReCycle |
|---|---|---|---|
| RoboCasa (24 tasks) | 24 × 50 = 1,200 | 67.3 ± 0.3 | 69.1 ± 0.5 |
| LIBERO-10 (10 taks) | 10 × 50 = 500 | 94.3 ± 0.5 | 98.0 ± 0.3 |
Q1. Is also optimized? Why is the latent-action part updated?
Yes. has no separate parameters. It is the output of the shared denoiser under the inverse mask (Eq. 6). is computed on the image latent , but its gradient flows (i) through the forward pass into and (ii) through back into via the inverse pass.
Why update it: the policy, forward, and inverse roles share , so the action slot is one representation used by all three. The cycle loss requires to be usable as the forward model's conditioning, i.e., it must carry the information that determines the transition. The supervised loss on simultaneously ties the same slot to the robot's executable action space. If were held fixed, the method would reduce to forward-only prediction, which performs worse (Table 5: 70.0% vs. 76.7%).
We hope these clarifications and additional experiments address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the score.
I appreciate the author’s response and the additional results they provided, which have addressed my concerns, except my final question.
I think the authors may have misunderstood my question, as their response mainly explains basic knowledge of forward and backward propagation in deep learning. I would like to clarify it. Why the authors choose to update the action-related part when the cycle loss is only supervised on image latents, rather than freezing the action-related parts and only updating the visual prediction part. Alternatively, have the authors considered using an alternating optimization strategy for the action-related and visual-prediction parts instead of updating them jointly? What is the motivation?
If the author can properly answer this question, I would be happy to raise my rating score.
Response to T365
We thank the reviewer for the careful reading and the precise line references. Below we address each point. All corrections will appear in the revised PDF.
W1. Ablation of slot-wise latent mixup
Our main paper and appendix already contain this ablation. We will reference it more clearly from Sec. 3.3 and Sec. 4.
We will move a compact version of Table 10 into the main paper. We will also add a mixup-target ablation, which isolates whether the gain comes from narrowing the visual gap or from the mixed target being easier to predict:
W2. Evidence for gradient conflict, variance reweighting, and the warmup confound
(a) Direct gradient measurements. We agree the original claim was inferred, not measured. We now report the cosine similarity between ∇θLpolicy and ∇θLcycle on the shared DiT parameters, averaged over 100 batches (Robocasa):
Without a warm start, the two gradients are negatively correlated with large cycle-gradient norm. After a 40k warm start, the conflict is substantially reduced. This supports the claim in Sec. 3.3, and the figure will go into the appendix.
(b) Variance reweighting and negative weights.
We thank the reviewer for pointing this out. Lvar is a small, auxiliary component for stabilizing training. The core of ReCycle is the cycle objective together with the warm start and slot-wise mixup. Table 8 reflects this: Lvar adds only +0.67 pp (78.50 → 79.17%), while the warm start (+21.9 pp) and mixup (+16.3 pp) account for most of the improvement. We will make this clearer in the revision.
What the term does. During the cycle phase, the shared representation is also being updated by gradients from out-of-domain video. As a result, the quality of the policy's action predictions on labeled samples can vary considerably within a batch. Lvar responds to this by reweighting the labeled action loss according to each sample's current error relative to the rest of the batch. The effective per-sample weight is wi=1+λvarAi, where Ai is the batch-standardized advantage. Because Ai has zero mean within each batch, the mean weight is always exactly 1. The term therefore does not change the overall scale of the supervised loss. It only shifts emphasis among samples in the batch, so that the supervised signal stays anchored to the samples the policy currently handles well while the cycle term is adapting the representation.
Negative weights. The reviewer is correct that wi can become negative. With \lambda_\{var}=0.5, this happens when Ai<−2, i.e., for samples whose error is more than two batch standard deviations above the mean. These samples receive a small gradient-ascent term on their action loss, and our description of this as "downweighting" was inaccurate. We will correct the wording. In practice this case is rare: under approximate normality, about 2% of samples fall below −2. The effect is also bounded, since λvar caps its magnitude.
In the revision, we will describe Lvar as an optional stabilizer. ReCycle's contributions will be stated in terms of the cycle objective, the warm start, and mixup.