- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Juno, a framework that integrates an action-conditioned JEPA world model into the pretraining, policy learning, and deployment adaptation of VLA models. Juno introduces a Dynamic CLS loss to transfer local motion changes into a global representation, fuses patch features from the current frame, and uses a separate predictive branch to align with future latent states. During deployment, Juno-TTT first updates the world model using valid trajectories, then freezes the adapted teacher and updates the policy using only verified successful trajectories. In the experiments section, Juno achieves average success rates of 68.5% on SimplerEnv and 59.6% on RoboCasa, compared with 60.9% and 47.8% for the base model. Real-robot results further demonstrate generalization and adaptation to several environmental changes.
Critical strengths
- The method is comprehensive, with clearly connected components. The paper organizes its method around three challenges: aligning representations with control, coordinating prediction with action learning, and adapting the model after deployment. The JEPA module provides visual features, future-state supervision, and a dynamics learning objective at different stages, resulting in a coherent design. Dynamic CLS also preserves a compact global-state interface, facilitating integration with existing VLA models.
- The Juno outperforms the base model on both simulation benchmarks, improving performance on 21 of the 24 RoboCasa tasks. In real-robot experiments, the frozen policy maintains success rates of 70%–75% under cumulative changes to the background, height, and object. Component ablations further indicate that feature fusion and future-state alignment jointly contribute to the final performance.
- Juno-TTT makes use of both successful and failed trajectories. During deployment, all valid trajectories help update the world model, while only successful ones are used to update the policy. This allows the model to learn from failures without copying failed actions.
Critical weaknesses
- The paper does not separately test the contribution of updating the world model or using failed trajectories. It also does not compare Juno-TTT with simply continuing action imitation on the same successful trajectories. Thus, it is unclear how much of the improvement comes from adapting the dynamics model and how much comes from further training on successful data. This weakens the evidence for updating the world model before the policy.
- Explicit tests of distribution shifts are mainly conducted on a real robot; comparable simulation tests of shifts during deployment are lacking. The real-robot experiments also focus mainly on placing an object into a bowl, with only 20 trials per condition. Under dynamic lighting, the increase from 55% to 70% amounts to three additional successes. Without results from multiple independent training seeds or uncertainty estimates, it is difficult to assess the stability of smaller gains and how broadly the adaptation method applies.
- The second stage relies on verified successful trajectories, but the paper does not state whether success is judged by a human, task-specific rules, or a detector. It also does not describe how invalid segments are removed in enough detail. If all deployment trajectories fail, the second stage has no successful data for action imitation, and the paper does not explain how it handles this case. These details affect the method’s autonomy, deployment cost, and reproducibility.
- The effect of Dynamic CLS is not fully explained. Its loss gives more weight to patches with larger visual changes, but such changes may come from lighting or background motion rather than action-relevant objects. The ablation shows an overall performance gain from Dynamic CLS, but does not establish that the learned representation captures the intended control-relevant dynamics. A direct analysis of which patches receive high weights and how CLS changes relate to actions would strengthen this claim.
Questions for authors
Refer to the Weakness
2 comments
W3. Whether success is judged by a human, task-specific rules, or a detector / How invalid segments are removed / If all deployment trajectories fail
A3. Success is determined by the simulator in SimplerEnv and by a human observer on the real robot.
- Labels and filtering. On the robot, the observer marks success when the designated object is placed in the designated bowl without intervention. A tracking fault requiring an operator reset counts as a failure (Appendix C, real-world protocol). For dynamics training, a usable transition contains an observation, the executed action, and its next observation. In SimplerEnv we trim trailing idle actions while retaining aligned observations. Valid transitions from unsuccessful attempts remain available to Stage A.
- No successful episodes. Thank you for pointing out this limitation of Juno-TTT. If all deployment episodes fail, Stage A can still update the world model from the observed consequences of the robot’s actions. Stage B requires successful episodes to provide a suitable alignment signal for the policy, so its update cannot proceed. However, none of the VLA studies we reviewed has demonstrated that test-time parameter updates using only failed deployment episodes can improve success on the original task without additional demonstrations or human corrections. Your question is an important problem, and we will investigate it in future work, thank you.
W4. Which patches receive high weights and how CLS changes relate to actions
A4. We compared where LeWM and V-JEPA 2 patch features change between frames (Fig. 2a, p. 3). We then examined a ten-frame sequence in which the gripper first moves right and then closes. For each pair of consecutive frames, we computed the change in CLS and compared the directions of the nine resulting vectors ().
- Patch weights. In Fig. 2a, LeWM feature changes concentrate around the gripper–object interaction, while V-JEPA 2 responds more diffusely. The figure visualizes temporal cosine distance. Dynamic CLS assigns higher weights to patches with larger L2 feature changes (Eq. 3), so lighting or background motion can also receive high weight.
- CLS changes. In Fig. 6, CLS changes point in more similar directions within each motion than across the two motions. In this sequence, the direction of change in the global CLS token distinguishes gripper translation from closing.
Downstream effect. Using Dynamic CLS instead of standard LeWM CLS improves average success by 2.9 percentage points on SimplerEnv and 4.9 on RoboCasa-GR1 ().
| Teacher / target | Avg. | Put Spoon on Towel | Put Carrot on Plate | Stack Green Block on Yellow Block | Put Eggplant in Yellow Basket |
|---|---|---|---|---|---|
| V-JEPA 2 | 63.5 | 67.7 | 52.1 | 35.4 | 99.0 |
| LeWM (patch mean) | 62.2 | 81.3 | 60.4 | 8.3 | 99.0 |
| LeWM (CLS) | 65.6 | 76.0 | 61.5 | 28.1 | 96.9 |
| LeWM (CLS) + Dynamic CLS (ours) | 68.5 | 86.5 | 59.4 | 31.3 | 96.9 |
| Teacher / target | Avg. | PnP + Close | PnP From Cuttingboard | PnP From Placemat | PnP From Plate | PnP From Tray |
|---|---|---|---|---|---|---|
| LeWM (CLS) | 54.7 | 52.7 | 58.8 | 53.0 | 59.5 | 50.4 |
| LeWM (CLS) + Dynamic CLS (ours) | 59.6 | 57.3 | 64.8 | 55.5 | 68.0 | 53.6 |
Thank you again for raising these points. They led us to separate the effect of updating the world model from that of continued imitation.
Thank you for the detailed comments. The additional experiments are now in the revised manuscript, with new material shown in blue. We answer your questions in turn.
W1. The contribution of updating the world model or using failed trajectories / Continuing action imitation on the same successful trajectories
A1. We added two ablations on SimplerEnv. First, we used the same successful episodes to compare three policy updates: action imitation alone, alignment to the original teacher, and alignment after updating the teacher (Table 12). Second, we kept Stage B fixed to successful episodes and varied the episode types used in Stage A, matching the amount of data across conditions (Table 11).
Updating the teacher (revised Appendix D.3, p. 23, Table 12)
Selecting episodes by stage (revised Appendix D.3, p. 23, Table 11)
W2. Comparable simulation tests of shifts during deployment are lacking
A2. Our simulation and real-robot experiments cover different shifts.