- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes VE-JEPA, an offline RL architecture that jointly trains an encoder through latent prediction, reward prediction, and IQL value learning. A separate policy is trained using advantage weights and acts directly on encoded observations without planning. Experiments on D4RL locomotion and AntMaze demonstrate the importance of value supervision and examine how the benefits of latent prediction and slower encoder updates vary across tasks and training budgets.
Critical strengths
-
The paper compares different encoder training objectives while continuing to train all component heads. These experiments provide useful evidence that reward prediction alone does not recover the control performance obtained when Q and V losses also update the encoder.
-
The main locomotion evaluation uses ten seeds per method and dataset, and the selected encoder learning rate is evaluated on seven additional seeds on AntMaze medium-play. Confidence intervals and analyses of sensitivity to training budget and target return selection help clarify the robustness and scope of the findings.
Critical weaknesses
-
The additional control benefit of latent prediction is not clearly established. In Table 5, the confidence intervals include zero on Hopper and HalfCheetah, while most of the improvement on Walker2d comes from one seed. It remains unclear what latent prediction contributes beyond value supervision.
-
The representation diagnostics do not sufficiently explain how Q and V supervision changes the encoder to improve control. Moreover, the findings are limited to IQL on D4RL locomotion and AntMaze, leaving their generality across other value learning objectives and tasks unclear.
Questions for authors
-
Could you compare VE-JEPA with an existing JEPA or predictive representation method under the same downstream IQL objectives and access to task rewards? The separate T-JEPA studies in Appendix D do not establish the relative benefit of the predictive objective within the VE-JEPA architecture. A comparison that controls for encoder architecture and training budget would help clarify what the proposed approach contributes beyond existing representation learning methods.
-
How do Q and V losses separately affect the information preserved by the encoder? Could you evaluate reward prediction from frozen representations when Q and V losses are allowed to update the encoder separately, while keeping all heads trained? Controlling for shared gradient clipping would help distinguish representation changes from effects on other model components.
-
Do the findings on value supervision and encoder learning rate hold with another offline RL method and on OGBench [1] single task manipulation environments? Repeating the central encoder supervision and prediction removal comparisons would help establish whether the findings extend beyond IQL and D4RL.
-
Appendix E reports an IQM of 74.7 for DC with selected target returns, compared with 74.2 for VE-JEPA at the exact 200K checkpoint. Does the reported advantage over DC persist when conditioning targets are selected using separate validation evaluations and both methods use the same checkpoints and number of evaluation episodes?
[1] Park, Seohong, et al. "Ogbench: Benchmarking offline goal-conditioned rl." International Conference on Learning Representations. 2025.