- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses information allocation in compact JEPA world models by separating predictive latent from learned-query residual-context embeddings. Action-conditioned prediction and SIGReg regularization shape the predictive latent, while temporal context regularization encourages the residual-context embeddings to capture persistent information. A joint reconstruction decoder provides additional supervision to both streams. Evaluation includes closed-loop planning in four simulated environments. The reported results show improved performance over a compact LWM baseline while retaining a similar computational cost.
Critical strengths
-
The method improves performance while preserving compact latent-space planning. With the same number of parameters, LRC-JEPA outperform the baseline in four experiment environments.
-
The paper provides direct evidence about information allocation. The physical-state probes and cross-trajectory context replacement support different functional roles for and . The PCA-compression and static-pattern perturbation experiments additionally test properties relevant to compactness and nuisance robustness. These analyses provide meaningful support for functional specialization beyond task-performance improvements.
Critical weaknesses
-
The methodological advance over prior separation mechanisms is insufficiently explained. ContextWM [1] already separates context from latent dynamics, combines them through cross-attention reconstruction, and uses the dynamic state for control. Iso-Dream [2] also evaluates a reconstruction-only auxiliary branch that is discarded during imagination and deployment in its distractor setting. LRC-JEPA differs through its learned-query context, temporal regularization, and reward-free JEPA planning formulation. However, the paper does not clearly explain the methodological advances beyond the established separation principle.
-
The planning gains are not sufficiently isolated as a consequence of the proposed functional separation. The comparison with LWM simultaneously introduces reconstruction supervision, a context branch, and temporal context regularization. Matching the active encoder therefore does not isolate disentanglement as the source of improvement.
[1] Wu et al., Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning. NeurIPS, 2023. [2] Pan et al., Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models. NeurIPS, 2022.
Questions for authors
-
What specific limitation of prior separation mechanisms does LRC-JEPA address, and how do its architectural and training changes resolve such limitation in the reward-free JEPA?
-
How does LRC-JEPA compare with a LWM augmented with comparable reconstruction supervision and training resources?