- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
The work explores two reasons why joint-embedding predictive architectures (JEPAs) fail in stochastic environments. The first is that the objective does not incentivize the encoder to keep information relevant to decision making, and the second is that chaining multiple one-step predictions of the next state does not produce the expected future state. The authors propose an alternative method that trains an encoder with a reconstruction objective and trains predictions of the next latent state with flow matching rather than regression. This method is compared to two JEPA variants in Craftax and a set of custom visual control environments. The latter environment offers several variants with differing types of stochasticity. The results show that the proposed model outperforms the JEPA variants, with the biggest difference occurring in most stochastic configurations.
Critical strengths
1. The custom environment with its stochastic variants is appropriate given the phenomenon the paper aims to test. The simplicity aids in interpreting the results.
Critical weaknesses
1. It is not clear what research question, if any, is being answered. The two mechanisms presented in Section 2 are largely known. The problem of the objective not incentivizing the encoder to keep information relevant to decision making is clear from the objective and does not require theory or experiments to verify. The issue with expectation models is also well understood both in work cited by the paper (Wan et al., 2019; Grimm et al., 2021), and by other work like that by Antonoglou et al. (2022) on stochastic MuZero. Diagnostic papers can be a great service to the community, but only when it is clear what they aim to diagnose. The work presents many experimental results and is densely packed with interpretations of the various pieces of each experiment. However, without knowing what question the authors’ aim to answer, I fail to find the purpose of the experiments and significance of the findings is unclear to me.
2. Experiments lack rigor and contain confounders. The most egregious example is that the flow matching method backpropagates gradients from the reward head through the next state prediction module, while both JEPA variants do not. This gives the flow matching method a reason to maintain information relevant to decision making that the JEPA models lack. This difference alone could account for why the flow matching model outperforms both JEPA variants, invalidating several of the other explanations proposed to explain the difference in performance.
The above two are my primary critiques, but the manuscript contains many more issues. The writing is hard to follow and often using pronouns with unclear antecedents, obscuring conclusions. Details essential for interpreting experiments (like how the readout in Test 1 is performed) are all relegated to the Appendix. The work draws strong conclusions from three seeds.
Questions for authors
Is there a central research question that unites the findings as part of a shared story, and that survives the confounding factor I mention in weakness 2?
2 comments
Thank you for clarifying the research question and running additional experiments to address the confounder I raised. The revision is an improvement, and I have updated my presentation, soundness, and overall scores from 1 to 2. However, I still have substantial concerns.
Confound: I appreciate the new controls, but the comparison between a JEPA without task-head gradients and a flow model with them remains a headline result. The revision shows that task-head gradients largely prevent the JEPA encoder from suppressing task-relevant information, yet this suppression remains a central part of the paper's argument.
Motivation and significance: I now understand the research question, but I remain unclear about its significance. Once task supervision addresses the representation issue, the main remaining failure appears to be in the transition predictor. We already know that expectation models cannot accurately model multiple possible outcomes, and that generative models can. While the paper provides additional analysis of these failures, their significance is unclear. With the question of retaining task-relevant information largely resolved by reward head gradients, it is unclear why JEPA is central to the paper at all. The main finding seems to concern the known limitations of expectation models, rather than anything specific to JEPAs.
It has not affected my score, but I also find the authors' excessive use of AI, and the resulting convoluted prose, very frustrating. I have no objection to using AI, but it is the authors' responsibility to ensure that the final text is clear and accessible to human readers.
Response to Reviewer r2wx
You asked whether one research question unites the findings and whether it survives the reward-head confound. The submitted version did not state the question clearly or isolate the effect of task-head supervision. The revision addresses both concerns.
The question. Can an agent learn, inside a JEPA trained end to end, to handle an event that it cannot predict? If it cannot, is the fault in what the encoder keeps or in what the predictor forecasts? The revision is organized around this: the test (Section 2), the encoder (Section 3), the predictor (Section 4) and the repair (Section 5).
We agree that both mechanisms are known in principle. That the objective does not ask for decision-relevant state can be read off the objective, and expectation models are well understood. The related work now says this and cites Antonoglou et al. (2022) with Wan et al. and Grimm et al. What cannot be read off the objective is how a trained model behaves, and that is what the paper measures. The LeWorldModel encoder does see the wall, and the gradient through the prediction target shrinks the wall's weight in the latent by more than a thousand times. Stopping that gradient restores it. With background noise the encoder outputs noise at a variance that we predict without a free parameter. Inverse dynamics, task heads, a teacher and a stopped target all leave the agent unable to survive the wall, and on the fixed schedule only reversing both the learned target and the regression predictor fixes it. The failed models pass a collapse check and reach a low prediction loss, so we propose the agent test as the diagnostic.
The confound. The flow model's reward and termination heads did send gradient into its dynamics and LeWorldModel's did not, and that could have explained the gap. We tested it from both sides. A JEPA trained jointly with the same heads keeps the wall in its encoder (obstacle weight near 0.3, against 0.0001 without them), and its agents still survive 1% on the fixed schedule and no more than 5% in any obstacle world. A flow model whose heads send no gradient to the dynamics still survives 88% on the fixed schedule and 70% with random arrivals (Table 1).
The head gradient does matter in one place. With background noise or a background texture the flow model fails without it, and the paper states this in its limitations. We read it as support for the conclusion that relevance has to come from the task. The same supervision does not rescue the JEPA. With a stopped target, inverse dynamics and those heads, a policy trained on real transitions through the JEPA's encoder survives 73 to 99% in the four obstacle worlds, and agents trained inside the model stay below 9%. What is left is the regression predictor.
Writing, readouts, seeds. The main text was rewritten as one argument, and each readout is defined where it is first used (Sections 3.1 and 3.2). Table 1 reports means and 95% intervals over independently fitted world models. With three fits the intervals are wide and the caption says so. Despite the wide intervals, the separation in mean control performance is large: in the obstacle worlds all 81 JEPA fits remain below 9%, while the reconstruction model's means range from 72% to 94%. Section 4 no longer says that agents exploit the model. It reports what we measured: the regression models predict 14% deaths for their own policies and 44% occur, where the flow-matching models predict 14% and 16% occur.