- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes to learn an energy-based model to quantify the distance between latent states in JEPA-style world models. They show that when this model is used for planning, it enables achieving goals even when the dynamics model is trained with data collected with random actions. There is a theoretical analysis of the proposed approach and the model is evaluated on several environments, with a focus on navigation tasks.
Critical strengths
- Theoretical analysis: while I found the paper hard to follow, there is a theoretical motivation that helps to understand what the authors try to achieve a bit better.
- Diverse environments.
- The main insight that an EBM that quantifies the distance to goal can help planning when the dynamics are learned from random data is interesting (though not entirely convincing).
- Seemingly better results on data collected with random actions.
- Extensive appendix.
Critical weaknesses
- Writing: the paper, starting from the abstract, was not easy to follow. The abstract is just the culmination of sentences that do not provide motivation or informative descriptions of the proposed method. The paper does not have a very coherent storyline. A lot of terms are used without explanation, for example (L42): “capture regional connectivity before mixing erases these distinctions.” What is regional connectivity? What does mixing refer to? L88: “positive prediction horizons”. What are positive prediction horizons?. This trend continues throughout the entire paper. My impression is that I’m reading a very concise output of an LLM. In addition, the sections are probably organized in the wrong order, as the method section mentions a lot of terms that are found later, e.g., “the free goal head” (L114). Lines 160-161: sentence is cut half-way through, no connector to the following one.
- All implementation details are deferred to the appendix for some reason, so the main text jumps from theory to experiment without any practical connection.
- Notations: use the same parameters . Does it mean they are the same network with different outputs or are they different networks? Unclear.
- “Exploration”: I am not sure I understand what this paper has to do with exploration. There is no interaction with the environment, the proposed approach learns purely from offline data that might be collected with random actions, and learns an EBM to give an estimation for the distance from the goal for planning. To my understanding, the authors try to train the baselines with random data (that means that for LeWM for example, the dynamics model is trained random transitions) and the results show that they are not usable for planning. So the insight here is that if the dynamics model does not cover a wide range of behaviors, it is not usable for training, which is not surprising. I’m not convinced that the additional EBM overcomes that coverage issue.
- The related work section in the main text is very lacking and barely covers existing exploration methods.
Questions for authors
- My main concern is that if I understood the paper, then the novelty here is very limited, and the pompous theoretical analysis around it does not really help the case. I admit that perhaps I did not understand the paper very well as I found the writing very hard to follow. I believe the paper would benefit from a significant writing revision.
4 comments
W4: Clarity, notation, and implementation
We agree that several concepts and implementation details were insufficiently explained, and have made targeted revisions to address the specific issues raised.
Clarity and organization. The Introduction now explains "regional connectivity" as the existence of feasible paths between regions, and "mixing" as the loss of dependence on the starting location as a random walk approaches its stationary distribution. Section 2.1 now defines the horizon as the number of steps between observations. We have also corrected the incomplete sentence at L160–161.
Notation. Section 2.2 now explicitly states that and share a backbone with source and target heads. The heads are tied for navigation and separate for manipulation.
Implementation. The learning objective and planning procedure are described in Section 2, with detailed implementation settings provided in Appendix D. We have clarified the connection between these components in the main text.
Related work. Section 5 now distinguishes our approach from JEPA-style predictive learning and exploration-policy methods, with extended discussions in Appendix A.
We sincerely appreciate the reviewer's feedback, which has helped us improve the paper. We hope these clarifications and revisions address the concerns, and we would welcome any further questions.
I don't believe this paper is ready for publication
I thank the authors for their effort during the rebuttal period. I went over the revised paper and the other reviews and responses as well. My impression has, unfortunately, not changed - I find the paper barely readable and my impression is that it is mostly LLM-generated (apologies if not, but this is my impression). I also don't understand how the first part of the response is related to my review. The seemingly better results do not justify publication as writing is a major part of how we communicate findings and propose ideas. From my understanding, the novelty here is very limited and the usage of "exploration" is quite misleading. I'll keep my score.
We thank the reviewer for revisiting the revised manuscript. We have taken the writing concerns seriously and made concrete revisions to the terminology, notation, organization, and implementation details.
Our previous response directly addressed the reviewer's concerns about exploration and novelty. We would like to restate two central points to make this connection more explicit.
1. Random exploration is the source of experience, not the policy we aim to learn.
Our central question is how goal-free exploratory experience can be converted into reusable knowledge for goal-directed planning. We do not propose an exploration policy or claim to overcome insufficient data coverage. The fact that learning is performed offline does not change the exploratory nature of the experience from which the data were collected.
More importantly, random trajectories are not merely coverage data. Although actions are random, the resulting trajectories are constrained by the environment. Their multi-horizon temporal statistics encode information about the underlying geometry and connectivity. Our matched-data experiments show that learning and using this temporal structure matters even when all methods train with the same data.
2. Our method learns multi-horizon temporal relations, rather than directly fitting goal distance.
We learn horizon-conditioned temporal relations through NCE, without action labels or shortest-path supervision. Our theoretical analysis characterizes how different horizons capture complementary geometric and connectivity information, motivating their use in planning.
We further develop planning algorithms that leverage these learned multi-horizon temporal relations. They combine long-range guidance from the temporal model with one-step dynamics for local action selection, without requiring multistep dynamics rollouts.
These distinctions directly address the reviewer's original concerns about the characterization and novelty of our method. Our contribution lies in learning, theoretically characterizing, and developing planning algorithms that utilize multi-horizon temporal structure from random exploratory experience.
Finally, regarding the impression that the manuscript is mostly LLM-generated, we respectfully disagree. We have addressed the specific writing concerns through concrete revisions. We believe the manuscript should be evaluated based on its clarity, technical formulation, theoretical analysis, and empirical evidence.
Consider an agent randomly walking through a maze without knowing where its future goal will be. Although its actions are random, its movements are constrained by the environment, so the resulting trajectories reveal how different locations are connected.
We do not learn the random policy or directly fit a goal-distance function. Instead, we learn multi-horizon temporal relations from observation pairs (xt,xt+τ), estimating how likely one observation is to follow another after τ steps relative to its marginal frequency:
Gθ(x,y,τ)≈logp0(y)pdata(y∣x,τ).
Our conditional EBM learns these relations through NCE, without action labels or shortest-path supervision. Our theoretical analysis further explains how these temporal statistics relate to geodesic geometry and regional connectivity.
Different horizons reveal different structures. Proposition 1 shows that short-horizon temporal scores recover geodesic geometry in the diffusion limit, while Proposition 2 characterizes the spectral structure of longer-horizon scores, explaining how slowly decaying modes preserve information about connectivity between regions. At sufficiently long horizons, however, the random walk approaches its stationary distribution, making the temporal relations increasingly independent of the starting location. This theoretical analysis explains why different horizons capture complementary spatial information and motivates multi-horizon planning. Fig. 1 illustrates these properties:
W2: How does this enable planning?
Once a goal is specified, the agent needs to decide which action moves it closer to that goal. We separate this into two components:
The agent selects an action, executes it, and replans. Unlike dynamics-based world-model planners, our method does not require multistep dynamics rollouts. Long-range guidance is directly provided by the learned temporal relations, while dynamics prediction is needed only for local action selection.
Importantly, our contribution is not simply adding an EBM as a goal-distance metric. We learn horizon-conditioned temporal statistics directly from experience, rather than fitting a supervised distance function. These learned relations provide long-range planning guidance without explicitly predicting intermediate states.
W3: Why random exploration?
We agree that our use of "exploration" may have been unclear. We do not propose or learn an exploration policy. Rather, we use exploration in the behavioral sense: collecting experience without a task-specific goal or reward, with random walks being the simplest example. The resulting trajectories are used for offline learning. Our question is whether such goal-free experience can later support goal-directed planning. We have clarified this distinction in Section 2.1 and expanded the discussion of exploration-policy methods in Section 5.
We also agree that sufficient data coverage is necessary, and do not claim that our method overcomes missing coverage. However, coverage alone does not guarantee effective long-range planning. To examine this, we train all methods on the same 4M-step random-walk dataset, which visits every free cell:
Despite using the same exploratory data, the baselines struggle on Giant, while our method achieves substantially higher success. This suggests that how temporal structure is learned from the data matters beyond state coverage alone.