- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes InJepa, a latent-state planning architecture for image-goal navigation. A goal-conditioned Proposal JEPA generates candidate desired states, an action model translates state differences into actions, and a Forward JEPA predicts their consequences. Within each imagined candidate, predicted successor states condition subsequent intention proposals and action inference. Candidates are ranked using terminal goal distance and intention-realization consistency, after which the first action is executed and planning restarts from a real observation. The method is evaluated on 150 tasks in unseen MP3D scenes, with experiments across three visual encoders and ablations of feedback paths, planning configurations, and training objectives.
Critical strengths
- The architecture provides a coherent way to couple desired-state proposals with predicted action consequences. Distinguishing an intended next state from the state actually predicted under the inferred action is useful: their discrepancy can influence both candidate evaluation and subsequent candidate construction. The formulation, training objectives, and deployment procedure make this distinction clear.
- The feedback-path ablations provide convincing evidence for the specific mechanism. Post hoc F, F→G, and Full share model weights, candidate budgets, and the same scoring function, while changing where predicted states are fed back. Reach SR improves from 56.00% to 59.56% to 65.33%, with corresponding improvements in ReachSPL (Table 2). This is stronger evidence than the cross-system comparison because it controls several important confounding factors. The appendices also document training, deployment, and baseline adaptations in considerable detail.
Critical weaknesses
- Restricted evaluation setting. Clean150 contains only 150 tasks with initially visible, unobstructed goals and restricted distances, detours, and turning angles. This primarily tests local goal-directed navigation, leaving performance on occluded goals and more complex routes unclear.
- Overstated novelty claim. INTACT and FF-JEPA already establish intent-conditioned control and latent subgoal planning. The contribution should be framed more narrowly around feeding predicted consequences into learned intention proposals within each candidate, rather than a broadly new intention-first paradigm.
- Confounded action-first comparison. Removing Q also changes candidate sampling and eliminates the intention-consistency penalty. Consequently, the performance gap does not isolate the benefit of intention-based construction. A comparison with better-matched sampling and scoring is needed to substantiate this claim.
Questions for authors
- How does the method perform when goals are initially outside the field of view or occluded, or when reaching them requires larger detours? Does the advantage of full feedback over post-hoc evaluation persist under these conditions?
- Can the contribution of Q be separated from sampling and scoring changes using a matched, independently trained action-proposal baseline or an INTACT-style fixed-goal controller? In particular, how much of the improvement remains when the intention-consistency penalty is removed?
6 comments
Response to Reviewer 7New [2/4]
Q1: How does the method perform when goals are initially outside the field of view or occluded, or when reaching them requires larger detours? Does the advantage of full feedback over post-hoc evaluation persist under these conditions?
Response:
Thank you for asking whether the feedback advantage persists under these more demanding conditions. Full retains its advantage over Post hoc F in both new robot conditions. The robot setting and system results are reported in Table 5【1/4】.
Table 27. Robot feedback mechanisms (ours). Mean ± sample SD over three seeds. SR in %, SPL without leading zeros.
| Mechanism | Standard | Out of view | Detour | All |
| SR (%) ↑ / SPL ↑ | SR (%) ↑ / SPL ↑ | SR (%) ↑ / SPL ↑ | SR (%) ↑ / SPL ↑ | |
| Proposal only | 47.56 ± 2.04 / .390 ± .017 | 10.00 ± 3.33 / .067 ± .022 | 22.22 ± 1.92 / .162 ± .014 | 33.58 ± 1.54 / .268 ± .012 |
| Post hoc F | 68.00 ± 2.67 / .636 ± .025 | 21.11 ± 3.85 / .173 ± .031 | 41.11 ± 1.92 / .364 ± .017 | 51.60 ± 2.60 / .473 ± .023 |
| F→G | 72.00 ± 3.53 / .673 ± .033 | 26.67 ± 6.67 / .219 ± .055 | 47.78 ± 1.92 / .418 ± .017 | 56.54 ± 2.99 / .515 ± .026 |
| Full | 84.44 ± 2.78 / .815 ± .027 | 35.56 ± 5.09 / .306 ± .044 | 57.78 ± 3.85 / .520 ± .035 | 67.65 ± 3.50 / .636 ± .032 |
Post hoc F, F→G, and Full share the same V-JEPA checkpoint, scoring, K=8/H=4, and one executed action before replanning. Full exceeds Post hoc F by 14.44 SR points for out of view goals and 16.67 points for detours. Both SR and SPL improve in every seed in both conditions. Its advantages over F→G are 8.89 and 10.00 SR points. Since F→G already updates action generation from predicted states, the additional gain supports updating the local intentions themselves under these more demanding conditions.
Combined with the matched mechanism comparisons, this supports consequence feedback into intentions when direct visual guidance is initially unavailable or an indirect route is required.
We have added these robot experiments. Table 27 and the key difficult condition comparisons in Section 4.5 and Appendix G.3 retain red highlighting.
Response to Reviewer 7New [3/4]
W2: Overstated novelty claim. INTACT and FF-JEPA already establish intent-conditioned control and latent subgoal planning. The contribution should be framed more narrowly around feeding predicted consequences into learned intention proposals within each candidate, rather than a broadly new intention-first paradigm.
Response:
Thank you for this important and constructive suggestion. We agree that the novelty claim should focus on the specific feedback coupling within each candidate. We have refined the novelty framing in both the Abstract and Introduction. The Abstract now presents InJepa as an architecture for intention first coupled predictive planning. The first contribution explicitly locates its novelty in feeding predicted consequences into learned local intention proposals within each candidate. These changes are highlighted in blue in the Abstract and purple in the first contribution of the Introduction. Related novelty clarifications elsewhere in the manuscript are also highlighted in purple.
The revised statement describes an “architecture for intention first coupled predictive planning that feeds predicted consequences into learned local intention proposals within each candidate.” It then explains the resulting role of the world model: leading stepwise plan construction through this coupling, in addition to evaluating candidates. This wording locates the contribution in the specified feedback dependency, rather than claiming intent conditioned control or latent subgoal planning in general.
We recognize the foundations established by INTACT in intent conditioned control and by FF-JEPA in latent subgoal planning. Following your suggested framing, we emphasize the dependency that we investigate: the state predicted after realizing one learned intention becomes the input to the next learned intention proposal within the same candidate:
The same predicted state also conditions subsequent action inference. Consequently, local intentions evolve according to what preceding actions are predicted to achieve. The world model participates in constructing the candidate's desired transitions, rather than only scoring a completed plan.
To support this more precise claim, we retain the Full versus F→G comparison, which tests the dependency you identified. Both variants retain Q, predict action consequences, and feed predicted states into G. Only Full additionally feeds them into the next Q proposal. The complete mechanism results and encoder controls are:
Table 2. Planning mechanism ablations of InJepa (ours). SR is in percent. Three evaluation seeds.
| Deployment | Seed 0 | Seed 1 | Seed 2 | Mean ± SD |
| SR (%) ↑ / SPL ↑ | SR (%) ↑ / SPL ↑ | SR (%) ↑ / SPL ↑ | SR (%) ↑ / SPL ↑ | |
| Proposal only | 32.67 / 0.2426 | 34.67 / 0.2775 | 28.00 / 0.2377 | 31.78 ± 3.42 / 0.2526 ± 0.0217 |
| Post hoc F | 58.00 / 0.5359 | 55.33 / 0.5096 | 54.67 / 0.4921 | 56.00 ± 1.76 / 0.5126 ± 0.0220 |
| F→G | 62.67 / 0.5728 | 56.67 / 0.5151 | 59.33 / 0.5395 | 59.56 ± 3.01 / 0.5425 ± 0.0289 |
| F→Q+G (full) | 68.00 / 0.6357 | 62.67 / 0.5951 | 65.33 / 0.6132 | 65.33 ± 2.67 / 0.6147 ± 0.0203 |
| Action-first InJepa | 48.67 / 0.4073 | 40.00 / 0.3302 | 40.00 / 0.3368 | 42.89 ± 5.00 / 0.3581 ± 0.0428 |
Table 22. Feedback mechanism controls across encoders for InJepa (ours). Three seeds with 150 tasks per seed.
CroCo
| Mechanism | SR (%) ↑ | ReachSPL ↑ |
| Proposal only | 49.11 ± 1.02 | 0.4591 ± 0.0095 |
| Post hoc F | 56.67 ± 0.67 | 0.5298 ± 0.0062 |
| F→G | 57.78 ± 1.02 | 0.5384 ± 0.0095 |
| Full F→Q+G | 61.78 ± 0.38 | 0.5670 ± 0.0127 |
| No Q, argmax | 44.67 ± 0.00 | 0.4115 ± 0.0000 |
DINOv3
| Mechanism | SR (%) ↑ | ReachSPL ↑ |
| Proposal only | 23.56 ± 5.67 | 0.1938 ± 0.0466 |
| Post hoc F | 48.22 ± 1.39 | 0.4520 ± 0.0130 |
| F→G | 49.78 ± 1.02 | 0.4536 ± 0.0093 |
| Full F→Q+G | 53.11 ± 3.01 | 0.4861 ± 0.0317 |
| No Q, argmax | 38.67 ± 0.00 | 0.3474 ± 0.0000 |
Adding feedback into Q raises mean SR from 59.56% to 65.33% with V-JEPA, with gains of 4.00 points for CroCo and 3.33 for DINOv3. These comparisons support the revised claim about feeding predicted consequences into subsequent learned intention proposals across representations. We have aligned the contribution statement and its experimental interpretation with this specific mechanism, as you recommended.
The key mechanism comparisons in Appendix D.2 and F.1 retain red highlighting.
Response to Reviewer 7New [1/4]
Dear Reviewer 7New,
We sincerely thank you for the thoughtful review and constructive suggestions on harder navigation conditions, the precise contribution, and a matched fixed goal control. These questions helped us sharpen the contribution and strengthen its experimental support. Purple highlights locate the specific contribution statement. Shared key robot and controller evidence retains red highlights, and the seed sensitivity evidence retains green. Other manuscript revisions remain blue. Uncertainties denote sample standard deviations over three evaluation seeds.
Response:
Thank you for this valuable suggestion to test more demanding navigation conditions. We extended evaluation to zero shot navigation on a Lite3 quadruped in five indoor scenes, with 135 trials per seed across three seeds, totaling 405 trials per method. Each seed includes 75 standard, 30 initially out of view, and 30 detour trials. Arrival requires reaching within 0.5 m of the goal. All methods share start and goal assignments and safety termination rules. Loss of goal guidance followed by an imminent collision requiring intervention counts as failure.
Table 5. Real world navigation. Mean ± sample SD over three seeds. SR in %, SPL without leading zeros.
The system results provide a complementary check. Averaging the individual runs gives overall SR/SPL of 67.65%/0.636 for Full and 62.96%/0.608 for LWM-CroCo. Full’s SR advantages over LWM-CroCo are 24.44 points on out of view trials and 4.44 on detours.
We have added these robot experiments. Table 5 and the key difficult condition comparisons in Section 4.5 retain red highlighting.
Complementary offline evaluation.
Beyond closed loop navigation, we also added offline planning and action identification evaluations. PE and OE measure planned endpoint position and orientation errors. ACC3/ACC5 measure identification of the reference action sequence among three or five candidates.
Table 4. Offline planning and action selection on 150 common queries.
InJepa-DINOv3 achieves the lowest PE and OE. With the same VJEPA encoder, InJepa reduces PE and OE by 8.2% and 4.1% relative to LWM. All three InJepa variants lead in ACC3/ACC5. Together with the robot results, these measurements extend the evidence to goal directed plan construction and reliable action identification across visual representations.
Table 4 and the key analysis in Section 4.4 are highlighted in red.