acceptodds
Under review as a conference paper at ICLR 2027

Intention First: Latent State Planning with Coupled JEPAs for Visual Navigation

Abstract

Visual navigation requires goal directed planning informed by predictions of environment dynamics. Representative navigation world model methods propose complete action candidates before predicting consequences for selection or search updates. For each candidate, predictions primarily evaluate proposed actions rather than guide its stepwise construction. To this end, we propose InJepa, an intention first planning paradigm centered on latent states. A goal conditioned JEPA proposes latent intentions, and an action conditioned JEPA continually predicts future latent states. Both operate in a shared latent space to align their predictive representations. Within each candidate, predicted consequences feed into the next intention proposal and action inference, forming a closed loop of intention, action, and consequence. This extends the world model's role beyond evaluation to leading candidate construction through latent intentions. We evaluate InJepa through closed loop image goal navigation in Habitat MP3D. The results show effective multistep navigation through intention guided action generation and consequence feedback. Experiments across encoders further support applicability across visual representations. These findings support latent intentions as the central planning object and show that world models can directly guide plan construction through coordinated reasoning about intentions and consequences.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Image Goal Navigation, World Models, Joint Embedding Predictive Architecture, Latent State Planning, Embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7NewOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes InJepa, a latent-state planning architecture for image-goal navigation. A goal-conditioned Proposal JEPA generates candidate desired states, an action model translates state differences into actions, and a Forward JEPA predicts their consequences. Within each imagined candidate, predicted successor states condition subsequent intention proposals and action inference. Candidates are ranked using terminal goal distance and intention-realization consistency, after which the first action is executed and planning restarts from a real observation. The method is evaluated on 150 tasks in unseen MP3D scenes, with experiments across three visual encoders and ablations of feedback paths, planning configurations, and training objectives.

Critical strengths
  1. The architecture provides a coherent way to couple desired-state proposals with predicted action consequences. Distinguishing an intended next state from the state actually predicted under the inferred action is useful: their discrepancy can influence both candidate evaluation and subsequent candidate construction. The formulation, training objectives, and deployment procedure make this distinction clear.
  2. The feedback-path ablations provide convincing evidence for the specific mechanism. Post hoc F, F→G, and Full share model weights, candidate budgets, and the same scoring function, while changing where predicted states are fed back. Reach SR improves from 56.00% to 59.56% to 65.33%, with corresponding improvements in ReachSPL (Table 2). This is stronger evidence than the cross-system comparison because it controls several important confounding factors. The appendices also document training, deployment, and baseline adaptations in considerable detail.
Critical weaknesses
  1. Restricted evaluation setting. Clean150 contains only 150 tasks with initially visible, unobstructed goals and restricted distances, detours, and turning angles. This primarily tests local goal-directed navigation, leaving performance on occluded goals and more complex routes unclear.
  2. Overstated novelty claim. INTACT and FF-JEPA already establish intent-conditioned control and latent subgoal planning. The contribution should be framed more narrowly around feeding predicted consequences into learned intention proposals within each candidate, rather than a broadly new intention-first paradigm.
  3. Confounded action-first comparison. Removing Q also changes candidate sampling and eliminates the intention-consistency penalty. Consequently, the performance gap does not isolate the benefit of intention-based construction. A comparison with better-matched sampling and scoring is needed to substantiate this claim.
Questions for authors
  1. How does the method perform when goals are initially outside the field of view or occluded, or when reaching them requires larger detours? Does the advantage of full feedback over post-hoc evaluation persist under these conditions?
  2. Can the contribution of Q be separated from sampling and scoring changes using a matched, independently trained action-proposal baseline or an INTACT-style fixed-goal controller? In particular, how much of the improvement remains when the intention-consistency penalty is removed?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 7New [1/4]

Dear Reviewer 7New,

We sincerely thank you for the thoughtful review and constructive suggestions on harder navigation conditions, the precise contribution, and a matched fixed goal control. These questions helped us sharpen the contribution and strengthen its experimental support. Purple highlights locate the specific contribution statement. Shared key robot and controller evidence retains red highlights, and the seed sensitivity evidence retains green. Other manuscript revisions remain blue. Uncertainties denote sample standard deviations over three evaluation seeds.

W1: Restricted evaluation setting. Clean150 contains only 150 tasks with initially visible, unobstructed goals and restricted distances, detours, and turning angles. This primarily tests local goal-directed navigation, leaving performance on occluded goals and more complex routes unclear.

Response:

Thank you for this valuable suggestion to test more demanding navigation conditions. We extended evaluation to zero shot navigation on a Lite3 quadruped in five indoor scenes, with 135 trials per seed across three seeds, totaling 405 trials per method. Each seed includes 75 standard, 30 initially out of view, and 30 detour trials. Arrival requires reaching within 0.5 m of the goal. All methods share start and goal assignments and safety termination rules. Loss of goal guidance followed by an imminent collision requiring intervention counts as failure.

Table 5. Real world navigation. Mean ± sample SD over three seeds. SR in %, SPL without leading zeros.

MethodStandardOut of viewDetourAll
SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑
InJepa-VJEPA (ours)84.44 ± 2.78 / .815 ± .02735.56 ± 5.09 / .306 ± .04457.78 ± 3.85 / .520 ± .03567.65 ± 3.50 / .636 ± .032
LWM-CroCo87.56 ± 0.77 / .858 ± .00811.11 ± 1.92 / .098 ± .01753.33 ± 3.33 / .496 ± .03162.96 ± 1.28 / .608 ± .012
LWM-VJEPA46.67 ± 1.33 / .427 ± .0122.22 ± 1.92 / .017 ± .01427.78 ± 5.09 / .228 ± .04232.59 ± 2.22 / .291 ± .019
NoMaD30.67 ± 2.67 / .217 ± .01926.67 ± 3.33 / .159 ± .02012.22 ± 5.09 / .076 ± .03225.68 ± 1.54 / .173 ± .011

The system results provide a complementary check. Averaging the individual runs gives overall SR/SPL of 67.65%/0.636 for Full and 62.96%/0.608 for LWM-CroCo. Full’s SR advantages over LWM-CroCo are 24.44 points on out of view trials and 4.44 on detours.

We have added these robot experiments. Table 5 and the key difficult condition comparisons in Section 4.5 retain red highlighting.

Complementary offline evaluation.

Beyond closed loop navigation, we also added offline planning and action identification evaluations. PE and OE measure planned endpoint position and orientation errors. ACC3/ACC5 measure identification of the reference action sequence among three or five candidates.

Table 4. Offline planning and action selection on 150 common queries.

MethodPE (m) ↓OE (rad) ↓ACC3 (%) ↑ACC5 (%) ↑
NoMaD0.44880.8172N/AN/A
RAE-NWM0.88511.050582.6775.33
LWM-CroCo0.26960.227388.0079.33
LWM-VJEPA0.29840.234786.0076.00
InJepa-DINOv3 (ours)0.21730.174598.0096.00
InJepa-CroCo (ours)0.27880.226999.3398.67
InJepa-VJEPA (ours)0.27400.225198.6798.67

InJepa-DINOv3 achieves the lowest PE and OE. With the same VJEPA encoder, InJepa reduces PE and OE by 8.2% and 4.1% relative to LWM. All three InJepa variants lead in ACC3/ACC5. Together with the robot results, these measurements extend the evidence to goal directed plan construction and reliable action identification across visual representations.

Table 4 and the key analysis in Section 4.4 are highlighted in red.

AuthorsOpenReview ↗

Response to Reviewer 7New [2/4]

Q1: How does the method perform when goals are initially outside the field of view or occluded, or when reaching them requires larger detours? Does the advantage of full feedback over post-hoc evaluation persist under these conditions?

Response:

Thank you for asking whether the feedback advantage persists under these more demanding conditions. Full retains its advantage over Post hoc F in both new robot conditions. The robot setting and system results are reported in Table 5【1/4】.

Table 27. Robot feedback mechanisms (ours). Mean ± sample SD over three seeds. SR in %, SPL without leading zeros.

MechanismStandardOut of viewDetourAll
SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑
Proposal only47.56 ± 2.04 / .390 ± .01710.00 ± 3.33 / .067 ± .02222.22 ± 1.92 / .162 ± .01433.58 ± 1.54 / .268 ± .012
Post hoc F68.00 ± 2.67 / .636 ± .02521.11 ± 3.85 / .173 ± .03141.11 ± 1.92 / .364 ± .01751.60 ± 2.60 / .473 ± .023
F→G72.00 ± 3.53 / .673 ± .03326.67 ± 6.67 / .219 ± .05547.78 ± 1.92 / .418 ± .01756.54 ± 2.99 / .515 ± .026
Full84.44 ± 2.78 / .815 ± .02735.56 ± 5.09 / .306 ± .04457.78 ± 3.85 / .520 ± .03567.65 ± 3.50 / .636 ± .032

Post hoc F, F→G, and Full share the same V-JEPA checkpoint, scoring, K=8/H=4, and one executed action before replanning. Full exceeds Post hoc F by 14.44 SR points for out of view goals and 16.67 points for detours. Both SR and SPL improve in every seed in both conditions. Its advantages over F→G are 8.89 and 10.00 SR points. Since F→G already updates action generation from predicted states, the additional gain supports updating the local intentions themselves under these more demanding conditions.

Combined with the matched mechanism comparisons, this supports consequence feedback into intentions when direct visual guidance is initially unavailable or an indirect route is required.

We have added these robot experiments. Table 27 and the key difficult condition comparisons in Section 4.5 and Appendix G.3 retain red highlighting.

AuthorsOpenReview ↗

Response to Reviewer 7New [3/4]

W2: Overstated novelty claim. INTACT and FF-JEPA already establish intent-conditioned control and latent subgoal planning. The contribution should be framed more narrowly around feeding predicted consequences into learned intention proposals within each candidate, rather than a broadly new intention-first paradigm.

Response:

Thank you for this important and constructive suggestion. We agree that the novelty claim should focus on the specific feedback coupling within each candidate. We have refined the novelty framing in both the Abstract and Introduction. The Abstract now presents InJepa as an architecture for intention first coupled predictive planning. The first contribution explicitly locates its novelty in feeding predicted consequences into learned local intention proposals within each candidate. These changes are highlighted in blue in the Abstract and purple in the first contribution of the Introduction. Related novelty clarifications elsewhere in the manuscript are also highlighted in purple.

The revised statement describes an “architecture for intention first coupled predictive planning that feeds predicted consequences into learned local intention proposals within each candidate.” It then explains the resulting role of the world model: leading stepwise plan construction through this coupling, in addition to evaluating candidates. This wording locates the contribution in the specified feedback dependency, rather than claiming intent conditioned control or latent subgoal planning in general.

We recognize the foundations established by INTACT in intent conditioned control and by FF-JEPA in latent subgoal planning. Following your suggested framing, we emphasize the dependency that we investigate: the state predicted after realizing one learned intention becomes the input to the next learned intention proposal within the same candidate:

ph+1k=Q(fhk,mt,ξk).p_{h+1}^{k}=Q(f_h^{k},m_t,\xi^{k}).

The same predicted state also conditions subsequent action inference. Consequently, local intentions evolve according to what preceding actions are predicted to achieve. The world model participates in constructing the candidate's desired transitions, rather than only scoring a completed plan.

To support this more precise claim, we retain the Full versus F→G comparison, which tests the dependency you identified. Both variants retain Q, predict action consequences, and feed predicted states into G. Only Full additionally feeds them into the next Q proposal. The complete mechanism results and encoder controls are:

Table 2. Planning mechanism ablations of InJepa (ours). SR is in percent. Three evaluation seeds.

DeploymentSeed 0Seed 1Seed 2Mean ± SD
SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑
Proposal only32.67 / 0.242634.67 / 0.277528.00 / 0.237731.78 ± 3.42 / 0.2526 ± 0.0217
Post hoc F58.00 / 0.535955.33 / 0.509654.67 / 0.492156.00 ± 1.76 / 0.5126 ± 0.0220
F→G62.67 / 0.572856.67 / 0.515159.33 / 0.539559.56 ± 3.01 / 0.5425 ± 0.0289
F→Q+G (full)68.00 / 0.635762.67 / 0.595165.33 / 0.613265.33 ± 2.67 / 0.6147 ± 0.0203
Action-first InJepa48.67 / 0.407340.00 / 0.330240.00 / 0.336842.89 ± 5.00 / 0.3581 ± 0.0428

Table 22. Feedback mechanism controls across encoders for InJepa (ours). Three seeds with 150 tasks per seed.

CroCo

MechanismSR (%) ↑ReachSPL ↑
Proposal only49.11 ± 1.020.4591 ± 0.0095
Post hoc F56.67 ± 0.670.5298 ± 0.0062
F→G57.78 ± 1.020.5384 ± 0.0095
Full F→Q+G61.78 ± 0.380.5670 ± 0.0127
No Q, argmax44.67 ± 0.000.4115 ± 0.0000

DINOv3

MechanismSR (%) ↑ReachSPL ↑
Proposal only23.56 ± 5.670.1938 ± 0.0466
Post hoc F48.22 ± 1.390.4520 ± 0.0130
F→G49.78 ± 1.020.4536 ± 0.0093
Full F→Q+G53.11 ± 3.010.4861 ± 0.0317
No Q, argmax38.67 ± 0.000.3474 ± 0.0000

Adding feedback into Q raises mean SR from 59.56% to 65.33% with V-JEPA, with gains of 4.00 points for CroCo and 3.33 for DINOv3. These comparisons support the revised claim about feeding predicted consequences into subsequent learned intention proposals across representations. We have aligned the contribution statement and its experimental interpretation with this specific mechanism, as you recommended.

The key mechanism comparisons in Appendix D.2 and F.1 retain red highlighting.

Reviewer_BcrnOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes intention-first planning for navigation with world models. A goal-conditioned Proposal JEPA proposes a latent intention, an action model converts the difference between intention and current state into actions, and a forward-JEPA predicts the consequences of the resulting actions. Candidates are scored by proximity to the goal and the intention's realization error, and executed step by step. InJepa-VJEPA, trained on trajectories from StreamVLN, achieves around 65% SR on 150 image-goal tasks in unseen MP3D scenes and surpasses LWM-VJEPA, NoMaD, and RAE-NWM, but LWM-CroCo (70.22) remains the best overall. A staircase of mechanism ablations, offline diagnostics, and 3 encoder variants supports the design.

Critical strengths

1.The paper discusses its relation to FF-JEPA, HWM, LDP, and INTACT. Particularly useful is the comparison at fixed weights between Post hoc F, F→G, and the full model, with an average SR of 56.00, 59.56, and 65.33 respectively under the same evaluation regime, which constitutes evidence for the feedback pathway. Proposal only reaches a mean of 31.78, but that is because candidate generation is unchanged and selection happens by the final intention-goal distance rather than Jk; its inferiority to Post hoc F reflects the transition to a branching-based evaluation grounded in F.

2.The paper reproduces LWM-CroCo and LWM-VJEPA from the same StreamVLN source, the latter being retrained after replacing the visual front-end. Appendix E.1 explicitly states that the data filtering and partitions are not identical across methods. The full model is evaluated with 3 seeds, which vary sampling at evaluation rather than training initialization, and the paper acknowledges that LWM-CroCo is the best. The manuscript additionally specifies further configuration and evaluation deliverables.

Critical weaknesses

1.The gain reverses when the encoder changes, without explanation. With CroCo, InJepa reaches 61.78±0.38, more than 8 points behind the current "action-first" LWM-CroCo (70.22±1.02), while the ranking flips with V-JEPA (65.33 versus 42.44). The compatibility-fit explanation is only mentioned, never analyzed. The comparative advantage is not stable across encoders, and the ranking reversal itself makes it impossible to distinguish whether intention-first planning fails in itself under CroCo, or whether it merely loses its edge against baselines trained with a compatibility evaluation. A feedback ablation under CroCo in matched settings would have separated these two cases.

2.The evidence is narrower than the claim of a new paradigm. As specified in Appendix F.2, Clean150 requires that the target initially be in the camera's line-of-sight field of view, a geodesic distance of 5 to 10 m, a detour factor of at most 1.3, and cumulative path turning of at most 120 degrees, which leaves occluded targets and topological exploration untested. The action-first variant reuses the full checkpoint while changing action sampling and removing the consistency term, and Appendix D.4 acknowledges that these changes are not isolated, so it is two modes of operation being compared, not intention generation itself. Table 15 measures latency on a single task, about 1.42 seconds per decision for InJepa-VJEPA versus 0.41 seconds for LWM-CroCo, and the cost-benefit trade-off at larger scale remains unknown.

Questions for authors

1.Table 15 was measured on 1 task, with 10 to 30 calls per method. Can the timing be extended to more scenes? And how should the roughly 3.5-fold per-decision latency difference relative to LWM-CroCo be weighed in a practical comparison?

2.Can the "compatibility" be verified directly, for example via the separability of goal states in CroCo's and V-JEPA's feature spaces, the precision of the forward model's predictions, or the calibration of the intention's realization error?

3.Is the sensitivity to λc in Figure 4(c) robust across evaluation seeds?

4.If the initial conditions of Clean150 — line of sight and low detour — are relaxed (occluded targets, large detours), does the feedback gain persist?

7 comments
AuthorsOpenReview ↗

Response to Reviewer Bcrn [1/6]

Dear Reviewer Bcrn,

We sincerely appreciate your careful review and insightful suggestions on representation compatibility, evaluation scope, and practical cost. Your concrete questions helped us design more informative controls and diagnostics. Green highlights identify the key compatibility, seed sensitivity, and timing evidence. Shared mechanism and robot conclusions retain red highlights. Other manuscript revisions remain blue. Navigation results report means and sample standard deviations across three evaluation seeds. Timing standard deviations are computed across the 150 tasks.

W1: 1.The gain reverses when the encoder changes, without explanation. With CroCo, InJepa reaches 61.78±0.38, more than 8 points behind the current "action-first" LWM-CroCo (70.22±1.02), while the ranking flips with V-JEPA (65.33 versus 42.44). The compatibility-fit explanation is only mentioned, never analyzed. The comparative advantage is not stable across encoders, and the ranking reversal itself makes it impossible to distinguish whether intention-first planning fails in itself under CroCo, or whether it merely loses its edge against baselines trained with a compatibility evaluation. A feedback ablation under CroCo in matched settings would have separated these two cases.

Response: Thank you for suggesting matched feedback ablations. We added controls and diagnostics to separate the feedback contribution from encoder dependent performance.

Matched feedback controls across encoders.

Following your suggestion, we added matched feedback ablations with CroCo and DINOv3. Both show improved navigation from feeding predicted consequences into learned intentions. For each encoder, we fix the E12 checkpoint, K=8, H=4, and execution of one action before replanning.

Table 22. Feedback mechanism controls across encoders for InJepa (ours). Three seeds with 150 tasks per seed.

CroCo

MechanismSR (%) ↑ReachSPL ↑
Proposal only49.11 ± 1.020.4591 ± 0.0095
Post hoc F56.67 ± 0.670.5298 ± 0.0062
F→G57.78 ± 1.020.5384 ± 0.0095
Full F→Q+G61.78 ± 0.380.5670 ± 0.0127
No Q, argmax44.67 ± 0.000.4115 ± 0.0000

DINOv3

MechanismSR (%) ↑ReachSPL ↑
Proposal only23.56 ± 5.670.1938 ± 0.0466
Post hoc F48.22 ± 1.390.4520 ± 0.0130
F→G49.78 ± 1.020.4536 ± 0.0093
Full F→Q+G53.11 ± 3.010.4861 ± 0.0317
No Q, argmax38.67 ± 0.000.3474 ± 0.0000

Full improves over Post hoc F by 5.11 SR points with CroCo and 4.89 with DINOv3, and over F→G by 4.00 and 3.33 points. All four gains hold in every seed, for both SR and ReachSPL. This answers the distinction in W1: the feedback mechanism remains effective with CroCo, even though LWM-CroCo is the stronger complete system in simulation. The CroCo result therefore does not indicate a failure of consequence feedback into learned intentions. Cross system ranking additionally reflects how each complete planner uses its representation, which we examine through direct diagnostics and planning accuracy.

We connect these controls to the encoder diagnostics and the InJepa versus LWM planning comparison in our response to Q2【2/6】.

AuthorsOpenReview ↗

Response to Reviewer Bcrn [2/6]

Q2: 2.Can the "compatibility" be verified directly, for example via the separability of goal states in CroCo's and V-JEPA's feature spaces, the precision of the forward model's predictions, or the calibration of the intention's realization error?

Response: Thank you for suggesting these direct tests of representation compatibility.

(2) Direct compatibility diagnostics and their relation to planning.

We tested all three suggested properties. Goal ordering checks feature distance against geodesic proximity using paired locations with fixed camera heading on 150 tasks. F relative RMSE divides prediction error under recorded actions by the error of predicting no state change on 4,096 matched validation pairs. Realization calibration compares predicted intention mismatch with mismatch after simulator replay of the same candidate actions. MAE is normalized by each encoder's persistence RMS at the same depth. Spearman correlation measures ordering within each query over eight candidates on the same 150 tasks. Aggregates weight steps 1–4 equally.

Table 23. Representation diagnostics on matched data for InJepa (ours).

EncoderGoal ordering (%) ↑F relative RMSE ↓Calibration MAE ↓Spearman ρ ↑
VJEPA85.360.56540.29070.5961
CroCo77.960.51860.28410.1900
DINOv387.540.60620.30990.7478

Table 24. Encoder compatibility by prediction step for InJepa (ours). Each correlation uses the same 150 tasks and eight candidates per task. Relative errors use the encoder specific persistence baseline.

h=1F rel. RMSE ↓MAE ↓ρ ↑
VJEPA0.53130.32460.5514
CroCo0.46980.21780.2495
DINOv30.60060.35840.7163
h=2F rel. RMSE ↓MAE ↓ρ ↑
VJEPA0.54240.28820.6556
CroCo0.49250.27650.2619
DINOv30.58510.30750.7622
h=3F rel. RMSE ↓MAE ↓ρ ↑
VJEPA0.57750.28150.6278
CroCo0.53600.31600.1497
DINOv30.60700.29190.7740
h=4F rel. RMSE ↓MAE ↓ρ ↑
VJEPA0.61040.26860.5495
CroCo0.57610.32610.0989
DINOv30.63230.28200.7387

Accurate state prediction alone does not ensure effective goal directed planning. CroCo has 8.3% lower relative F error than V-JEPA, but 7.40 points lower goal ordering accuracy and realization correlation of 0.1900 versus 0.5961. InJepa needs its representation both to specify local targets and to predict their realization.

CroCo's realization correlation falls from 0.2495 at step one to 0.0989 at step four, despite relative F error remaining below one. DINOv3 has the strongest goal and realization ordering, but 7.2% higher relative F error and 6.6% higher calibration MAE than V-JEPA. Compatibility depends jointly on goal geometry, consequence prediction, and realization ordering.

We also adopted LWM's published offline evaluation framework to connect these diagnostics to plan accuracy (PE/OE) and action identification (ACC3/ACC5).

Table 4. Offline planning and action selection on 150 common queries. InJepa uses F for ACC.

MethodPE (m) ↓OE (rad) ↓ACC3 (%) ↑ACC5 (%) ↑
NoMaD0.44880.8172N/AN/A
RAE-NWM0.88511.050582.6775.33
LWM-CroCo0.26960.227388.0079.33
LWM-VJEPA0.29840.234786.0076.00
InJepa-DINOv3 (ours)0.21730.174598.0096.00
InJepa-CroCo (ours)0.27880.226999.3398.67
InJepa-VJEPA (ours)0.27400.225198.6798.67

Plan accuracy reveals encoder dependent differences. With CroCo, LWM has lower PE (0.2696 versus 0.2788), while InJepa has slightly lower OE (0.2269 versus 0.2273). With V-JEPA, InJepa has lower PE/OE (0.2740/0.2251 versus 0.2984/0.2347). Meanwhile, all three InJepa encoders achieve high ACC3/ACC5, so accurate identification of supplied action sequences does not by itself ensure equally accurate plan construction. Together with the goal and realization diagnostics, this connects encoder dependence to the demands of constructing local futures. Table 22【1/6】 establishes that feedback improves each encoder's planner. These results distinguish a consistent feedback contribution from encoder dependent system rankings.

Table 23 and the key compatibility conclusions in Appendix F.2–F.3 are marked in green. Shared Tables 4 and 22 and the matched feedback conclusion in Appendix F.1 retain red highlighting.

AuthorsOpenReview ↗

Response to Reviewer Bcrn [3/6]

Q4: 4.If the initial conditions of Clean150 — line of sight and low detour — are relaxed (occluded targets, large detours), does the feedback gain persist?

Response: Thank you for this important question about feedback beyond the original visibility and route constraints.

Feedback gains persist in both added conditions. Frozen models run zero shot on Lite3 in five indoor scenes: 135 trials per seed, three seeds, 405 per method, with 75 standard, 30 out of view, and 30 detour trials per seed. Arrival is within 0.5 m. Methods share start and goal assignments and safety rules. Losing goal guidance and requiring intervention to avoid collision counts as failure. Post hoc F, F→G, and Full share weights, scoring, K=8/H=4, and one action per replan.

Table 5. Real world navigation. Mean ± sample SD over three seeds. SR in %, SPL without leading zeros.

MethodStandardOut of viewDetourAll
SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑
InJepa-VJEPA (ours)84.44 ± 2.78 / .815 ± .02735.56 ± 5.09 / .306 ± .04457.78 ± 3.85 / .520 ± .03567.65 ± 3.50 / .636 ± .032
LWM-CroCo87.56 ± 0.77 / .858 ± .00811.11 ± 1.92 / .098 ± .01753.33 ± 3.33 / .496 ± .03162.96 ± 1.28 / .608 ± .012
LWM-VJEPA46.67 ± 1.33 / .427 ± .0122.22 ± 1.92 / .017 ± .01427.78 ± 5.09 / .228 ± .04232.59 ± 2.22 / .291 ± .019
NoMaD30.67 ± 2.67 / .217 ± .01926.67 ± 3.33 / .159 ± .02012.22 ± 5.09 / .076 ± .03225.68 ± 1.54 / .173 ± .011

InJepa achieves the highest overall SR and SPL. Relative to LWM-CroCo, its SR is 24.44 points higher for out of view goals and 4.44 points higher on detours (Table 5).

Table 27. Robot feedback mechanisms (ours). Mean ± sample SD over three seeds. SR in %, SPL without leading zeros.

MechanismStandardOut of viewDetourAll
SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑SR (%) ↑ / SPL ↑
Proposal only47.56 ± 2.04 / .390 ± .01710.00 ± 3.33 / .067 ± .02222.22 ± 1.92 / .162 ± .01433.58 ± 1.54 / .268 ± .012
Post hoc F68.00 ± 2.67 / .636 ± .02521.11 ± 3.85 / .173 ± .03141.11 ± 1.92 / .364 ± .01751.60 ± 2.60 / .473 ± .023
F→G72.00 ± 3.53 / .673 ± .03326.67 ± 6.67 / .219 ± .05547.78 ± 1.92 / .418 ± .01756.54 ± 2.99 / .515 ± .026
Full84.44 ± 2.78 / .815 ± .02735.56 ± 5.09 / .306 ± .04457.78 ± 3.85 / .520 ± .03567.65 ± 3.50 / .636 ± .032

Full exceeds Post hoc F by 14.44 SR points out of view and 16.67 on detours, with higher SR and SPL in every seed. Gains over F→G are 8.89 and 10.00 points. Updating intentions thus adds value beyond updating G when direct sight or a direct route is unavailable. All variants reobserve after actions, isolating feedback within planning.

We highlight these robot results and feedback comparisons in red in Section 4.5, Table 27, and Appendix G.3.

Reviewer_umBXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes InJepa, an "intention-first" planning framework for visual navigation. In detail, the method first proposes goal-conditioned latent intentions, translates intentions into actions, and uses a forward JEPA model to predict future latent states. Experiments on Habitat MP3D scenarios compare InJepa against several navigation baselines and include ablation studies on intention generation and consequence feedback.

Critical strengths

The paper attempts to analyze the contribution of intention proposals and consequence feedback through controlled variants

Critical weaknesses
  1. The proposed Clean150 benchmark is heavily curated with multiple manually designed constraints and filtering rules. It is unclear whether the resulting evaluation setting is representative of standard visual navigation benchmarks and whether comparisons remain fully fair across methods.
  2. The paper attributes gains to “intention-first” planning, but many compared baselines are fundamentally action-sequence or open-loop planners, while InJepa relies on iterative closed-loop feedback during candidate construction. The current experiments do not clearly isolate whether improvements come from latent intentions or from the closed-loop feedback mechanism itself.
  3. Table 1 shows that LWM-CroCo achieves better navigation performance than the proposed InJepa variants. Therefore, the experiments do not convincingly support the claim that intention-first planning is a superior planning paradigm.
  4. The paper argues that latent intentions should become the primary planning object, yet the evidence is derived from a single navigation benchmark and a relatively limited set of environments. The presented experiments are insufficient to support such a broad conclusion.
  5. Minor suggestions: The sizes of the lines in the image are too uneven, making it difficult to clearly read the text and the key points.
Questions for authors
  1. Why was evaluation conducted on the custom Clean150 benchmark rather than established navigation benchmarks with standard protocols?
  2. Since InJepa uses iterative closed-loop feedback while several baselines primarily perform open-loop planning, how can the authors disentangle the benefit of latent intentions from the benefit of the feedback mechanism itself?
  3. What is the key conceptual difference between the proposed intention-first framework and prior latent subgoal/state planning methods
  4. See the concerns raised in the weakness part.
6 comments
AuthorsOpenReview ↗

Response to Reviewer umBX [1/5]

Dear Reviewer umBX,

Thank you for your thoughtful review and valuable suggestions, which strengthened our evidence and presentation.

W1: The proposed Clean150 benchmark is heavily curated with multiple manually designed constraints and filtering rules. It is unclear whether the resulting evaluation setting is representative of standard visual navigation benchmarks and whether comparisons remain fully fair across methods.

Q1: Why was evaluation conducted on the custom Clean150 benchmark rather than established navigation benchmarks with standard protocols?

Response: Thank you for this important question. Clean150 isolates sustained goal directed planning from global search. It tests this controlled setting, rather than the full ImageGoal distribution. Habitat's PointNav generator supplies tasks [1, 2]. A reference follower checks reachability, and image screening removes reconstruction defects and uninformative close views. No learned policy results enter construction or filtering. All methods share 150 tasks in 11 unseen scenes, observations, 200 control steps, and a 1 m arrival radius. Deployed planners share tasks, while matched controls isolate our mechanism.

Habitat's MP3D ImageGoal tasks combine exploration and navigation, allowing 1–30 m shortest paths without initial goal visibility [1, 2]. NoMaD distinguishes these capabilities and notes that separate models often handle them [3]. To the best of our knowledge, existing navigation world models [4, 5, 6, 7] provide no shared public closed loop simulation protocol. Clean150 fixes tasks and execution conditions to isolate planning.

Following your suggestion, we adopted LWM's published evaluation framework [5], adding all its public offline evaluation tasks and zero shot robot navigation alongside controlled simulation. On the same 150 queries, PE/OE measure planning endpoint position/orientation errors, and ACC3/ACC5 identify the target reaching action sequence among three/five candidates. InJepa uses F for ACC. NoMaD has no scoring interface.

Table 4. Offline planning and action selection.

MethodPE (m) ↓OE (rad) ↓ACC3 (%) ↑ACC5 (%) ↑
NoMaD0.44880.8172N/AN/A
RAE-NWM0.88511.050582.6775.33
LWM-CroCo0.26960.227388.0079.33
LWM-VJEPA0.29840.234786.0076.00
InJepa-DINOv3 (ours)0.21730.174598.0096.00
InJepa-CroCo (ours)0.27880.226999.3398.67
InJepa-VJEPA (ours)0.27400.225198.6798.67

InJepa-VJEPA reduces PE/OE from LWM-VJEPA's 0.2984/0.2347 to 0.2740/0.2251. Across encoders, its ACC3/ACC5 reach 98.00–99.33%/96.00–98.67%, supporting accurate plans and reliable consequence evaluation.

Robot trials relax the original visibility and route restrictions: five scenes, 135 trials per seed (75 standard, 30 out of view, 30 detour), and three seeds totaling 405 trials per method. Methods share the Lite3, start/goal assignments, 0.5 m arrival threshold, and safety termination rules. Intervention for imminent collision after loss of goal guidance counts as failure.

Table 27. Robot feedback mechanisms of InJepa (ours). Mean ± sample SD.

SR (%) ↑

MechanismStandardOut of viewDetourAll
Proposal only47.56 ± 2.0410.00 ± 3.3322.22 ± 1.9233.58 ± 1.54
Post hoc F68.00 ± 2.6721.11 ± 3.8541.11 ± 1.9251.60 ± 2.60
F→G72.00 ± 3.5326.67 ± 6.6747.78 ± 1.9256.54 ± 2.99
Full84.44 ± 2.7835.56 ± 5.0957.78 ± 3.8567.65 ± 3.50

SPL ↑

MechanismStandardOut of viewDetourAll
Proposal only0.390 ± 0.0170.067 ± 0.0220.162 ± 0.0140.268 ± 0.012
Post hoc F0.636 ± 0.0250.173 ± 0.0310.364 ± 0.0170.473 ± 0.023
F→G0.673 ± 0.0330.219 ± 0.0550.418 ± 0.0170.515 ± 0.026
Full0.815 ± 0.0270.306 ± 0.0440.520 ± 0.0350.636 ± 0.032

Full exceeds Post hoc F by 14.44/16.67 SR points on out of view/detour tasks and F→G by 8.89/10.00 points. Feedback gains persist with initially unseen goals and indirect routes. Offline, simulation, and robot tests jointly examine plan quality, mechanism effects, and transfer.

Appendix A, Tables 4 and 27, and the key conclusions in Sections 4.4–4.5 and Appendix G.3 are highlighted in red.

[1] Habitat: A Platform for Embodied AI Research, ICCV 2019.

[2] MP3D ImageGoal Configuration and PointNav Episode Generator, Habitat-Lab GitHub, accessed 2026.

[3] NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration, ICRA 2024.

[4] Navigation World Models, CVPR 2025.

[5] Predicting Consequences and Reinforcing Navigation Policies with Latent World Models, ECCV 2026.

[6] RAE-NWM: Navigation World Model in Dense Visual Representation Space, Arxiv 2026.

[7] Policy-Guided World Model Planning for Language-Conditioned Visual Navigation, CVPR Workshops 2026.

AuthorsOpenReview ↗

Response to Reviewer umBX [2/5]

W2: The paper attributes gains to “intention-first” planning, but many compared baselines are fundamentally action-sequence or open-loop planners, while InJepa relies on iterative closed-loop feedback during candidate construction. The current experiments do not clearly isolate whether improvements come from latent intentions or from the closed-loop feedback mechanism itself.

Q2: Since InJepa uses iterative closed-loop feedback while several baselines primarily perform open-loop planning, how can the authors disentangle the benefit of latent intentions from the benefit of the feedback mechanism itself?

Response: Thank you for raising this important distinction. Two complementary controls isolate learned intentions and feedback within a candidate.

First, with Q retained, Post hoc F, F→G, and Full share weights, candidate budgets, and scoring. Only the location of predicted state feedback changes.

Table 2. Planning mechanisms of InJepa (ours). Three seeds, mean ± sample SD.

SR (%) ↑

DeploymentSeed 0Seed 1Seed 2Mean ± SD
Proposal only32.6734.6728.0031.78 ± 3.42
Post hoc F58.0055.3354.6756.00 ± 1.76
F→G62.6756.6759.3359.56 ± 3.01
F→Q+G (full)68.0062.6765.3365.33 ± 2.67
Action-first InJepa48.6740.0040.0042.89 ± 5.00

SPL ↑

DeploymentSeed 0Seed 1Seed 2Mean ± SD
Proposal only0.24260.27750.23770.2526 ± 0.0217
Post hoc F0.53590.50960.49210.5126 ± 0.0220
F→G0.57280.51510.53950.5425 ± 0.0289
F→Q+G (full)0.63570.59510.61320.6147 ± 0.0203
Action-first InJepa0.40730.33020.33680.3581 ± 0.0428

Full improves on F→G by 5.78 SR points (59.56% to 65.33%) and raises SPL from 0.5425 to 0.6147. Both feed predicted states into G. Full additionally feeds them into subsequent Q proposals.

Table 3. Intention diagnostics of InJepa (ours). Seed 0, four steps weighted equally.

Q alignment ↓

Deploymenth1h2h3h4Mean
Proposal only0.40570.52660.58810.61790.5346
Post hoc F0.40570.52660.58810.61790.5346
F→G0.40570.52660.58810.61790.5346
F→Q+G (full)0.40570.52340.57960.60410.5282

Realization error ↓

Deploymenth1h2h3h4Mean
Proposal only0.18370.32830.38960.45200.3384
Post hoc F0.17450.28930.35500.41710.3090
F→G0.17350.28270.34670.40600.3022
F→Q+G (full)0.14500.20470.24770.29160.2222

Q alignment measures the best candidate's error against the recorded future state. Realization error measures the selected branch's mismatch between intended and predicted states. The first three mechanisms leave Q recurrence unchanged, hence identical alignment. Full reduces fourth step alignment error from 0.6179 to 0.6041 and mean realization error from F→G's 0.3022 to 0.2222. Together with navigation gains, this supports feedback keeping later intentions consistent with predicted reachable states.

Second, we remove Q while retaining F recurrence. Both this fixed goal controller and Full use the same trained G/F, argmax G actions, and terminal goal score, with consistency scoring disabled. No separate random action sampler is introduced.

Table 11. Aligned G action selection in InJepa (ours). Success counts out of 150 per seed. No Q ReachSPL is a three seed mean.

ControlλcSeed 0Seed 1Seed 2SR (%) ± SD ↑
Fixed goal G→F, no Q061616140.67 ± 0.00
Full without consistency072647146.00 ± 2.91
Full1102949865.33 ± 2.67
ControlPath metricValue ↑
Fixed goal G→F, no QSPL mean ± SD0.3720 ± 0.0000
Full without consistencySPL mean ± SD0.4385 ± 0.0370
FullSPL mean ± SD0.6147 ± 0.0203

Without consistency scoring, Full succeeds in 72 / 64 / 71 tasks versus 61 / 61 / 61 without Q. Mean SR increases by 5.33 points, with gains of 7.33, 2.00, and 6.67 points across seeds. Mean ReachSPL rises from 0.372 to 0.438. Since both retain F recurrence, this supports learned local intentions beyond direct final goal conditioning.

All mechanisms replan after each executed action. The distinction is feedback within imagined candidates, not access to fresh observations after acting. These controls separate learned intentions, feedback into intentions, and consistency based scoring.

Table 11 and the key comparisons in Appendix D.1–D.2 are highlighted in red.

AuthorsOpenReview ↗

Response to Reviewer umBX [3/5]

W3: Table 1 shows that LWM-CroCo achieves better navigation performance than the proposed InJepa variants. Therefore, the experiments do not convincingly support the claim that intention-first planning is a superior planning paradigm.

Response: Thank you for highlighting this comparison. We have also narrowed the related claims in the Abstract and Introduction to the specific contribution of feeding predicted consequences into learned intention proposals within each candidate. LWM-CroCo has the highest simulation SR in Table 1. To clarify the contribution and its practical relevance, we have added matched mechanism controls and zero shot robot experiments. Together, they test the effect of learned intentions and consequence feedback with fixed model components, and their performance in physical navigation.

Table 1. Closed loop navigation. SR is Reach SR in percent and SPL is ReachSPL. Three evaluation seeds.

MethodSeed 0Seed 1Seed 2Mean ± SD
SR (%) / SPLSR (%) / SPLSR (%) / SPLSR (%) / SPL
NoMaD29.33 / 0.205226.67 / 0.173328.67 / 0.190028.22 ± 1.39 / 0.1895 ± 0.0159
RAE-NWM13.33 / 0.052215.33 / 0.061011.33 / 0.047013.33 ± 2.00 / 0.0534 ± 0.0070
LWM-CroCo71.33 / 0.686669.33 / 0.674370.00 / 0.673170.22 ± 1.02 / 0.6780 ± 0.0075
LWM-VJEPA43.33 / 0.401842.00 / 0.383842.00 / 0.353942.44 ± 0.77 / 0.3798 ± 0.0242
InJepa-DINOv3 (ours)50.00 / 0.451856.00 / 0.514353.33 / 0.492153.11 ± 3.01 / 0.4861 ± 0.0317
InJepa-CroCo (ours)62.00 / 0.561862.00 / 0.581461.33 / 0.557761.78 ± 0.38 / 0.5670 ± 0.0127
InJepa-VJEPA (ours)68.00 / 0.635762.67 / 0.595165.33 / 0.613265.33 ± 2.67 / 0.6147 ± 0.0203

The offline results in Table 4【1/5】 further show competitive planning accuracy. InJepa-DINOv3 achieves the lowest PE and OE of 0.2173 m and 0.1745 rad, respectively. InJepa-VJEPA obtains 0.2740 m PE and 0.2251 rad OE, reducing the errors by 8.2% and 4.1% relative to LWM-VJEPA.

For action selection, InJepa-VJEPA achieves 98.67% on both ACC3 and ACC5, compared with 88.00% and 79.33% for LWM-CroCo. Across three encoders, InJepa reaches 98.00–99.33% ACC3 and 96.00–98.67% ACC5, supporting reliable identification of the target reaching action sequence through consequence prediction. Together, PE/OE and ACC demonstrate accurate plan construction and reliable candidate evaluation, complementing the mechanism controls and robot navigation results.

The new robot results provide an additional system level comparison:

Table 5. Real world navigation. Three seed mean ± sample SD. SR (%). SPL omits leading zeros.

MethodStandardOut of viewDetourAll
SR (%) / SPLSR (%) / SPLSR (%) / SPLSR (%) / SPL
InJepa-VJEPA (ours)84.44 ± 2.78 / .815 ± .02735.56 ± 5.09 / .306 ± .04457.78 ± 3.85 / .520 ± .03567.65 ± 3.50 / .636 ± .032
LWM-CroCo87.56 ± 0.77 / .858 ± .00811.11 ± 1.92 / .098 ± .01753.33 ± 3.33 / .496 ± .03162.96 ± 1.28 / .608 ± .012
LWM-VJEPA46.67 ± 1.33 / .427 ± .0122.22 ± 1.92 / .017 ± .01427.78 ± 5.09 / .228 ± .04232.59 ± 2.22 / .291 ± .019
NoMaD30.67 ± 2.67 / .217 ± .01926.67 ± 3.33 / .159 ± .02012.22 ± 5.09 / .076 ± .03225.68 ± 1.54 / .173 ± .011

The comparison distinguishes system performance from the contribution of the planning mechanism. LWM-CroCo leads Clean150 and the standard robot trials. InJepa achieves the highest overall robot SR and SPL, exceeding LWM-CroCo by 4.69 SR points and 0.028 SPL, with SR gains of 24.44 points on out of view goals and 4.44 on detours. The matched controls in Tables 2 and 11【2/5】 separately establish gains from learned intentions and consequence feedback. Thus the evidence supports an effective planning architecture with strong zero shot transfer, while the best complete system depends on the setting. This is the contribution established by the comparisons, rather than a universal ranking of planning paradigms.

We have added this robot comparison. Table 5 and the corresponding result interpretation in Section 4.5 are marked in red.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.