- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Paper proposes a novel world model, GLAM, a goal-conditioned predictive approach that jointly estimates future map representations and robot-centric waypoint latents from global spatiotemporal memory. The design is functionally motivated by cognitive-map accounts that connect structured spatial memory with prospective navigation.
A complementary end-to-end navigation system, GLAM NAV, combines a High-Level Planning Module, spatiotemporal memory, GLAM, and an Action Expert. The High-Level Planning Module performs perception, mapping, retrieval, and goal checking. The GLAM predictive world model produces future map tokens and a latent plan, and the Action Expert grounds a retrieved target or predicted frontier into reachable motion. Only real observations and poses update spatiotemporal memory.
Training uses oracle-guided Object Nav trajectories collected in Habitat over HM3D scene assets. The GLAM NAV system was evaluated in simulation on the HM3D ObjectNav benchmark. Success Rate(SR) and Success weighted byPath Length(SPL) were selected as the comparison metrics. The proposed system significantly outperforms 10 other SOTA methods for the same benchmark on both SR and SPL.
Ablation studies were done on GLAM NAV to investigate the effects of with additional interventions.
Critical strengths
- Paper proposes a novel world model, GLAM, which is triggered when the main navigation system cannot retrieve relevant memory or input observation to ground the execution plan. The GLAM will then produces future map tokens (prediction) and a latent plan to guide the potential execution plan based on the goal and prediction.
- Experimental results outperform many SOTA methods significantly.
Critical weaknesses
- (minor) Novelty is incremental as it borrows heavily from JEPA-like latent-space prediction paradigm.
- The experiment setup is singular on only the HM3D ObjectNav benchmark with direct reference only to BSC-Nav. Other SOTA methods results are taken their full-benchmark reports under HM3D-Semantics v0.2 and cannot serve as direct comparison. (Credits to authors for acknowledging this in their Limitations section.
- (minor) The purpose, motivation and key findings of the ablation study are not well-explained.
Questions for authors
Refer to Weakness 3, please explain the purpose, motivation and key findings of the ablation study.
5 comments
Weakness2
Thank you for highlighting the evaluation scope and comparability concerns. We agree that the current quantitative navigation evaluation is limited to HM3D ObjectNav. We would also like to clarify the coverage of the main results.
- The main evaluation covers all 1,000 episodes.
The GLaM Nav results in Table 1 are evaluated on the complete 1,000-episode HM3D-Semantics v0.2 evaluation set. The 100-episode ablations and the 60-episode trajectory showcase are separate supplementary studies. Literature results reported on the same benchmark version provide meaningful benchmark-level comparisons, while their provenance and evaluation settings should be stated explicitly.
- We will distinguish reproduced results from literature reports.
BSC-Nav is reproduced using its official implementation on the same episode list as GLaM Nav. The other Table 1 entries are taken from published reports. We will annotate each row with its dataset version, evaluation scope, and numerical source, and document differences in input modalities, initial scene memory, and planning/control components.
- We will distinguish system comparison from model attribution.
The main table evaluates complete systems. The contribution of GLaM’s predictive design is examined more directly through the default-frontier versus waypoint-guidance comparison and the architecture-matched joint versus waypoint-only training ablation. We will state the configuration and task set of each comparison separately.
We will retain the limitations of a single benchmark and a limited number of independently reproduced baselines, and align our claims with the evidence provided. Future work will extend evaluation to additional navigation benchmarks and unseen environments, with more directly evaluated baselines under clearly specified protocols, to assess cross-dataset and cross-environment generalization.
Question1-Weakness3
Thank you for the suggestion. The ablations are intended to answer two connected questions: whether predictive learning improves exploration, and how retrieval, localization, and execution affect the resulting navigation performance. We agree that the original presentation should explain the motivation and findings of each comparison more clearly.
Table 2 uses a dedicated 4B configuration on 100 tasks from four HM3D scenes. The additional training ablation is reported separately on 100 development tasks. We will organize the discussion around four questions.
- Does predictive guidance improve exploration when memory cannot localize the goal?
Frontier selection is central to semantic exploration, as illustrated by VLFM. We test whether GLaM’s predicted waypoints provide useful guidance within our navigation system. Replacing default frontier selection with waypoint guidance changes SR/SPL from 77.00% / 29.45% to 83.00% / 28.79%.
The finding is that predictive guidance helps complete more tasks, but does not simultaneously improve SPL in this diagnostic setting. This comparison tests the deployment-time contribution of the predictor.
- With the predictor fixed, do retrieval timing and additional execution checks improve performance?
The motivation is to determine whether better use of new observations or more restrictive execution rules improves how predictive proposals are used.
| Configuration | SR (%) | SPL (%) |
|---|---|---|
| Waypoint-guided reference | 83.00 | 28.79 |
| Retrieve after arrival and scanning | 83.00 | 30.83 |
| Path-cost gating | 83.00 | 28.63 |
| Additional RGB STOP check | 72.00 | 23.39 |
Retrieval after arrival and scanning improves SPL at unchanged SR in this comparison. Additional cost constraints or STOP verification do not necessarily help. These results motivate attention to retrieval timing and execution decisions, rather than assuming that adding checks will improve navigation.
- Do target reachability and candidate scoring affect navigation?
These diagnostics examine whether the retrieved location is executable and whether candidate ranking preserves useful spatial evidence. They use a separate reference configuration with revised cluster scoring and failed-target handling:
| Configuration | SR (%) | SPL (%) |
|---|---|---|
| Revised localization reference | 82.00 | 27.82 |
| Ground the target at a reachable observation point | 84.00 | 28.77 |
| Pair-count normalization with minimum-support filtering | 49.00 | 15.62 |
The findings support reachable target grounding and show substantial sensitivity to the candidate-scoring configuration. The normalized variant changes both scoring and support filtering, so the result applies to that complete variant. Aggregate metrics alone do not establish the detailed failure mechanism.
- Does future-map supervision add value beyond waypoint-only training?
This additional experiment directly tests the motivation for joint prediction. We retain the same architecture and training budget and disable only future-map supervision. Joint training lowers waypoint latent MSE from 0.15026 to 0.14853 on the same 494 offline validation transitions. On the same 100 development tasks, it improves SR from 79.00% to 83.00% and SPL, on the 0–1 scale, from 0.27159 to 0.28786.
This provides direct evidence that future-map supervision benefits both waypoint prediction and downstream navigation in the evaluated configuration.
We will state the research question, reference configuration, and evaluation scope for each group. The interventions are evaluated independently, and improvements across groups should not be added together. This organization makes clear which findings concern predictive learning and which concern the surrounding navigation system.
Weakness1
Thank you for the comment. We agree that prediction in representation space is an established learning principle, and its JEPA origin is acknowledged in Section 4.1. GLaM’s contribution lies in the formulation of goal-conditioned joint prediction over online spatial memory and its connection to navigation.
The specific design has three parts:
Spatially organized prediction state. The original I-JEPA and V-JEPA organize prediction targets over image regions and video space-time locations, respectively. GLaM uses voxel memory obtained by spatially aggregating observations across viewpoints using depth and pose. The prediction context therefore represents accumulated spatial observations.
Joint prediction for navigation. Conditioned on historical spatial memory, the navigation goal, and the current pose, GLaM jointly predicts future map representations and robot-centric waypoint latents. Training aligns the memory states before and after motion with the corresponding displacement. Future-map supervision consequently constrains the shared model to learn future spatial state alongside the navigation proposal.
Connection to execution. A pretrained, frozen waypoint encoder-decoder provides latent targets during training and converts predicted latents into waypoints at deployment. GLaM Nav uses these waypoints to guide frontier selection, connecting the joint prediction task to closed-loop exploration.
The architecture- and budget-matched training ablation supports the practical value of this joint design. On the same 100 development tasks, joint supervision achieves 83.00% SR and 0.28786 SPL, compared with 79.00% and 0.27159 for waypoint-only supervision. This is evidence for the benefit of future-map supervision in the evaluated setting.
We will sharpen the contribution statement around spatial-memory organization, the goal-conditioned joint prediction task, and the waypoint interface, while clearly crediting the inherited learning principle. The cognitive-map connection will remain a functional motivation for the design.