acceptodds
Under review as a conference paper at ICLR 2027

GLAM: TRAINING A LATENT WORLD MODEL OVER GLOBAL SPATIOTEMPORAL MEMORY FOR ACTIVE EXPLORATION AND NAVIGATION

Abstract

Active exploration and semantic navigation require an embodied agent to build memory from partial observations, predict how the evolution of observed spatial memory may support future motion, and convert that prediction into actionable plans. We present GLAM, a goal-conditioned latent world model trained over global spatiotemporal memory, and GLAM NAV, the complete navigation system built around it. Given historical map tokens, a navigation goal, and the current robot pose, GLAM jointly predicts future map representations and robot-centric waypoint latents, allowing future spatial context and navigation intent to be inferred in a shared representation space. The model follows a JEPA-like latent prediction paradigm, operates directly on map-level latent tokens rather than RGB reconstruction, and uses a pretrained waypoint encoder–decoder to supervise and decode navigation plans within GLAM NAV. Training data are collected in Habitat over HM3D v0.2 by sampling oracle-guided ObjectNav trajectories and their map transitions. On the full HM3D-Semantics v0.2 ObjectNav benchmark, GLAM NAV achieves a state-of-the-art success rate of 82.30% with 34.00% success weighted by path length.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world model, embodied ai, navigation, cogtive-map

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZQNbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper proposes a novel world model, GLAM, a goal-conditioned predictive approach that jointly estimates future map representations and robot-centric waypoint latents from global spatiotemporal memory. The design is functionally motivated by cognitive-map accounts that connect structured spatial memory with prospective navigation.

A complementary end-to-end navigation system, GLAM NAV, combines a High-Level Planning Module, spatiotemporal memory, GLAM, and an Action Expert. The High-Level Planning Module performs perception, mapping, retrieval, and goal checking. The GLAM predictive world model produces future map tokens and a latent plan, and the Action Expert grounds a retrieved target or predicted frontier into reachable motion. Only real observations and poses update spatiotemporal memory.

Training uses oracle-guided Object Nav trajectories collected in Habitat over HM3D scene assets. The GLAM NAV system was evaluated in simulation on the HM3D ObjectNav benchmark. Success Rate(SR) and Success weighted byPath Length(SPL) were selected as the comparison metrics. The proposed system significantly outperforms 10 other SOTA methods for the same benchmark on both SR and SPL.

Ablation studies were done on GLAM NAV to investigate the effects of with additional interventions.

Critical strengths
  1. Paper proposes a novel world model, GLAM, which is triggered when the main navigation system cannot retrieve relevant memory or input observation to ground the execution plan. The GLAM will then produces future map tokens (prediction) and a latent plan to guide the potential execution plan based on the goal and prediction.
  2. Experimental results outperform many SOTA methods significantly.
Critical weaknesses
  1. (minor) Novelty is incremental as it borrows heavily from JEPA-like latent-space prediction paradigm.
  2. The experiment setup is singular on only the HM3D ObjectNav benchmark with direct reference only to BSC-Nav. Other SOTA methods results are taken their full-benchmark reports under HM3D-Semantics v0.2 and cannot serve as direct comparison. (Credits to authors for acknowledging this in their Limitations section.
  3. (minor) The purpose, motivation and key findings of the ablation study are not well-explained.
Questions for authors

Refer to Weakness 3, please explain the purpose, motivation and key findings of the ablation study.

5 comments

Thank you for the comment. We agree that prediction in representation space is an established learning principle, and its JEPA origin is acknowledged in Section 4.1. GLaM’s contribution lies in the formulation of goal-conditioned joint prediction over online spatial memory and its connection to navigation.

The specific design has three parts:

  1. Spatially organized prediction state. The original I-JEPA and V-JEPA organize prediction targets over image regions and video space-time locations, respectively. GLaM uses voxel memory obtained by spatially aggregating observations across viewpoints using depth and pose. The prediction context therefore represents accumulated spatial observations.

  2. Joint prediction for navigation. Conditioned on historical spatial memory, the navigation goal, and the current pose, GLaM jointly predicts future map representations and robot-centric waypoint latents. Training aligns the memory states before and after motion with the corresponding displacement. Future-map supervision consequently constrains the shared model to learn future spatial state alongside the navigation proposal.

  3. Connection to execution. A pretrained, frozen waypoint encoder-decoder provides latent targets during training and converts predicted latents into waypoints at deployment. GLaM Nav uses these waypoints to guide frontier selection, connecting the joint prediction task to closed-loop exploration.

The architecture- and budget-matched training ablation supports the practical value of this joint design. On the same 100 development tasks, joint supervision achieves 83.00% SR and 0.28786 SPL, compared with 79.00% and 0.27159 for waypoint-only supervision. This is evidence for the benefit of future-map supervision in the evaluated setting.

We will sharpen the contribution statement around spatial-memory organization, the goal-conditioned joint prediction task, and the waypoint interface, while clearly crediting the inherited learning principle. The cognitive-map connection will remain a functional motivation for the design.

Thank you for highlighting the evaluation scope and comparability concerns. We agree that the current quantitative navigation evaluation is limited to HM3D ObjectNav. We would also like to clarify the coverage of the main results.

  1. The main evaluation covers all 1,000 episodes.

The GLaM Nav results in Table 1 are evaluated on the complete 1,000-episode HM3D-Semantics v0.2 evaluation set. The 100-episode ablations and the 60-episode trajectory showcase are separate supplementary studies. Literature results reported on the same benchmark version provide meaningful benchmark-level comparisons, while their provenance and evaluation settings should be stated explicitly.

  1. We will distinguish reproduced results from literature reports.

BSC-Nav is reproduced using its official implementation on the same episode list as GLaM Nav. The other Table 1 entries are taken from published reports. We will annotate each row with its dataset version, evaluation scope, and numerical source, and document differences in input modalities, initial scene memory, and planning/control components.

  1. We will distinguish system comparison from model attribution.

The main table evaluates complete systems. The contribution of GLaM’s predictive design is examined more directly through the default-frontier versus waypoint-guidance comparison and the architecture-matched joint versus waypoint-only training ablation. We will state the configuration and task set of each comparison separately.

We will retain the limitations of a single benchmark and a limited number of independently reproduced baselines, and align our claims with the evidence provided. Future work will extend evaluation to additional navigation benchmarks and unseen environments, with more directly evaluated baselines under clearly specified protocols, to assess cross-dataset and cross-environment generalization.

AuthorsOpenReview ↗

Question1-Weakness3

Thank you for the suggestion. The ablations are intended to answer two connected questions: whether predictive learning improves exploration, and how retrieval, localization, and execution affect the resulting navigation performance. We agree that the original presentation should explain the motivation and findings of each comparison more clearly.

Table 2 uses a dedicated 4B configuration on 100 tasks from four HM3D scenes. The additional training ablation is reported separately on 100 development tasks. We will organize the discussion around four questions.

  1. Does predictive guidance improve exploration when memory cannot localize the goal?

Frontier selection is central to semantic exploration, as illustrated by VLFM. We test whether GLaM’s predicted waypoints provide useful guidance within our navigation system. Replacing default frontier selection with waypoint guidance changes SR/SPL from 77.00% / 29.45% to 83.00% / 28.79%.

The finding is that predictive guidance helps complete more tasks, but does not simultaneously improve SPL in this diagnostic setting. This comparison tests the deployment-time contribution of the predictor.

  1. With the predictor fixed, do retrieval timing and additional execution checks improve performance?

The motivation is to determine whether better use of new observations or more restrictive execution rules improves how predictive proposals are used.

ConfigurationSR (%)SPL (%)
Waypoint-guided reference83.0028.79
Retrieve after arrival and scanning83.0030.83
Path-cost gating83.0028.63
Additional RGB STOP check72.0023.39

Retrieval after arrival and scanning improves SPL at unchanged SR in this comparison. Additional cost constraints or STOP verification do not necessarily help. These results motivate attention to retrieval timing and execution decisions, rather than assuming that adding checks will improve navigation.

  1. Do target reachability and candidate scoring affect navigation?

These diagnostics examine whether the retrieved location is executable and whether candidate ranking preserves useful spatial evidence. They use a separate reference configuration with revised cluster scoring and failed-target handling:

ConfigurationSR (%)SPL (%)
Revised localization reference82.0027.82
Ground the target at a reachable observation point84.0028.77
Pair-count normalization with minimum-support filtering49.0015.62

The findings support reachable target grounding and show substantial sensitivity to the candidate-scoring configuration. The normalized variant changes both scoring and support filtering, so the result applies to that complete variant. Aggregate metrics alone do not establish the detailed failure mechanism.

  1. Does future-map supervision add value beyond waypoint-only training?

This additional experiment directly tests the motivation for joint prediction. We retain the same architecture and training budget and disable only future-map supervision. Joint training lowers waypoint latent MSE from 0.15026 to 0.14853 on the same 494 offline validation transitions. On the same 100 development tasks, it improves SR from 79.00% to 83.00% and SPL, on the 0–1 scale, from 0.27159 to 0.28786.

This provides direct evidence that future-map supervision benefits both waypoint prediction and downstream navigation in the evaluated configuration.

We will state the research question, reference configuration, and evaluation scope for each group. The interventions are evaluated independently, and improvements across groups should not be added together. This organization makes clear which findings concern predictive learning and which concern the surrounding navigation system.

Reviewer_saRkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents GLAM, a goal-conditioned latent world model that jointly predicts future map representations and robot-centric waypoint latents from accumulated spatial memory. Trained on oracle-guided ObjectNav trajectories, GLAM is integrated into GLAM NAV, a navigation system combining persistent memory, target retrieval, and motion planning. On HM3D-Semantics v0.2, the system improves navigation success rate over the reported baselines, although with lower path efficiency than BSC-Nav. The paper also includes diagnostic ablations and qualitative real-world demonstrations of exploration and memory reuse.

Critical strengths

A key strength is GLAM’s joint prediction of future spatial-memory representations and goal-conditioned waypoint latents, which extends semantic navigation beyond retrieving previously observed information toward anticipating future spatial context. Predicting in latent space without reconstructing future RGB observations is an interesting design, conceptually aligned with the cognitive-map motivation. I also appreciate the real-world deployment on an Astribot S1 alongside the simulation experiments. The demonstrations of online exploration and subsequent memory-guided return navigation provide encouraging qualitative evidence that the integrated system can operate with real sensing and motion, supporting its practical applicability.

Critical weaknesses

The experimental evaluation provides insufficient detail to understand GLAM NAV’s performance and its advantages over existing methods. The main quantitative comparison is concentrated in Table 1. Although the authors explicitly match the evaluation episodes and protocol for their BSC-Nav reproduction, comparisons with other baselines rely on published results without sufficiently detailing differences in sensing, controllers, and experimental settings. This makes it difficult to assess how much of the performance difference comes from the proposed predictive model rather than the surrounding system. Moreover, GLAM NAV improves SR over BSC-Nav from 78.50% to 82.30%, but substantially reduces SPL from 47.70% to 34.00%. Without comparative trajectory visualizations or failure-case analysis, it remains unclear whether this trade-off results from broader exploration, repeated revisiting, inefficient waypoint selection, or other planning behaviors. Such analyses would help establish when GLAM NAV is beneficial and explain the substantial loss in path efficiency.

Questions for authors
  1. Beyond the matched BSC-Nav reproduction, could the authors clarify how the baselines in Table 1 differ from GLAM NAV in input modalities, evaluation settings, and planning or control components? Which comparisons best isolate the contribution of GLAM’s predictive model?
  2. Could the authors provide representative trajectory visualizations from existing evaluation runs, comparing GLAM NAV with BSC-Nav? Examples illustrating both successes and failures would help explain why the higher SR is accompanied by substantially lower SPL.
  3. Table 2 reports a substantial performance drop with support-normalized cluster scoring, but neither the original cluster score nor its support-normalized variant is defined in the method section. Please specify how clusters are constructed, what constitutes support, and how the two scores are computed, and clarify whether these rules are introduced here or adopted from prior work.
6 comments

Thank you for highlighting this issue. We clarify the main differences below and will incorporate the configuration and result-source information into the revised comparison.

  1. Input representations and navigation components.
MethodObservations and representationPlanning or execution
GLaM NavRGB-D and pose; online Qwen-feature voxel memoryGoal retrieval, predicted-waypoint frontier guidance, and Action Expert execution
BSC-NavRGB-D and spatial memory; DINOv2 voxel featuresPrior memory construction, candidate retrieval and verification, and simulator geometry-based navigation
L3MVNRGB-D and pose; semantic mapLanguage-guided frontier selection and FMM local planning
SG-NavRGB-D and pose; online 3D scene graphGraph reasoning, frontier selection, and re-perception
VLFMRGB-D; language-conditioned value mapFrontier selection and a local PointNav policy
WMNav / MerNavRGB-D and pose; visual and local geometric informationVLM decisions with polar local-action interfaces
SkillNavCuriosity value mapScore-level behavioral skill intervention
Uni-NaVidRGB historyVideo-conditioned discrete-action prediction
Qwen-RobotNav-4B / 8BRGB historyWaypoint-trajectory prediction and execution
  1. Evaluation settings and numerical sources.

The main GLaM Nav result covers the complete 1,000-episode HM3D-Semantics v0.2 evaluation list used in our study. BSC-Nav is independently reproduced on the same list; the other rows contain literature-reported results. Sharing tasks does not imply identical perception, prior scene information, or control components.

  1. Comparisons that isolate the predictive model.

Two internal controls provide the most direct evidence:

  • Deployment-time guidance: Within the separate Table 2 configuration, replacing default frontier selection with GLaM waypoint guidance changes SR/SPL from 77.00%/29.45% to 83.00%/28.79%. This tests whether the predictor improves exploration decisions.
  • Future-map supervision: The architecture-matched training ablation retains waypoint prediction and changes the future-map loss weight. On the same 100 development tasks, joint supervision improves SR from 79.00% to 83.00% and SPL from 0.27159 to 0.28786, while also reducing offline waypoint errors.

We will distinguish the configurations and task sets of these two controls from the main evaluation. Together, they provide more direct evidence for the predictive design than cross-system comparisons, whose differences also include perception, memory, and execution components.

Thank you for this suggestion. We have added paired trajectory visualizations covering both successful and failed executions. The anonymous companion page provides static plots and an interactive viewer for all 60 development tasks. Each comparison uses the same scene, goal category, starting position, and initial heading. The plots show recorded executed positions, with task outcomes, path lengths, and SPL.

  1. Representative successes and failures.

We include examples covering all four outcome combinations:

EpisodeGLaM NavBSC-NavWhat the comparison shows
T018, sofaSuccess; 32.13 m; SPL 0.0942Success; 7.95 m; SPL 0.3806Both succeed, but GLaM Nav takes a substantially longer route.
T004, toiletSuccess; 35.25 mFailure; 6.75 mA shorter executed path does not necessarily satisfy the success condition.
T002, bedFailure after exhausting the time budgetSuccess; 7.24 mGLaM Nav can also fail to complete a task that BSC-Nav solves efficiently.
T013, chairFailureFailureThe visualization set also includes failures shared by both systems.

T018 directly illustrates the path-efficiency limitation: both systems find the target, but GLaM Nav travels farther and consequently receives lower SPL. In T004, BSC-Nav issues STOP at a geodesic distance of 1.251 m from the nearest valid goal viewpoint, outside the <1.0 m success threshold in this evaluation; GLaM Nav completes the task with a longer trajectory. These examples distinguish successful completion from short but unsuccessful execution.

  1. Aggregate analysis of the paired tasks.

Across the 60 tasks, 25 are solved by both methods, 18 only by GLaM Nav, 8 only by BSC-Nav, and 9 by neither.

MetricGLaM NavBSC-Nav
Successful tasks43/6033/60
SR (%)71.6755.00
Overall SPL0.27860.2814
Mean SPL on the 25 common successes0.38210.4829

Thus, GLaM Nav completes more tasks in this set, while BSC-Nav has higher average path efficiency on the tasks both methods solve. This separates two aspects of performance that overall SR alone cannot capture. The visualizations make the distinction concrete through examples of longer successful routes, unsuccessful termination, and budget exhaustion.

  1. Scope of the evidence.

These 60 fixed development tasks come from 10 training-seen scenes and are reported separately from the full 1,000-episode evaluation in Table 1. They illustrate the success–efficiency trade-off at the trajectory level, rather than numerically decomposing the entire main-table SPL gap. The traces reveal executed-path and termination differences, but do not by themselves establish whether each additional movement was caused by recognition errors, repeated exploration, or candidate switching.

We will incorporate these paired examples and the common-success analysis into the revision to make both the task-completion gains and the remaining path-efficiency limitation clearer.

Thank you for identifying this omission. The score is used for memory retrieval and goal localization in the High-Level Planning Module. We will add the cluster construction, support definition, scoring equations, and implementation attribution to the method section.

  1. Candidate clusters and support.

Each retrieved map entry has a 3D position and an effective nonnegative goal-related attention weight, after the shared attention preprocessing and failure-history down-weighting. Entries with weights at or above the median serve as candidate centers. Around each center, we form a radius-r neighborhood, with r = 0.6 m. If it contains more than 40 entries, we retain the 40 with the highest center-entry pair energy, defined below. These neighborhoods may overlap.

For a retained candidate cluster C, support is

nC=∣C∣.n_C=|C|.

Thus, support counts retained map entries, rather than independent viewpoints or repeated observations.

  1. Cumulative and normalized scores.

Let x denote the entry position and w its effective attention weight. The pair energy combines goal relevance with spatial proximity:

bij=wiwjexp⁡(−3∥xi−xj∥2r).b_{ij}=w_iw_j\exp\left(-3\frac{\lVert\mathbf{x}_i-\mathbf{x}_j\rVert_2}{r}\right).

The revised cumulative reference score is

Ssum(C)=∑i,j∈C,;i≤jbij.S_{\mathrm{sum}}(C)=\sum_{i,j\in C,\\;i\leq j}b_{ij}.

Each unordered pair is counted once, including self-pairs. The normalized score divides by the corresponding number of pairs:

Snorm(C)=Ssum(C)nC(nC+1)/2.S_{\mathrm{norm}}(C)=\frac{S_{\mathrm{sum}}(C)}{n_C(n_C+1)/2}.

It therefore measures mean pair energy. The selected target location is the center of the highest-scoring candidate.

The evaluated normalized configuration additionally uses conditional minimum-support filtering: if any candidate contains at least 3 entries, only candidates meeting this threshold are ranked; otherwise, all nonempty candidates remain eligible.

  1. Origin of the scoring rule.

The High-Level Planning Module inherits the candidate-neighborhood construction and distance-decayed pair energy from the underlying goal-localization implementation. For the Table 2 reference configuration, we standardized the sum so that each unordered pair, including self-pairs, contributes once. The normalized variant divides this sum by the number of such pairs and applies the conditional minimum-support filter described above. The GLaM model parameters are identical in both configurations.

  1. Interpretation of the performance drop.

Cumulative energy preserves the contribution of support size and spatial consistency, whereas mean energy can favor a smaller region with a stronger average response. A replay of a saved localization input illustrates this ranking change:

CandidateSupportCumulative energyMean pair energy
A322.080890.00394
B50.240380.01603

Both candidates satisfy the minimum-support threshold, but cumulative scoring selects A and normalized scoring selects B. This demonstrates that normalization can change the selected location even with the same spatial memory and attention weights; it does not establish which candidate is the true target.

On the same 100 diagnostic tasks with model parameters fixed, the full normalized configuration reduces SR/SPL from 82.00% / 27.82% to 49.00% / 15.62%. This supports retaining cumulative evidence in the evaluated localization setting. Because the variant also includes support filtering, the aggregate drop cannot be attributed to the normalization denominator alone. We will label it “pair-count-normalized cluster energy with conditional minimum-support filtering” and provide the complete definitions.

Reviewer_R6NoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GLAM, a goal-conditioned latent world model that jointly predicts future spatial map representations and navigation waypoints from historical spatial memory. The model is trained on oracle-guided trajectories using latent regression objectives. The complete GLAM-NAV system integrates this predictive model with spatial memory, high-level planning, and low-level navigation control. On HM3D ObjectNav, GLAM-NAV achieves 82.3% success rate, outperforming existing methods in this metric. The paper also includes planner-level ablations and qualitative real-world demonstrations.

Critical strengths
  1. An interesting direction for predictive spatial navigation. Learning future spatial representations directly in latent space, rather than reconstructing images, is a reasonable approach. Jointly predicting map representations and waypoints is potentially useful for goal-directed exploration.

  2. The method achieves the highest reported success rate on the full HM3D ObjectNav benchmark.

Critical weaknesses
  1. The contribution of future-map prediction is not sufficiently demonstrated. The central idea is to jointly predict future map representations and waypoints. However, it is unclear how future-map predictions actually contribute to navigation decisions. Sec 5.4 states that the predicted map tokens do not directly score exploration frontiers, and the ablation only evaluates the effect of waypoint guidance. Consequently, the experiments do not establish whether future-map prediction is necessary or whether similar performance could be achieved by directly predicting waypoints. This is particularly important because the model is trained on oracle-guided trajectories and does not condition on candidate future actions. The role of the proposed world model in planning therefore remains insufficiently validated.

  2. The empirical improvements are difficult to attribute to the proposed model, and navigation efficiency is considerably worse. Although GLAM-NAV improves success rate over BSC-Nav from 78.5% to 82.3%, SPL drops substantially from 47.7% to 34.0%. This is a significant trade-off for a navigation method. Moreover, the ablations are conducted on only 100 episodes from four scenes using a separate 4B model, rather than the configuration used for the main evaluation. They do not isolate the contribution of GLAM from the other components of the navigation system on the full benchmark. The real-world evaluation is also limited to qualitative demonstrations, providing little evidence of robustness or generalization.

Questions for authors
  1. Can the authors provide an ablation that removes the future-map prediction objective while retaining waypoint prediction, using the same backbone and training budget? This would help establish whether joint future-map prediction is actually beneficial.

  2. How are the predicted future map representations used to improve exploration decisions? Since the model is not conditioned on candidate future actions, what planning capability does it provide beyond direct goal-conditioned waypoint prediction?

  3. Why does GLAM-NAV achieve substantially lower SPL than BSC-Nav despite its higher success rate?

  4. Can the authors clarify the main model configuration and training hyperparameters, and provide quantitative real-world results or additional evaluation under trajectories that deviate from oracle demonstrations?

6 comments

Thank you for identifying this important control. We added a training ablation that removes future-map supervision while retaining waypoint prediction and the same model architecture.

Both models use Qwen3-VL-4B, retain the map and waypoint queries, and share the initialization, training data and sample order, optimization settings, frozen waypoint encoder–decoder, and a budget of 3,000 parameter updates. The training objective is

L=Lwp+λmapLmap\mathcal{L} = \mathcal{L}_{\mathrm{wp}} + \lambda_{\mathrm{map}}\mathcal{L}_{\mathrm{map}} , We set λmap=1\lambda_{\mathrm{map}}=1 for joint supervision and λmap=0\lambda_{\mathrm{map}}=0 for waypoint-only supervision.

The map term uses masked MSE, while the waypoint term averages MSE over valid latent targets, corresponding to the all-valid waypoint-mask case in Eqs. (4)–(5). The waypoint-loss weight is fixed at 1. Thus, the intervention isolates the additional future-map supervision without removing a prediction branch or changing the inference architecture.

MetricJoint supervisionWaypoint-only supervision
Validation waypoint latent MSE ↓0.148530.15026
Validation XY position error, m ↓0.712060.73356
Validation heading MAE, degrees ↓32.6633.48
SR on 100 development tasks, % ↑83.0079.00
SPL on 100 development tasks, 0–1 ↑0.287860.27159

The first three metrics are computed on the same 494 offline validation transitions. The navigation metrics are obtained through closed-loop evaluation on the same 100 development tasks. Joint supervision produces modest improvements in offline prediction accuracy and increases navigation SR by 4.00 percentage points and SPL by 0.01627.

These results provide direct evidence that future-map supervision benefits both waypoint prediction and downstream navigation in this controlled setting. We will report this training ablation separately from the Table 2 system diagnostics and the Table 1 main evaluation, with the model configurations and task populations explicitly identified.

Thank you for asking us to clarify how prediction affects exploration. Future-map prediction contributes primarily through auxiliary supervision during training; decoded waypoints provide the direct interface to exploration at deployment.

During training, GLaM jointly predicts goal-conditioned waypoint latents and the spatial-memory representations obtained after the corresponding navigation displacement. As described in Section 4.2, the training samples align the memory before motion, the memory after arrival, and the associated displacement. The two prediction objectives therefore share the same navigation context. Future-map supervision adds a constraint on the future spatial state while the model learns to propose motion.

At deployment, the decoded waypoint guides candidate-frontier selection, while the Action Expert handles reachability and motion execution. Predicted future-map tokens do not directly score frontiers and are not inserted into memory as observed evidence. Memory updates and goal confirmation remain grounded in actual observations. The spatial memory is accumulated online; it does not provide prior access to the complete room layout.

We agree that GLaM is not conditioned on candidate future action sequences and therefore does not perform counterfactual action-sequence simulation or search. Its capability is goal-conditioned exploration guidance learned with future spatial supervision. The benefit beyond waypoint-only training is evaluated by the architecture-matched ablation in our response to your question 1, which shows improvements in both prediction accuracy and closed-loop navigation.

We will clarify this distinction between the training objective and the deployment interface, and describe GLaM’s role as predictive exploration guidance rather than action-sequence search.

Thank you for highlighting this trade-off. In Table 1, GLaM Nav increases SR from 78.50% to 82.30% relative to BSC-Nav, while SPL decreases from 47.70% to 34.00%. This indicates improved task completion, with a substantial remaining limitation in successful-path efficiency.

SPL combines success and path efficiency:

SPL=1N∑i=1NSiLimax⁡(Li,Pi)=SR×η‾success\mathrm{SPL} = \frac{1}{N}\sum_{i=1}^{N} S_i\frac{L_i}{\max(L_i,P_i)} = \mathrm{SR}\times\overline{\eta}_{\mathrm{success}}

Here, N is the number of evaluated episodes. For each episode, the success indicator is 1 if navigation succeeds and 0 otherwise. The path-efficiency ratio is the shortest-path length divided by the larger of the shortest-path and executed-path lengths. SR is expressed as a fraction between 0 and 1, and the final factor denotes the mean path efficiency over successful episodes. Higher SR can therefore coexist with lower SPL when successful routes are less efficient.

  1. Initial scene information and path-cost accounting differ. The published BSC-Nav pipeline constructs scene memory through pre-exploration. GLaM Nav instead acquires observations and builds spatial memory during the navigation task, so its exploratory motion contributes to the evaluated path. In our supplementary BSC-Nav evaluation, prior memory-construction cost is excluded from per-episode metrics. This distinction provides context for the comparison, although it does not by itself quantify the full SPL gap or establish a lower total deployment cost for GLaM Nav.

  2. Paired trajectories show broader success coverage alongside less efficient successful routes. On a separately reported set of 60 fixed development tasks from 10 training-seen scenes, GLaM Nav and BSC-Nav succeed on 43 and 33 tasks, respectively, with overall SPL of 0.2786 and 0.2814. On the 25 tasks that both methods solve, their mean SPL is 0.3821 and 0.4829, respectively. This directly illustrates the remaining path-efficiency limitation even when both systems reach a valid goal.

Representative successes and failures are available on our anonymous companion page(mentioned that you have to turn the JS on). This supplementary BSC-Nav run uses a substituted vision-language endpoint and is documented separately from the main-table evaluation. These development results provide behavioral evidence, rather than a numerical decomposition of the full Table 1 gap.

  1. Internal ablations identify relevant execution choices. In Table 2, adding waypoint guidance increases SR from 77.00% to 83.00%, while SPL changes from 29.45% to 28.79%. With the predictor fixed, performing memory retrieval after arrival and scanning increases SPL to 30.83%, with SR remaining 83.00%. These results indicate that useful exploration proposals must be coupled with effective retrieval and execution to produce efficient routes.

We will explicitly present both the benefit of successful online navigation without prebuilt scene memory and the remaining limitation in path efficiency, while avoiding attribution of the entire gap to a single recognition or planning component.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.