acceptodds
Under review as a conference paper at ICLR 2027

From Latent Loops to Better Navigation: Understanding Diffusion-Based World Models with Cognitive-Map-Inspired Dynamics

Abstract

Diffusion-based world models have recently demonstrated strong visual navigation performance without explicitly constructing maps, yet the latent mechanisms behind this capability remain unclear. We analyze diffusion-based navigation world models through the lens of cognitive map theory. Using persistent homology, we find that these models develop low-dimensional latent spatial structures during navigation generation, suggesting the emergence of cognitive-map-like representations. However, these structures are unstable across layers and long-horizon autoregressive rollouts, which may contribute to accumulated prediction drift. Motivated by this observation, we propose a brain-inspired framework that augments a frozen diffusion world model with a compositional cognition module. This module introduces structured, action-conditioned grid–place dynamics as an explicit spatial prior to stabilize latent spatial representations and improve long-horizon consistency. Experiments on real-world navigation datasets show improved prediction, navigation, and representation stability over strong diffusion-based baselines. Component ablations support the contribution of structured grid–place conditioning, while correlation analysis reveals an association between topological persistence and trajectory accuracy.

Primary area:
applications to neuroscience & cognitive science
Keywords:
cognitive map, world model, diffusion model, brain-inspired

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TtwwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigated the representational geometry of diffusion-based world model activation (NWM in particular) across different time steps in a trajectory and found a loop-like structure, which the authors interpreted as a navigation-relevant learned spatial structure. Motivated by the instability of loop-like structure across layers, a path integration component relying on structured transitions between multi-scale grid modules was introduced. The modification improved the stability of loop-like structure across layers and improved the generation and navigation performance over the baseline. The necessity of each proposed component is demonstrated through ablations.

Critical strengths
  1. The result of persistent homology analysis provides an interesting perspective on the representational geometry supporting diffusion-based world models' navigation capacities.
  2. The proposed architecture achieves consistent improvements in visual prediction and navigation performance over the original NWM across multiple real-world navigation datasets. The improvements in long-horizon prediction are particularly encouraging.
Critical weaknesses

Major Weakness

1. Forced connection between latent loop structure and “cognitive maps”.

The latent loop could reflect visually contiguous features across trajectory time steps rather than spatial structure. Although the authors use Fig. 3 to support spatial encoding, their perturbation disrupts both spatial and visual continuity, making it impossible to disentangle the two explanations. Moreover, visual encoding could equally explain the correlation between latent-loop structure and navigation performance: degraded visual representations could simultaneously weaken the loop structure and impair navigation performance. The authors should introduce controls that disentangle spatial encoding from visual encoding.

2. The connection between the representation analysis and the introduced model components seems forced.

Even if the latent loop encodes spatial structure, the introduced component feels like a generic HPC–MEC-inspired architecture rather than one specifically motivated by the observed latent topology. In particular, it is unclear how the observation of an unstable loop structure motivates the specific grid–place dynamics introduced in the model. The improved stability of the latent loop across layers is an empirical consequence of the proposed architecture rather than something guaranteed by construction. The authors should better establish the mechanistic connection between their representation analysis and architectural design.

3. The use of Isomap is insufficiently justified.

The authors state that “Isomap helps preserve intrinsic geodesic structure when the latent states lie on a curved manifold.” However, persistent homology can already be applied directly to high-dimensional point clouds, including those sampled from curved manifolds. Furthermore, Isomap modifies pairwise distances and could alter the resulting topology. Can the authors apply the same analysis directly to the original latent representations, or to PCA-reduced representations without Isomap, to demonstrate that the observed loop structure is not an artifact of dimensionality reduction?

4. Fig. 1 contains points with death times smaller than birth times.

In the persistence diagram, why are there points with death < birth? By construction, death ≥ birth in persistent homology. Can the authors clarify whether this is a plotting error or whether a different convention is being used?

5. The choice of persistence threshold τ\tau = 0.1 is unjustified.

Why was τ\tau = 0.1 selected as the threshold for significant loop-like structures (L954)? It would be more convincing if the authors visualized the low-dimensional representations (as in Fig. 1) for samples with persistence around τ\tau = 0.1 to demonstrate that this threshold meaningfully distinguishes loop-like structures. A sensitivity analysis across different thresholds would also strengthen the justification.

Minor Weakness

6. The interpretation of Fig. 7 is questionable.

Although the reported correlation between H1 persistence and navigation error is interesting, most samples have persistence values below the authors' threshold of τ\tau = 0.1. By their own definition, these samples do not exhibit significant loop-like structure. Therefore, while the correlation itself is statistically valid, it does not necessarily support the claim that stronger cognitive-map-like representations lead to better navigation performance (L429–430). Furthermore, as mentioned in Weakness 1, visual prediction quality could explain both stronger persistence and lower navigation error. The authors should examine whether the correlation remains robust after controlling for visual prediction quality and across different persistence thresholds.

7. The claim in L215 is premature.

The authors state that “The analysis above links latent topological organization to navigation performance.” However, none of the analyses in Section 4 directly establishes this relationship. The relevant correlation analysis is only presented later in Section 6.3 (Fig. 7). The authors should either revise this claim or explicitly refer to the later analysis.

Questions for authors

1. Unclear which hidden states are used for the representation analysis (L148–150).

At which diffusion step are the hidden states extracted? Or are the activations aggregated across diffusion steps? Although the appendix mentions intermediate diffusion steps, the exact procedure remains unclear. Additionally, what does the 16-second horizon (64 latent states) mean in this context? How is each latent state represented as a point in the persistence analysis? Can the authors clarify?

2. Unclear distinction between generation and navigation (L150–151).

The authors define generation as action-conditioned direct generation and navigation as continuous autoregressive rollout. Does this mean that generation uses ground-truth context throughout, whereas navigation conditions on the model's own predictions from earlier trajectory steps? The authors should clarify the precise experimental distinction.

3. Insufficient characterization of the consistency of the latent loop structure across trajectories.

Fig. 1 and Fig. 2 show single examples. Although Appendix E.7 reports statistics across 60 random samples, the reported fraction of layers exceeding the persistence threshold does not fully characterize how consistently the loop structure emerges across trajectories. Can the authors report the distribution of maximum H1 persistence across trajectories and visualize representative samples with different persistence levels?

13 comments

We thank the reviewer for recognizing the value of the paper's analysis and method, and for raising specific questions about the experiments and presentation. Our responses to each question are provided below:

  1. Supplementary analysis for Figure 3 (Weakness 1)

We clarify that Section 4.3 examines the role of correct context--action--future-target relationships in latent representations. Following this intervention, both performance on the correct test set and the latent loop-like structure deteriorate, indicating that the observed structure depends on consistent correspondences within the task. This provides evidence of navigation relevance. To address your further question about distinguishing spatial factors from visual and temporal factors, we conduct an inference analysis using the existing frozen NWM, keeping the ground-truth context of each trajectory fixed. The first group receives the correct actions; in the second group, the action conditions are replaced with actions from other trajectories in the same scene with similar durations and action magnitudes. We compare the latent-space trajectories under these two settings for 20 RECON trajectories. We find that replacing the actions causes only a slight performance degradation, and the latent-space trajectories do not exhibit the collapse observed in Section 4.3. We speculate that this is related to the model's out-of-distribution generalization capability; our experiment reflects an association between visual encoding and latent structure. We will explain the limitations of the original experiment and add the subsequent conclusions in the revised Section 4.3.

  1. The role of the representation analysis (Weakness 2)

The representation analysis directs our attention to state consistency during prediction: loop-like organization appears in some layers of the original model, but this organization changes with network depth and autoregressive prediction. The architecture therefore introduces explicit action-update rules, constraining the evolution of the additional spatial conditioning by displacement. Specifically, the perceptual history establishes a place representation, the place-to-grid mapping determines a discrete state, and actions advance the state through periodic permutations. The updated state is then mapped back to spatial conditioning and supplied to the frozen backbone. Different periods encode displacement over different ranges; environment-dependent projections distinguish the interpretations of the same periodic state in different scenes. The observed latent loops motivate the design, while the roles of the components are tested through ablations. In Table 4, the full method achieves an ATE of 0.959, compared with 1.023 for identity transitions, 1.016 without remapping, and 1.059 for the generic MLP. These results support the roles of the specific dynamics and mappings. The periodic boundaries and permutation rules are internal constraints of the module, whereas the increased persistence of the backbone's hidden representations is an empirical result.

  1. Choice of dimensionality reduction methods (Weakness 3)

Isomap is one of the choices in our original analysis pipeline, intended to preserve geodesic distances on the latent manifold and thereby preserve its structure. We vary these dimensionality reduction methods and parameters using the same trajectories, network layers, and diffusion step. The results are shown below:

PCA dimensionsIsomap dimensionsNeighborsGeneration: NWMGeneration: OursRollout: NWMRollout: Ours
1610100.10590.11620.07360.0820
3210100.11660.12250.07700.0848
1615100.09920.11290.06680.0798
161050.10030.11270.07540.0853
16Not used—0.14670.15260.03620.0409
Not usedNot used—0.13720.14140.03310.0345

Bold indicates the parameters selected for our method. Although the computed persistence values vary across parameter settings, they all exhibit the same trend.

  1. Plotting issue (Weakness 4)

The plots use birth scale and persistence lifetime as their coordinate convention, with the vertical axis representing death scale minus birth scale. Labeling this axis as Death was an error. Under this convention, a vertical coordinate below the birth scale does not imply that the original death scale is below the birth scale. We will correct the vertical-axis label to persistence lifetime and explain this in the figure caption.

  1. Sensitivity analysis of the persistence threshold (Weakness 5)

For sensitivity to the persistence threshold, we vary the threshold and obtain the following results:

Threshold τGeneration: NWMGeneration: OursRollout: NWMRollout: Ours
0.05020/28 (71.4%)23/28 (82.1%)16/28 (57.1%)16/28 (57.1%)
0.07515/28 (53.6%)17/28 (60.7%)14/28 (50.0%)15/28 (53.6%)
0.10010/28 (35.7%)14/28 (50.0%)9/28 (32.1%)12/28 (42.9%)
0.1255/28 (17.9%)11/28 (39.3%)5/28 (17.9%)7/28 (25.0%)
0.1505/28 (17.9%)8/28 (28.6%)5/28 (17.9%)5/28 (17.9%)
0.2003/28 (10.7%)4/28 (14.3%)2/28 (7.1%)2/28 (7.1%)
  1. Interpretation of Figure 7 (Weakness 6)

Figure 7 uses maximum H1H_1 persistence as a continuous metric, whereas 0.1 is used in other experiments to compute the proportion of layers exceeding the threshold. The correlation analysis between continuous scores and ATE can include samples below the threshold. The figure directly supports the statement that ``continuous topological strength is correlated with trajectory error.''

To examine whether the association in Figure 7 can be explained by differences in visual prediction quality, we compute LPIPS between the predicted frames and corresponding ground-truth frames for each trajectory and average it over time to obtain a per-trajectory visual prediction error. We then use this error as a control variable in a partial correlation analysis between persistence and ATE. After controlling for visual prediction error, the partial correlation coefficient is −0.371-0.371, indicating that the association remains negative.

  1. The opening statement of Section 5 (Weakness 7)

We thank the reviewer for pointing out this ambiguity. Section 4 primarily analyzes latent topological features and their stability across layers, whereas the correlation analysis between persistence and navigation error is presented in Section 6.3 (Figure 7). We will revise the opening transition sentence of Section 5 to explicitly reference this analysis and distinguish the representation observations in Section 4 from the performance-association results in Section 6.3.

  1. Hidden-state extraction procedure (Question 1)

The current extraction setup uses the 125th model call among 250 reverse sampling steps. We average the output of the specified CDiT layer over the image-token dimension to obtain a 1152-dimensional vector, which serves as the state point for that prediction time. The current image configuration corresponds to 196 tokens. A 16-second horizon corresponds to 64 prediction times at 4 Hz. With the network layer and diffusion step fixed, the 64 state vectors from the same trajectory form a point cloud. The analysis is then repeated for the other network layers.

  1. Difference between generation and navigation (Question 2)

In the representation analysis, direct generation always uses the same initial ground-truth observation context and separately predicts future time points using the corresponding action and time conditions. In the autoregressive setting, each newly predicted frame is added to the context, and the earliest frame is removed, so subsequent predictions depend on previously generated results. Both settings are action-conditioned; the difference is whether the context is recursively updated.

  1. Additional persistence diagrams (Question 3)

Appendix E.9 of the paper presents additional examples, dimension-reduced representations, and persistent homology plots for both generation and navigation.

If our response has addressed your concerns, we would appreciate it if you could reconsider your overall assessment of the paper.

Reviewer_ASexOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this work, authors analyze long-horizon navigation ability of navigation world models (NVM). They analyzed hidden states of pretrained CDiT-XL model, that was used in the NVM baseline. They find that point clouds of hidden states collected along generated trajectories may form persistent low-dimensional loop-like structures. They provide experimental results showing a correlation between the persistence of these loop-like structures and navigation performance along sampled trajectories. Based on this observation, authors augment the original frozen CDiT backbone with a trainable compositional cognition module, that produces structured conditioning to stabilize latent spatial representations and improve long-horizon navigation consistency. For spatial organisation in this module, the authors introduce a special structure consisting of multiple discrete 2-dimensional toroidal grid modules. Proposed module learns a correspondence between navigation sensory history and a discrete grid representation. An action-conditioned transition then updates the grid state, which is projected back into a place-like representation and used as a condition for future state generation. Authors provide qualitative and quantitative comparisons of their augmented model with diffusion-based world models and the original navigation world model. They also provide ablation studies of the proposed condition mechanisms.

Critical strengths
  • Detailed analysis of persistent topological structures in diffusion-based world models and their correlation with navigation performance.

  • An interesting and novel approach to spatial conditioning, motivated by the observed correlations, latent-space analysis, and biological considerations.

Critical weaknesses

Limited reproducibility due to missing implementation details of several important transformations. Insufficient experimental evidence supporting the generalizability of the proposed approach to other backbones and the stability of its results across independent training runs.

Questions for authors

Clarity issues The text corresponding to Fig.4 introduce operators P and Q, while figure itself uses W_pg and W_gp. The relations between these labels are explained only later in appendix. Also caption lists panels (a) and (b) as grid-to-place and place-to-grid, respectively, when figure shows it in reverse order.

On page 20 trade-off between memory consumption and quality is mentioned, but only for different sizes of proposed module. What is the memory-overhead in comparison with the base model?

Reproducibility issues.

The code page at Anonymous.4open.science has expired, so I can’t check there additional details about the model. What is the explicit mathematical definition of T(a)? Details of Encoder and Decoder architectures connecting place representations p and sensory states s are not provided.

Experimental problems.

Reported means and std appear to be computed from five samples from each trained model. Was proposed model trained only once? If so, results across several independent training runs would be needed to assess the stability of the improvements. Also, Tables 5 and 6 do not report standard deviations, which should be provided.

Go Stanford reports results only at 4s, as in original baseline paper. Since the paper focuses on long-horizon stability, is it possible to provide a comparison at longer horizons on this dataset?

The method itself is tested in the main experiments using only one fixed frozen backbone and one set of pretrained weights. Could the authors specify the exact NVM checkpoint used and provide results with alternative pretrained checkpoints (either with or w/o Ego4D) and other backbones?

The topology analysis uses dimensionality reduction (PCA to 16 dimensions and Isomap to 10). Appendix E3 provides a stability theorem, showing that persistence diagrams remain stable if preprocessing approximately preserves the intrinsic metric structure. However, it is not shown that preprocessing with chosen parameters satisfy this condition. Additional robustness check under different preprocessing choices is mentioned but is not provided for PCA and Isomap. Either evidence that the assumptions of the stability theorem work in this case or results with different combinations of PCA and Isomap dimensions would be helpful.

5 comments

We thank the reviewer for recognizing the value of the paper's analysis and method, and for raising specific questions about the experiments and presentation. Our responses to each question are provided below:

  1. Notation and figure caption (Clarity issues 1)

When P and Q are first introduced in the main text, we will explain their correspondence to the two types of projection matrices in the figure and unify the notation across the main text, Figure 4, and the appendix. In Figure 4, panel (a) corresponds to place-to-grid, and panel (b) corresponds to grid-to-place. The caption will be corrected to match the directions shown in the figure.

  1. Resource overhead relative to the original model (Clarity issues 2)

A static calculation based on the current architecture shows that the trainable parameters of the added module amount to approximately 0.76% of the NWM backbone's parameters.

  1. Action transitions and encoder--decoder architecture (Reproducibility issues)

T(a)T(a) is a periodic translation operator on the grid at each scale. The normalized translation coordinates passed to the module are rounded to obtain integer displacements δx\delta x and δy\delta y. In a module with period λ\lambda, a grid location (r,c)(r,c) moves to ((r+δx)mod  λ,(c+δy)mod  λ)((r+\delta x) \mod \lambda, (c+\delta y) \mod \lambda). The complete transition combines the permutation matrices of all modules into a block-diagonal matrix, so the states at different scales are updated separately.

Regarding the encoder--decoder architecture, the perceptual interface consists of VAE feature extraction and two multilayer perceptrons. The VAE latent for each frame is flattened and mapped to a 768-dimensional perceptual feature. The perception-to-place network uses linear layers with dimensions 768→768→512→1152768\rightarrow768\rightarrow512\rightarrow1152, together with LayerNorm, LeakyReLU, and a final ReLU. The place-to-perception network uses linear layers with dimensions 1152→512→768→7681152\rightarrow512\rightarrow768\rightarrow768, with LayerNorm and LeakyReLU in the intermediate layers.

Reconstruction supervision is applied in perceptual feature space, with gradients stopped through the target-feature branch. We will add a layer-by-layer architecture table to the appendix and explain the aggregation of the four-frame context, positional embeddings, and which projections are fixed, so that this interface can be independently reproduced.

  1. Independent training and standard deviations in the appendix (Experimental problems 1)

We clarify that the means and standard deviations in Section 6 are obtained from five generation samples using a fixed trained checkpoint, measuring randomness in the generation process. We will make this statistical protocol explicit in the main text and table notes. Regarding independent training, we trained the model with three random seeds. Under this evaluation protocol, the ATE on RECON becomes 0.962±0.0170.962\pm0.017, and the FVD becomes 170±6.107170\pm6.107.

Regarding the standard deviations in the appendix, we will add them to the appendix in the revised version.

  1. Long-horizon prediction on Go Stanford (Experimental problems 2)

We clarify that the 4-second setting on Go Stanford is used for direct comparison with the original NWM protocol for unseen environments. The paper already reports 16-second results on RECON and on other datasets in the appendix.

  1. Pretrained weights and tests with other backbones (Experimental problems 3)

We use the officially released CDiT-XL checkpoint. This version was pretrained on RECON, SCAND, TartanDrive, and HuRoN, without Ego4D. During training, the backbone is frozen, and only the added cognition module is optimized.

For other backbones, we tested the CDiT-B checkpoint with a single training seed. Its LPIPS for 16-second generation decreased from 0.474 to 0.452. We will also add this result to the appendix.

  1. Preprocessing and the conditions of the stability theorem (Experimental problems 4)

To examine dependence on preprocessing, we vary the PCA dimensionality, Isomap output dimensionality, and number of neighbors while keeping all other conditions unchanged. The results are shown below:

PCA dimensionsIsomap dimensionsNeighborsGeneration: NWMGeneration: OursRollout: NWMRollout: Ours
1610100.10590.11620.07360.0820
3210100.11660.12250.07700.0848
1615100.09920.11290.06680.0798
161050.10030.11270.07540.0853
16Not used—0.14670.15260.03620.0409
Not usedNot used—0.13720.14140.03310.0345

Bold indicates the parameters selected for our method. Although the computed persistence values vary across parameter settings, they all exhibit the same trend.

The persistent homology stability theorem cited in our paper describes a conditional relationship between metric perturbations and changes in persistence diagrams. For sensitivity to the persistence threshold, we vary the threshold and obtain the following results:

Threshold τGeneration: NWMGeneration: OursRollout: NWMRollout: Ours
0.05020/28 (71.4%)23/28 (82.1%)16/28 (57.1%)16/28 (57.1%)
0.07515/28 (53.6%)17/28 (60.7%)14/28 (50.0%)15/28 (53.6%)
0.10010/28 (35.7%)14/28 (50.0%)9/28 (32.1%)12/28 (42.9%)
0.1255/28 (17.9%)11/28 (39.3%)5/28 (17.9%)7/28 (25.0%)
0.1505/28 (17.9%)8/28 (28.6%)5/28 (17.9%)5/28 (17.9%)
0.2003/28 (10.7%)4/28 (14.3%)2/28 (7.1%)2/28 (7.1%)

If our response has addressed your concerns, we would appreciate it if you could reconsider your overall assessment of the paper.

Reviewer_cjtwOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an action-driven, multiscale grid–place module that adds spatial conditioning to a frozen NWM through discrete path integration. The design follows an analysis of spatial representations in diffusion-based navigation world models. Using persistent homology, which tracks topological structures across scales, the authors identify loop-like structures in some intermediate layers and relate their persistence to trajectory error. On RECON, the method reduces Absolute Trajectory Error (ATE) from 1.057 to 0.959 and Fréchet Video Distance (FVD) from 191.096 to 169.855; the appendix reports results on three additional datasets. I see the main contribution as connecting this representation analysis to a structured module with measurable performance gains. (§4–6, Tables 1–4, Appendix Tables 5–6)

Critical strengths
  1. The targeted ablations are convincing. I appreciate that Table 4 includes generic MLP conditioning, identity transitions, removal of winner-take-all selection (WTA), and removal of environment-dependent remapping. The full method outperforms these variants, giving stronger support for the action-driven grid–place structure than a comparison with the original NWM alone. I consider this the paper's strongest empirical contribution.
  2. The representation analysis complements the performance results. I appreciate that the authors examine topological persistence, unseen environments, and different model configurations alongside visual prediction and trajectory error. Figure 7 reports a correlation of −0.492 between persistence and ATE, and the appendix includes representation analyses over 60 random samples. I think these results make the hypothesis that spatial structure helps navigation worth investigating further. (§6.3, Figure 7, Appendix E.6–E.8)
Critical weaknesses
  1. I am not yet convinced that the loops reflect spatial coding. In §4.3, the authors mismatch 25% of context–future-frame pairs. This also disrupts visual and temporal correspondence, so the test does not isolate spatial factors. Figure 7 establishes a correlation without controlling for potential confounders such as trajectory difficulty and length. In addition, Appendix E.3 first reduces the features to 16 dimensions with PCA and then embeds them in 10 dimensions with Isomap, before applying an absolute persistence threshold of 0.1. It does not clearly specify scale normalization across layers or present the cited comparisons across preprocessing settings. The stability theorem for persistent homology does not, by itself, guarantee that this preprocessing preserves the original geometry. For me, these issues affect the central interpretation of “cognitive-map-like representations,” not just the terminology.
  2. I could not reconstruct the core training and action-transition steps from the description. Appendix A.3 describes the place-to-grid projection as trainable, while A.4 uses hard WTA. Under the stated equations, ordinary gradients cannot pass through this discrete selection. I would therefore need to know whether training uses a straight-through estimator or another approximation. It is also unclear how continuous displacement and yaw define transitions on the discrete torus, how non-integer displacements are handled, and which coordinate frame is used to accumulate displacements in Eq. (3). These details determine how the method learns and performs path integration. I do not think describing the transitions as “deterministic” resolves them. (Eqs. 3, 12–13, Appendix A.3–A.4)
Questions for authors
  1. Clarification and additional evidence: Is a common scale normalization applied to the point clouds across layers? Please identify or provide sensitivity analyses for the PCA/Isomap parameters and persistence threshold. Is there a control that preserves visual and temporal statistics while changing only spatial correspondence, to support the spatial interpretation of the loops?
  2. Implementation clarification: How is backpropagation through WTA implemented? Please define the mapping from continuous (u,ϕ)(u,\phi) to T(a)T(a), including the displacement coordinate frame and the handling of rotation and non-integer displacements.
  3. Results clarification: Beyond the five post-training samples described in §6.1, was variability across independent training runs evaluated? Please specify the number of test trajectories and whether the MLP baseline in Table 4 uses the same auxiliary losses and training budget. This would help assess the stability and source of the gains.
3 comments

We thank the reviewer for recognizing the value of the ablation experiments and representation analysis, and for raising specific questions about the spatial interpretation and implementation details. Our responses to each question are provided below:

  1. Cross-layer normalization and the effects of dimensionality reduction and thresholds on the topological conclusions (Weakness 1, Question 1)

We first clarify that the same scale-processing rule is applied to the point clouds from all layers. After reducing the representations to 16 dimensions with PCA and then to 10 dimensions with Isomap, we center each point cloud and normalize it by its maximum radius. Persistent homology is computed on the 10-dimensional representations; the three-dimensional projections are used only for visualization.

To examine dependence on preprocessing, we vary the PCA dimensionality, Isomap output dimensionality, and number of neighbors while keeping all other conditions unchanged. The results are shown below:

PCA dimensionsIsomap dimensionsNeighborsGeneration: NWMGeneration: OursRollout: NWMRollout: Ours
1610100.10590.11620.07360.0820
3210100.11660.12250.07700.0848
1615100.09920.11290.06680.0798
161050.10030.11270.07540.0853
16Not used—0.14670.15260.03620.0409
Not usedNot used—0.13720.14140.03310.0345

Bold indicates the parameters selected for our method. Although the computed persistence values vary across parameter settings, they all exhibit the same trend.

The persistent homology stability theorem cited in our paper describes a conditional relationship between metric perturbations and changes in persistence diagrams. For sensitivity to the persistence threshold, we vary the threshold and obtain the following results:

Threshold τGeneration: NWMGeneration: OursRollout: NWMRollout: Ours
0.05020/28 (71.4%)23/28 (82.1%)16/28 (57.1%)16/28 (57.1%)
0.07515/28 (53.6%)17/28 (60.7%)14/28 (50.0%)15/28 (53.6%)
0.10010/28 (35.7%)14/28 (50.0%)9/28 (32.1%)12/28 (42.9%)
0.1255/28 (17.9%)11/28 (39.3%)5/28 (17.9%)7/28 (25.0%)
0.1505/28 (17.9%)8/28 (28.6%)5/28 (17.9%)5/28 (17.9%)
0.2003/28 (10.7%)4/28 (14.3%)2/28 (7.1%)2/28 (7.1%)
  1. Whether the future-frame mismatch experiment establishes spatial encoding (Weakness 1, Question 1)

Thank you for this question. We clarify that Section 4.3 examines the role of correct context--action--future-target relationships in latent representations. Following this intervention, both performance on the correct test set and the latent loop-like structure deteriorate, indicating that the observed structure depends on consistent correspondences within the task. This provides evidence of navigation relevance. To address your further question about distinguishing spatial factors from visual and temporal factors, we conduct an inference analysis using the existing frozen NWM, keeping the ground-truth context of each trajectory fixed. The first group receives the correct actions; in the second group, the action conditions are replaced with actions from other trajectories in the same scene with similar durations and action magnitudes. We compare the latent-space trajectories under these two settings for 20 RECON trajectories. We find that replacing the actions causes only a slight performance degradation, and the latent-space trajectories do not exhibit the collapse observed in Section 4.3. We speculate that this is related to the model's out-of-distribution generalization capability; our experiment reflects an association between visual encoding and latent structure. We will explain the limitations of the original experiment and add the subsequent conclusions in the revised Section 4.3.

  1. Whether the association in Figure 7 is explained by trajectory length and difficulty (Weakness 1)

Figure 7 reports a negative correlation between H1H_1 persistence and ATE across 30 RECON trajectories, with a correlation coefficient of −0.492-0.492. To examine whether this association can be explained by differences in visual prediction quality, we compute LPIPS between the predicted frames and corresponding ground-truth frames for each trajectory and average it over time to obtain a per-trajectory visual prediction error. We then use this error as a control variable in a partial correlation analysis between persistence and ATE. After controlling for visual prediction error, the partial correlation coefficient is −0.371-0.371, indicating that the association remains negative.

  1. Training implementation details (Weakness 2, Question 2)

We thank the reviewer for asking about the training implementation details. Regarding how WTA affects training, the place-to-grid projection is trained using a straight-through estimator to handle discrete selection. We will clarify the forward discrete encoding and backward estimation rules in Appendix A, and specify which parameters are fixed and which are updated.

Regarding the mapping from continuous actions to discrete transitions, we clarify that T(a)T(a) is a periodic translation operator on the grid at each scale. The normalized translation coordinates passed to the module are rounded to obtain integer displacements δx\delta x and δy\delta y. In a module with period λ\lambda, a grid location (r,c)(r,c) moves to ((r+δx)mod  λ,(c+δy)mod  λ)((r+\delta x) \mod \lambda, (c+\delta y) \mod \lambda). The complete transition combines the permutation matrices of all modules into a block-diagonal matrix, so the states at different scales are updated separately.

  1. Multi-seed experiments and the MLP implementation (Question 3)

We clarify that the means and standard deviations in Section 6 are obtained from five generation samples using a fixed trained checkpoint, measuring randomness in the generation process. We will make this statistical protocol explicit in the main text and table notes. Regarding independent training, we trained the model with three random seeds. Under this evaluation protocol, the ATE on RECON becomes 0.962±0.0170.962\pm0.017, and the FVD becomes 170±6.107170\pm6.107.

The MLP in Table 4 uses the same frozen NWM and conditioning interface as our method to examine whether generic learnable conditioning can replace structured spatial conditioning. Its training losses and budget are the same as those of the original method.

If our response has addressed your concerns, we would appreciate it if you could reconsider your overall assessment of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.