acceptodds
Under review as a conference paper at ICLR 2027

Learning Composable Transition Representations for Visual Navigation

Abstract

What latent representation should an image-goal navigation policy use? Direct visual policies are efficient, but their representations are often learned primarily through action supervision and may fail to capture the transition structure underlying viewpoint changes. We propose NavCompose, a self-supervised pretraining paradigm that learns structured navigation latents from unlabeled visual trajectories. NavCompose encodes relative visual transitions and explicitly regularizes them to be composable across consecutive observations and continuous along local trajectories. The learned representation is renderable and supports interpolation between observed viewpoints as well as short-horizon extrapolation beyond them, providing a compact model of local visual transitions. For downstream image-goal navigation, we freeze the pre-trained backbone and train only a lightweight waypoint prediction head. Across multiple navigation datasets, NavCompose achieves state-of-the-art waypoint prediction performance while maintaining strong accuracy with substantially fewer labeled trajectories. Its lightweight downstream architecture further enables fast inference, outperforming existing navigation models in runtime efficiency. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Visual Navigation, Unsupervised Pretrain

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JKDjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NavCompose, a self-supervised framework that learns composable and continuous visual transition representations from unlabeled trajectories for image-goal navigation. It combines three training objectives: pairwise transition grounding via view reconstruction, compositional regularization for consecutive viewpoint changes, and continuity regularization for latent interpolation and short-horizon extrapolation. The learned transition encoder is then frozen, and a lightweight waypoint prediction head is trained on labeled navigation trajectories. Across four navigation benchmarks—Go Stanford, RECON, SACSoN, and SCAND—NavCompose improves waypoint prediction accuracy over existing navigation and representation-learning baselines. The authors also report favorable label efficiency and inference speed, along with qualitative interpolation/extrapolation results and real-world robot navigation experiments.

Critical strengths
  1. A well-motivated representation-learning formulation for visual navigation. The paper tackles an important question: how to learn navigation-relevant representations from unlabeled visual trajectories instead of relying mainly on action or waypoint supervision.

  2. Strong downstream performance and computational efficiency. The experimental results are encouraging across multiple navigation benchmarks. NavCompose achieves the best average ATE/RPE among the compared methods (0.499/0.133), while keeping the downstream architecture lightweight.

Critical weaknesses

1.There is not yet enough evidence that the learned transition space is geometrically consistent and genuinely compositional.

Although the composition and continuity objectives do encourage structure, the learned representation may still entangle appearance or scene content with viewpoint motion. In particular, the composition operation combines latent addition with a learned residual correction, but it lacks an explicit disentanglement mechanism or group-theoretic constraint that would ensure pose-consistent composition across different scenes and trajectories. As a result, it remains unclear whether the learned algebra captures transferable geometric transformations or mainly reflects trajectory-specific visual correlations.

  1. Real-world evaluation is limited, and practical robustness is not yet well established.

The real-world experiments cover four routes with only three trials per method per route, amounting to 12 trials per method. Although NavCompose achieves a promising 75% success rate, the small number of trials makes it hard to judge the statistical reliability of the observed improvements. In addition, all methods rely on route-specific topological maps and a shared downstream navigation stack, which limits how much the results demonstrate robustness in less structured or previously unseen environments.

Questions for authors

Q1. Geometric consistency and disentanglement (Clarification).

To what extent does the learned transition latent encode relative viewpoint motion independently of scene appearance and content? Since the composition operator includes a learned residual correction, what prevents it from learning scene-dependent transformations that do not generalize consistently across trajectories? Can the authors provide evidence of geometric consistency, such as cross-scene transition transfer, inverse consistency, or cycle consistency?

Q2: Reliability of real-world navigation results (Clarification and additional evidence).

Given that the real-world evaluation includes only three trials per route and uses route-specific topological maps, how sensitive are the reported success rates to trial variability and environmental conditions? Can the authors provide additional trials, uncertainty estimates, or results on previously unseen routes to better substantiate the practical robustness of NavCompose?

3 comments
AuthorsOpenReview ↗

## Reviewer JKDj

We thank the reviewer for the insightful feedback. We have revised the paper accordingly.

W1 / Q1. Geometric consistency and disentanglement of the transition latent. We added an analysis in Appendix A.1 (Table 8). On held-out scenes from RECON and ScanNet, we test three properties, none of which is used as a training objective:

  • Cross-scene transfer. A latent extracted in one scene and applied to another scene with the same ground-truth relative motion reaches 18.6 / 17.5 dB PSNR on RECON / ScanNet, close to the target's own latent (20.7 / 18.1 dB) and far above the zero latent (14.4 / 13.4 dB) and a random latent (14.0 / 13.2 dB).
  • Inverse consistency. The backward latent is almost exactly the negation of the forward one, and applying both reconstructs the source view at 27.9 / 32.1 dB.
  • Cycle consistency. Closed loops a→b→c→a render closer to the start view (18.5 / 16.7 dB) than open chains (16.1 / 15.3 dB) or loops closed by a random latent (14.5 / 14.0 dB).

Results (PSNR in dB, higher is better; Appendix A.1, Table 8):

TestLatent appliedRECONScanNet
Cross-scene transferown zb0→b1z_{b_0\to b_1} (reference)20.718.1
transferred za0→a1z_{a_0\to a_1} from another scene18.617.5
random other scene14.013.2
zero (no motion)14.413.4
Inverseforward + backward za→b+zb→az_{a\to b} + z_{b\to a}27.932.1
forward only za→bz_{a\to b}14.814.2
Cycleclosed loop a→b→c→a18.516.7
open chain a→b→c16.115.3
loop closed by a random motion14.514.0

Although none of the three properties is used as a training objective, all of them emerge in the learned latent. These results indicate that the latent encodes motion largely independently of scene content. Regarding the residual correction: the cycle test above uses the full composition ⊕\oplus including fϕf_\phi, and closed loops still render closer to the start view than open chains or randomly closed loops, suggesting that the residual preserves rather than breaks the geometric consistency of the latent. This is also encouraged by design: fϕf_\phi operates only on latents, and the renderer already receives the scene appearance from the source image, so the latent only needs to carry the motion.

W2 / Q2. Reliability of the real-world results. We extended the evaluation from 4 routes × 3 trials to 6 routes × 10 trials per method (Section 4.6, Table 7). NavCompose achieves an overall success rate of 80.0% (48/60), compared with 41.7% (25/60) for ViNT and 11.7% (7/60) for NoMaD, and achieves the highest success rate on every route (per-route results in the General Response). A trial counts as a success only if the robot reaches the physical goal region and stops within it without collision or human intervention.

  • Robustness to environmental conditions. To test sensitivity to environmental changes, we repeated the Office route under three perturbations that are absent when the topological map is recorded (new Appendix A.5, Figure. 9): (1) minor layout changes, where objects along the route are moved or added; (2) dynamic obstacles, where a pedestrian walks through the camera view; and (3) extreme lighting, where strong direct sunlight causes overexposure and hard shadows. The topological map, goal images and all model parameters are kept unchanged, so observations during execution can differ substantially in appearance from the stored subgoal images. In this rollout, NavCompose still reaches and stops within the goal region. This suggests that the learned transition representation captures viewpoint changes between observations rather than relying on exact appearance matching.

  • Route-specific topological maps and unseen routes. Following ViNT and NoMaD, we evaluate image-goal navigation, where the target is specified by images rather than coordinates. Goal images must therefore be collected beforehand: for each route, we teleoperate the robot once to record a visual trajectory and sample it into a topological map of goal images (Appendix A.4). This one-time recording only defines the goals; it is not used for training, and no model parameters are updated using observations from the evaluation routes. All three methods share the same map, node-selection procedure, goal-switching rule, and low-level controller, so the comparison isolates the navigation model.

Thank you for addressing my concerns! I will keep my positive score.

We sincerely thank the reviewer for the constructive discussion and positive assessment, which have helped us improve the paper.

Reviewer_qx9cOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce a method to pretrain latent representations for image-goal-conditioned navigation. Specifically, they introduce reconstruction losses that target the trajectory-level consistency of the representations. They pretrain an encoder, NavCompose, that can be used downstream with an action head fine-tuned on top of it. They demonstrate that with this kind of pretraining, downstream policies can produce representations conducive to improved action prediction, and they show preliminary results of end-to-end navigation improvements.

Critical strengths

The paper has a good number of strengths and provides a clear motivation for the method. The method is well described, and a reasonable set of experiments is provided:

  • I specifically like Figure 2 and the description of each loss term in the method section. These are well described and clearly aligned with each other.
  • The offline results provide a comprehensive overview of the field of navigation and image-goal-conditioned methods. For the most part, the prose is clear, and the figures are valuable for understanding the paper’s contributions
Critical weaknesses

While there are many strengths to the work, there are several things I believe would improve the paper:

  • Several technical details are missing that would better inform the reader of how the method was implemented:
    • The architecture of the encoder and render are not described and other useful details relating to training of these models are not provided. Are these models initialized from pretrained weights? Model size is also omitted.
    • The training data is briefly mentioned, but the makeup of the training data is not thoroughly described. How many samples are used in total, and what is the breakdown of samples used from each dataset? This would both show how much more data you can leverage and provide context for how many samples are used in other methods
    • What are the actual values for the deltas between the images used for pretraining? In practice, are these images several metres apart or under a metre? Does this also affect training stability/the goodness of the representations you get?
  • The experiments focus on offline results mostly, supported by a few real-world examples
    • I appreciate the large-scale eval that is possible with offline eval. These results show that the models can learn to better predict actions. However, these open-loop experiments can only offer so much insight into the downstream implications of the representations learned in NavCompose -The real-world evaluation, at this time, is rather limited and could use some clarification.
      • How is the goal region defined? Is this a physical radius or some other kind of specification? It seems like it is mostly based on the goal image in the appendix
      • How long are these trajectories? What is the spacing between image goal waypoints?
      • Error bars averaged across would likely be quite large - is it reasonable to consider extending the number of real-world trials to truly differentiate these methods?
      • Can you discuss the failure modes/differences between NavCompose and the baseline methods?
  • Other aspects of clarity that would be good to address:
    • LPIPS is introduced as a loss, but the acronym for the loss is not expanded anywhere initially and is not cited or described. It would be good to cite this and explain what it is (Zhang et al. “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric”)
    • The interpolation/extrapolation figures seem to have a dimpled artifact on them - what is the source of this?
    • It would be good to highlight the best method in each column in Table 1 to make the results easier to understand - I believe this would also make your work look stronger
    • Does the size of the action head affect performance? What is the architecture of the action head (MLP/transformer?)?
Questions for authors
  • What architecture do the encoder and renderer use? Are these based on decisions made in prior work? How many parameters are in each of these models? Are these models initialized with some pre-trained weights?
  • How might this be extended beyond image-goal navigation? How would you adapt this, for example, to language-conditioned navigation?
  • Over what horizon are actions predicted? Is this consistent across baseline methods?
6 comments
AuthorsOpenReview ↗

Reviewer qx9c — Part 1/2

(Part 1/2)

We thank the reviewer for the detailed and constructive feedback. We have revised the paper accordingly.

W1.1 / Q1. Architecture, prior work, initialization, and model size of the encoder and renderer. Both the encoder and the renderer are multi-view Transformers with 8 MVLayer blocks (width 1024, 16 heads), operating on 256 × 256 images with 16 × 16 patches. The architecture follows XFactor. Both networks are randomly initialized and trained from scratch, with 148.0M and 157.2M parameters, respectively. The renderer receives no target pixels, so all information about the target view must pass through the 256-dimensional transition latent. We have added the detailed architecture to Appendix B.2 (Figure 10).

W1.2. Composition of the training data. Stage 1 uses 11,907 non-navigation videos (6.13M frames) from five sources (DL3DV, ScanNet, LAVN, TartanGround, and CityWalker), together with the RGB frames of the trajectory data, without any labels (27,178 videos, 7.05M frames in total). Stage 2 uses the official ViNT training splits of four navigation datasets (15,271 trajectories, 0.92M frames, 106,688 samples). The unlabeled pool is thus about 7.7× larger than the labeled navigation data. A per-dataset breakdown is given in Appendix B.3.

W1.3. Distance between images used for pre-training and its effect on training stability.

Pre-training frames are sampled at a fixed frame interval within each video. For the datasets that provide pose annotations, we use them only to measure the resulting motion between the sampled frames. On these datasets, the relative motion is almost always below 7 m and 60°, and most frame pairs lie within 1–4 m and 10–40°. The pre-training images are therefore typically a few metres apart, so the latent is trained on local but non-trivial viewpoint changes. The interval does matter: in our experiments, overly large intervals made training unstable, while overly small intervals failed to yield an informative latent space.

W2. Limited real-world evaluation.

We thank the reviewer for these questions. We agree that open-loop results alone give limited insight into downstream navigation behavior, and we have expanded and clarified the real-world evaluation accordingly.

We extended the evaluation to 6 routes × 10 trials per method (Section 4.6, Table 7). NavCompose achieves 80.0% (48/60), compared with 41.7% (25/60) for ViNT and 11.7% (7/60) for NoMaD.

W2.1. Goal region and success criterion. The goal region is a physical one: a 1 m radius centered on the robot's position when the final goal image was captured. Success is judged by the robot's actual physical position, not by the model's own prediction: the goal image is only used by the navigation stack to decide when to switch to the next node and when to stop. A trial counts as a success only if the robot completes the route and comes to rest inside the goal region without collision or any corrective human intervention. Passing through the region and driving on is a failure. Any operator takeover to redirect or correct the robot before arrival, or failing to finish within the 15-minute safety limit, is also counted as a failure.

W2.2. Trajectory length and waypoint spacing. The six reference routes are approximately 7–12 m long. Following the time-based sampling in the public ViNT/NoMaD deployment code, we subsample the teleoperated RGB stream at 1 s intervals to build each route's topological map. The spacing between consecutive goal images therefore depends on the driving speed during collection and is not a fixed metric distance. All methods use the same topological map on a given route, together with the same node-selection procedure, goal-switching rule, and low-level controller (Appendix A.4). The policies receive only RGB observations from a forward-facing RealSense D435i camera, without depth, external positioning, or a separate obstacle-avoidance module.

W2.3 Number of trials. We agree that the original evaluation was too small to separate the methods. The updated evaluation contains 180 real-world trials, 60 per method (10 per route), across six scenes: Corridor, Garden, Pantry, Office, Bridge, and Canteen (Section 4.6, Table 7; per-route results in the General Response). Before every trial, the robot is placed in the same start region with approximately the same heading, and no model parameters are updated using observations from the evaluation routes.

(Continued in the next comment.)

AuthorsOpenReview ↗

## Reviewer qx9c — Part 2/2

(Part 2/2)

W2.4. Failure modes. The three methods fail in different ways. ViNT completes several routes, but in some scenes it overshoots the physical endpoint and does not finish inside the goal region; its other failures are wall collisions. NoMaD's predicted headings are frequently unstable, and it struggles to make sustained progress along the intended route. Ours completes substantially more routes but still suffers collisions. Appendix A.4 (Figure 8) shows two representative rollouts from the same start and goal: ViNT deviates from the route and collides with an obstacle in the first case, and reaches the goal vicinity but fails to stop in the second, while NavCompose reaches and stops within the goal region in both.

W3.1. LPIPS is not expanded or cited. We now cite LPIPS (Zhang et al., 2018) and briefly describe it as a perceptual distance computed on deep network features at its first use (Section 3.2, Eq. 1).

W3.2. Dimpled artifact in the interpolation/extrapolation figures. We thank the reviewer for the observation. Novel view synthesis is not the target task of NavCompose: the rendering objective serves as a training signal that shapes the latent space, and we did not tune the renderer for image fidelity. The rendered views are therefore of limited visual quality, and the dimpled pattern is a rendering artifact of this kind. We believe it is most likely a patch-grid effect from unpatchifying the 16 × 16 output patches, though we have not isolated the cause experimentally.

Importantly, the artifact does not affect navigation. The renderer is discarded at navigation time, and waypoints are predicted directly from the latent representation, so pixel-level rendering quality never enters the control loop. The interpolation/extrapolation figures are intended only as a qualitative illustration that the latent space varies consistently with camera motion.

W3.3. Highlighting the best method in Table 1. Thank you for the suggestion. We now highlight the best result in each column of Table 1 in bold.

W3.4. Size and architecture of the action head. NavCompose and each Unsupervised Geometry Model provide a dedicated pose embedding. The trajectory head is a lightweight 4-layer Transformer decoder (18.7M parameters) operating directly on this embedding: 8 learnable action queries and one distance query cross-attend to the pose embeddings and predict the 8 future waypoints and the distance to the goal in parallel.

To study the effect of head size, we train heads of different capacities on top of the same frozen encoder. Results on RECON:

Head size (trainable params)ATE ↓RPE ↓
Small (2.85M)0.2750.073
Default (18.7M)0.2500.067
Large (60.22M)0.2480.067

Performance saturates at the default size: increasing the head by about 3× brings only a marginal gain (0.250 → 0.248 ATE, identical RPE), while shrinking it by about 6.6× increases the error moderately. This indicates that most navigation-relevant information is already captured by the frozen representation, and a lightweight head is sufficient to read it out.

Q2. Extension beyond image-goal navigation, e.g., language-conditioned navigation. We thank the reviewer for this question. We see two natural extensions. First, language can be grounded into visual goals: a vision-language model maps the instruction to a goal image or a sequence of subgoal images, and NavCompose then navigates between them as in the current setting. Second, the trajectory head can be conditioned on a language embedding in addition to the transition latents, so that the latents encode the relation between views while language specifies the intended motion. Both options reuse the learned transition representation. Language conditioning is outside the scope of this work, which focuses on image-goal navigation, but we agree it is a very meaningful direction for future work.

Q3. Prediction horizon and consistency across baselines. Our trajectory head predicts 8 future waypoints, covering about 2 s at each dataset's waypoint spacing. For baselines whose native prediction length differs, the predicted trajectories are interpolated to the length of the ground-truth waypoints, so all compared methods are evaluated against the same ground-truth horizon on identical test samples.

Thank you for addressing my concerns!

Can you report the standard error or deviation for the averaged success rate as well? For reference, computed across scenes, the standard deviation and error are as follows

MethodOverall SRSD across environmentsSE across environments
ViNT41.7%41.216.8
NoMaD11.7%14.76.0
NavCompose80.0%22.89.3
Reviewer_xfTjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles image-goal navigation from the visual representation pretraining perspective with unlabeled video sequences. Past works have either focused on single-view representation or single-transition representations, often using action and pose supervision. The proposed NavCompose framework aims to use unlabeled videos and to learn a latent space that captures longer-horizon transitions smoothly via compositional and continuous trajectory regularization objectives. The learned visual transition encoder is used downstream to readout into predicted waypoints with limited data. On four real-world trajectory benchmarks as well as real-world navigation tasks, NavCompose outperforms task-specific navigation models, unsupervised visual foundation models, and unsupervised geometry models.

Critical strengths
  1. The results are strong in Table 1, even compared to task-specific navigation models.
  2. The method is intuitive and easy to follow in the presentation. The compositional and continuous structure regularization objectives make sense.
  3. It is good that the authors gave some real-world navigation results in Section 4.6.
  4. Qualitatively, the extrapolated images look pretty clean and correct from the inferred motion in the paired image context.
Critical weaknesses
  1. This method reads very related to latent action models such as LAPA, LAPO, Midway Network, DynaMo, etc., but these are not cited.
  2. Assumptions seem to favor smooth motion, but cannot capture more abrupt motion / state changes.
  3. Some phrases like "despite being pretrained solely on unlabeled visual observations" apply to most of the models in the table, so it is mostly a comparison with navigation models. Also, unclear how baselines like VAE and RAE are applicable here, especially since RAE is based off of another pretrained visual representation.
  4. The ablations being in the appendix is not ideal, considering it shows which parts are important to this method paper. I would emphasize these more in the main paper.
  5. The pairwise transition grounding and compositional transition structuring have already been proposed before, decreasing this paper's novelty. It is unclear whether the continuous structuring proposed by the authors is new.
  6. Number of trials used for real-world navigation seems low, compared to robotic papers, which tend to use at least 10 trials per task.
Questions for authors
  1. How would this method work for very fast and large transitions? Or for navigation where the start and end viewpoints are pretty different?
  2. How do you disentangle the effect of data from the different methods in Table 1?
  3. Can this pretraining be combined with the methods of task-specific navigation models like NWM somehow as an initialization?
  4. How do other methods scale with different fractions of labeled data?
  5. Why is NavCompose more efficient than other methods? It is unclear really which part of NavCompose should be more efficient, if the method revolves more around the pretraining objectives.
5 comments
AuthorsOpenReview ↗

Reviewer xfTj — Part 1/2

(Part 1/2)

We thank the reviewer for the constructive feedback. We have revised the paper accordingly.

W1. Missing citations of latent action models (LAPA, LAPO, Midway Network, DynaMo). We now discuss these works in Section 2.3. LAPO and LAPA infer latent actions as pseudo-labels for policy pre-training, while DynaMo and Midway Network use latent dynamics as a pre-training objective for frame-level encoders. In these methods, the latent is constrained at most by an information bottleneck and is either imitated or discarded after pre-training. In contrast, NavCompose explicitly structures the transition latent to compose across steps and scale with partial progress, grounds both properties through view rendering, and directly transfers this latent as the representation for image-goal navigation.

W2 / Q1. Smooth-motion assumption; very fast and large transitions; very different start and goal views. We added a robustness analysis. We group the test samples of all four benchmarks by heading change, start–goal displacement, and average speed, and evaluate the same checkpoint without re-training. Average ATE (m) over all four datasets:

Heading changeATE ↓Start–goal displacementATE ↓Average speedATE ↓
0–30°0.316< 1 m0.336< 0.6 m/s0.326
30–60°0.3151–2 m0.3190.6–1.0 m/s0.291
60–90°0.3942–4 m0.3841.0–1.5 m/s0.387
≥ 90°0.639≥ 4 m0.802≥ 1.5 m/s0.617

The error stays nearly constant up to a 60° heading change and increases mainly in the most extreme bins (≥90°, ≥4 m, ≥1.5 m/s). Appendix Figure 5 further shows that, even under large rotations, the predicted trajectories recover the correct turning direction and curvature. This analysis is added to the new Appendix A.2 (Table 9, Figure 5).

For state changes in the scene, we test real-world navigation under environmental perturbations in the new Appendix A.5 (Figure 9): minor layout changes, a pedestrian walking through the camera view, and strong direct sunlight, none of which are present when the topological map is recorded. In this rollout, NavCompose still reaches and stops within the goal region.

W3. "Despite being pretrained solely on unlabeled visual observations" applies to most baselines; applicability of VAE and RAE. We have rephrased Section 4.3. The comparison among pre-trained representations is now framed as a controlled comparison: all backbones are frozen and trained with an identical recipe, and the Unsupervised Foundation Models baselines share an identical readout and trajectory head, so their differences reflect representation quality. Details of how each backbone is read out, including VAE and RAE, are given in Appendix B.1 (Table 10). Since RAE and RAEv2 reuse frozen DINOv2 encoders, we read features from their pixel decoders, the only component unique to RAE.

W4. Ablations in the appendix. We moved the ablations of the pre-training objectives, the composition operator, and the latent dimension into the main paper (Section 4.4, Tables 2–4).

W5. Novelty of pairwise and compositional structuring; is continuous structuring new?

  • Pairwise grounding follows XFactor, as stated in Section 3.2, and is not claimed as our contribution.
  • Compositional structuring. Compositional latent structure has also been explored in recent latent-action models such as ALAM and AC-LAM, which organize action-like latents to support downstream manipulation policies. In contrast, we study compositionality as a representation-learning principle for image-goal navigation, where the learned transition latent encodes relative viewpoint changes and is directly transferred as a visual representation for waypoint prediction.
  • Continuous structuring is, to the best of our knowledge, proposed in this work. It is also effective: in Table 2, it improves over pairwise grounding on its own (ATE 0.289 → 0.273) and further improves when combined with composition (0.264 → 0.250 for the full model).

(Continued in the next comment.)

AuthorsOpenReview ↗

## Reviewer xfTj — Part 2/2

(Part 2/2)

W6. Number of real-world trials. We extended the real-world evaluation from 4 routes × 3 trials to 6 routes × 10 trials per method (60 trials for each method; Section 4.6, Table 7). NavCompose achieves an overall success rate of 80.0% (48/60), compared with 41.7% (25/60) for ViNT and 11.7% (7/60) for NoMaD, and achieves the highest success rate on every route (per-route results in the General Response). All methods share the same topological map and navigation stack, and a trial counts as a success only if the robot reaches the physical goal region and stops within it without collision or human intervention. The deployment protocol is detailed in Appendix A.4.

Q2. Disentangling the effect of data in Table 1. We separate the effect of data in three ways:

  • Labeled data. All frozen representations are trained on the same labeled navigation data, and Table 5 further varies its amount from 10% to 100%. NavCompose achieves the lowest error at every fraction.
  • Pre-training data scale. Our unlabeled pre-training pool (7.05M frames, Appendix B.3) is far smaller than the pre-training data of the Unsupervised Foundation Models, so our advantage does not come from more data.
  • Same pre-training data. XFactor, the method closest to ours, is trained on exactly the same pre-training data as NavCompose. NavCompose reduces the average ATE/RPE from 0.575/0.153 to 0.499/0.133 (Table 1), showing that the gain comes from the structure of the learned latent space rather than from the data.

Q3. Combining with task-specific navigation models such as NWM as an initialization. Thank you for the suggestion. NWM is difficult to adapt directly: its diffusion model generates future frames in a per-frame VAE latent space, whereas our latent encodes the relative transition between two observations rather than the state of a single frame, so it cannot serve as a drop-in replacement without redesigning the generative model. We therefore tested the idea on two task-specific navigation policies, ViNT and NoMaD, by initializing their visual encoders with our pre-trained transition encoder. Results on RECON:

MethodATE ↓RPE ↓
ViNT0.6590.215
ViNT + our pre-training0.2760.075
NoMaD1.2200.262
NoMaD + our pre-training0.2550.070
NavCompose (lightweight head)0.2500.067

Our pre-training substantially improves both policies, reducing ATE by 58% for ViNT and 79% for NoMaD. This shows that the learned transition representation is complementary to task-specific policy designs and can serve as a general initialization for navigation models. Extending this to world models such as NWM is an interesting direction for future work.

Q4. Scaling of other methods with different fractions of labeled data. We added ViNT and E-RayZer to the data-efficiency study. Results on RECON (ATE/RPE):

Training dataNavComposeViNTE-RayZer
10%0.288/0.0792.236/0.4280.301/0.084
30%0.266/0.0711.305/0.3660.281/0.077
50%0.258/0.0690.967/0.2640.265/0.073
100%0.250/0.0670.659/0.2150.253/0.070

ViNT, trained end-to-end, degrades sharply as labels are reduced (ATE rises from 0.659 to 2.236 at 10%), whereas both frozen pre-trained representations remain robust. NavCompose achieves the lowest ATE and RPE at every fraction, with a larger margin over E-RayZer when supervision is limited. These results are added to Section 4.4 (Table 5).

Q5. Why is NavCompose more efficient? The efficiency stems from the downstream inference pipeline rather than the pre-training objectives: NavCompose encodes only the current and goal images in a single forward pass of the frozen encoder, without the multi-frame context of ViNT or the iterative sampling of NoMaD and FlowNav (Section 4.4, "Model profile", Table 6). The pre-training objectives contribute to accuracy and label efficiency.

Thank you for your response. I have the remaining concerns below:

W1: Thanks for adding the latent action models to the related work. However, it remains unclear why these are not baselines for your method. Pairwise transition grounding, which predicts the target view from the source view using a transition latent inferred from both views, is very similar to how latent action models are trained. Some of these works also use the latent action downstream. For instance, LAPO decodes inferred latent actions into real actions using a small amount of labeled data, which is close to reading out waypoints from your transition latent. Overall, the distinction is not convincing to me.

W2/Q1: Thanks for the analysis on larger transitions. However, it lacks baselines, so it is unclear whether NavCompose degrades more or less than other methods.

W3: I am still very confused why VAE and RAE are used as baselines if the goal is to compare pre-trained frozen representations. Both VAE and RAE target creating a latent space for flow matching / diffusion. For RAE, the features are read from the pixel decoder, which is trained for image generation, so it does not make sense to me why this is a baseline for image-goal navigation.

W4. Thanks for moving them.

W5. The most salient novel component seems to be the continuous structuring. However, its impact is fairly small, going from composable-only to the full model.

W6: Thanks for adding more trials, this makes the real-world experiments more robust.

Q2: Since this targets visual representation learning, the data used to train the representations is a large confounding factor for the other baselines. Your pretraining pool includes the RGB frames of the navigation training data itself, so even though it is smaller than the data of the foundation models, it is much more aligned with the downstream benchmarks.

Overall, given the lack of clarity around why certain baselines were chosen and the pretraining data confounding factor, I would like to maintain my score at this time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

We sincerely thank all reviewers for their constructive and detailed feedback. Reviewers found our study of composable and continuous transition latents for image-goal navigation well motivated, and raised valuable points on the reliability of the real-world evaluation, robustness to large transitions, the geometric properties of the latent, and missing implementation details. We have revised the paper to address all of them. In the revised PDF, changes are color-coded by reviewer: xfTj in indigo, qx9c in amber, JKDj in teal, and concerns shared by several reviewers in dark red.

Real-world evaluation (all reviewers; Section 4.6, Table 7, Appendix A.4–A.5). We extended the real-world evaluation from 4 routes × 3 trials to 6 routes × 10 trials per method:

MethodCorridorGardenPantryOfficeBridgeCanteenOverall SR
ViNT9/101/100/105/101/109/1041.7%
NoMaD3/101/100/100/100/103/1011.7%
NavCompose10/104/107/109/108/1010/1080.0%

NavCompose achieves the highest success rate on every route, completes all trials on Corridor and Canteen, and shows the largest gains on Pantry and Bridge. The protocol is detailed in Appendix A.4:

  • Shared navigation stack. All methods use the same route-specific topological map, node-selection procedure, goal-switching rule, and low-level controller, so only the navigation model differs.
  • Strict success criterion. A trial is successful only if the robot reaches the physical goal region and stops autonomously within it, without collision or human intervention. Merely passing through the goal region is not counted as success.
  • RGB-only, no route-specific training. The policies receive only RGB observations, without depth, external positioning, or a separate obstacle-avoidance module. No model parameters are updated using observations from the evaluation routes.
  • Robustness and qualitative results. Appendix A.5 (Figure 9) evaluates NavCompose under layout changes, a walking pedestrian, and strong sunlight, and Appendix A.4 (Figure 8) shows representative rollouts of ViNT and NavCompose.

Together, these results show that NavCompose not only substantially outperforms existing navigation models in closed-loop real-world navigation, but also remains robust to scene changes, dynamic obstacles, and illumination shifts.

Main updates to the paper:

  1. Larger real-world evaluation (all reviewers; Section 4.6, Table 7). We extended the evaluation from 4 routes × 3 trials to 6 routes × 10 trials per method across six scenes. NavCompose achieves 80.0% (48/60), compared with 41.7% for ViNT and 11.7% for NoMaD, and the highest success rate on every route. The deployment protocol, goal region, success criterion, and topological-map construction are described in Appendix A.4.
  2. Ablations in the main paper (xfTj; Section 4.4, Tables 2–4). The ablations of the pre-training objectives, the composition operator, and the latent dimension have moved from the Appendix into the main paper.
  3. Label efficiency against more baselines (xfTj; Section 4.4, Table 5). We added ViNT and E-RayZer to the 10%–100% labeled-data study. NavCompose achieves the lowest error at every fraction.
  4. Related work on latent action models (xfTj; Section 2.3). We added a discussion of LAPO, LAPA, DynaMo, Midway Network, ALAM, and AC-LAM, and how NavCompose differs from them.
  5. Presentation (qx9c). LPIPS is now cited and described at first use (Section 3.2), and the best results in Table 1 are in bold.

New appendix sections:

  • A.1 Geometric consistency of the transition latent (JKDj; Table 8): cross-scene transfer, inverse consistency, and cycle consistency on held-out scenes.
  • A.2 Robustness to large transitions (xfTj; Table 9, Figure 5): ATE grouped by heading change, start–goal displacement, and speed.
  • A.4 Real-world deployment details (all reviewers; Figure 8): hardware, topological navigation protocol, evaluation protocol, and qualitative rollouts.
  • A.5 Robustness to environmental perturbations (xfTj, JKDj; Figure 9): real-world rollouts under layout changes, a walking pedestrian, and strong sunlight.
  • B.1 Representation baselines (xfTj; Table 10): the readout layers of each backbone, including VAE and RAE.
  • B.2 Architecture of the encoder and renderer (qx9c; Figure 10) and B.3 Training data (qx9c): architecture, parameter counts, and a per-dataset breakdown of pre-training and navigation data.

Additional experiments in the individual responses: initializing ViNT and NoMaD with our pre-trained encoder (xfTj Q3), which reduces their ATE on RECON by 58% and 79%, and an ablation of the trajectory-head size (qx9c W3.4).

We hope these revisions address the reviewers' concerns, and we would be glad to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.