- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper aims to transfer human navigation experience from large-scale egocentric web videos to quadruped robots by predicting future scene representations instead of pixels or pseudo-actions. The proposed representation is a latent learned by an autoencoder over segmentation masks and relative depth from off-the-shelf foundation models, and it serves as the prediction target of a world expert pretrained on web videos without robot actions. An action expert is then trained on teleoperated robot data to generate trajectories conditioned on the understanding and world experts. The paper also contributes a curated web-video and robot dataset and a pseudo-closed-loop metric, and reports improvements over VLN and VLA baselines on held-out robot data as well as competitive results on NAVSIM.
Critical strengths
-
Data. The curated egocentric web corpus and the real-world delivery teleoperation data, which the authors plan to release, would be a valuable resource for learning navigation from human videos.
-
Formulation. Factorizing the policy into an embodiment-shared scene-evolution predictor and an embodiment-specific action generator is a clean and well-motivated way to use action-free human videos.
-
The paper presentation is clear and easy to follow.
Critical weaknesses
-
Evaluation. All quadruped results are open-loop or a non-reactive 1-second replay on a test split drawn from the same teleoperation collection, and open-loop metrics are known to correlate poorly with closed-loop performance in navigation and driving. Evaluation on a closed-loop urban navigation benchmark, such as Wanderland [1], is needed to support the claimed navigation and social-compliance gains.
-
Performance. Even under the pseudo-closed-loop metric, the gains over the strongest baseline on the most important safety metrics, NC and WAC, are marginal, and web pretraining yields similarly small gains.
-
Representation. The paper does not explain why an implicit autoencoder latent is preferable to directly predicting the future segmentation and depth as explicit targets, as is common in chain-of-thought-style VLA supervision, and provides no comparison with such explicit targets. The claimed relational and domain-invariant properties are also unverified: the masks and depth remove appearance but not viewpoint, and no experiment shows that the representation transfers across domains or that human and robot representations actually align.
-
Insights. The alternative prediction targets in Table 2 perform only marginally differently from DISR, and the paper offers no analysis of why DISR outperforms DINOv3, V-JEPA, or depth-augmented DINO features.
-
Baselines. All baselines are trained only on robot data, so Table 1 conflates the proposed representation with the use of extra web data. Comparisons with navigation methods that learn from web videos via pseudo-actions, such as CityWalker and MBRA [2], as well as standard visual navigation baselines such as NoMaD, are needed to show that DISR is a better way to exploit human videos.
-
NAVSIM. The driving experiment uses no web pretraining and adds a trajectory scorer, so it does not test the paper's central claim of human-to-robot transfer.
[1] Liu, Xinhao, et al. "Wanderland: Geometrically grounded simulation for open-world embodied AI." CVPR. 2026.
[2] Hirose, Noriaki, et al. "Learning to drive anywhere with model-based reannotation." RA-L. 2025
Questions for authors
-
The web-video captions are generated from the full clip, including subsequent motion, and describe ego-motion. Do they therefore encode the camera wearer's future movement, and if so, how is this consistent with pretraining "without action labels"?
-
What are the QNS weights, and how were they chosen?