acceptodds
Under review as a conference paper at ICLR 2027

ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation

Abstract

We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model's ability to generalize to real-world navigation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Language-conditioned local navigation, Internet egocentric videos, Action-free video pretraining, World action models, Robot learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_njUsOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ScaleNav, a framework for language-conditioned local navigation that leverages action-free Internet egocentric videos to learn video/navigation priors. Using a three-stage training process (video pretraining, action alignment, and joint fine-tuning) built upon a unified world-action model with a Diffusion Transformer backbone, the method aims to translate Internet-scale visual data into executable robot actions. The authors evaluate ScaleNav on two simulation benchmarks as well as on a real quadruped robot, demonstrating improvements over several baselines.

Critical strengths

An interesting setting may open-up more discussions in agentic era

Critical weaknesses
  1. Questionable Problem Setting & Missing SOTA Comparisons on Real Robots The core setting focuses on local navigation where the target is already visible in the initial observation. For such a setting, the visual-to-action mapping should arguably be well-handled by existing SOTA foundation models and recent visual-language-action navigation frameworks. For instance, recent methods like NVIDIA LightNav or similar SOTA approaches likely achieve extremely high performance on real robots even under zero-shot or simpler adaptation settings. The authors fail to compare ScaleNav against these strong, contemporaneous SOTA baselines on real-world hardware, making it difficult to judge the genuine necessity and performance upper bound of the proposed multi-stage training framework. The simulation experiments also rely heavily on outdated baselines or adapters that feel somewhat outdated. Given the rapid progress in embodied AI and foundation navigation models, more recent and competitive SOTA simulation baselines should be included to validate the true efficacy of the proposed design.

  2. Limited Methodological Innovation From a technical standpoint, the framework largely consists of stitching together existing techniques—such as combining a pre-trained video diffusion backbone, a standard chunk-wise causal attention mechanism, a basic action alignment stage, and standard flow-matching objectives. There is a lack of deep conceptual or architectural innovation tailored specifically to the unique challenges of local navigation. Furthermore, the current real-robot and simulation results fail to convincingly demonstrate that these combined components yield a decisive, breakthrough advantage over properly tuned existing pipelines.

Questions for authors

Given that the target is already visible in the initial observation, what specific failure modes of existing SOTA methods does ScaleNav fundamentally solve, and to what extent do the individual training stages contribute to overcoming these specific failure modes rather than just providing a brute-force data scaling effect?

5 comments
AuthorsOpenReview ↗

Response to Reviewer njUs

Thank you for the detailed feedback. We clarify the challenges of navigation to initially visible targets, add comparisons with recent navigation models, and analyze failure modes and the roles of training and model components. We also clarify the core contribution and provide an update on the ongoing real-robot comparisons.

1. Task difficulty and recent baselines

An initially visible target provides a reference for identifying the instructed object, but does not specify a traversable route, executable motions, or a stopping decision. Viewpoints change and the target may leave the field of view. Our world-action model connects motion generation with predicted visual changes, while the initial-observation anchor preserves the reference needed to interpret the instruction.

To examine recent models’ performance in this setting, we added LightNav, NaVILA, and OmniNav. ScaleNav retains the highest SR among the evaluated single-view methods on both benchmarks. The complete comparison, including three-view OmniNav, is in CQ1 of our Common Questions response.

We are also conducting real-robot comparisons with recent navigation baselines and will provide the results once these experiments are complete.

2. Failure modes and training contributions

To identify the failures reduced by ScaleNav, we manually reviewed and annotated, in random order, all 104 failed rollouts from ScaleNav, its variants without Internet-video training and without the anchor, and fine-tuned StreamVLN and Uni-NaVid on the same 157 HabitatGS-LN tasks. Each rollout is assigned to one primary failure category.

Target grounding and maintenance: persistently approaching the wrong object category, or deviating before arrival without resuming the target approach, judged from the approached object and motion direction. These cases concern grounding across viewpoints.

Wrong instance: approaching an object of the correct category but not the instance specified by the instruction, judged against the initial reference and instruction constraints. These cases require grounding attributes and spatial relations.

Collision or blockage: physical contact with scene obstacles or an inability to move past them. Avoidance requires estimating clearance and generating motion that accounts for the robot’s footprint.

Stopping timing: stopping outside the success range in front of the correct target, or continuing forward after arrival, assessed from stopping records and target distance. These cases concern arrival judgment and termination.

Primary failure counts on the same 157 HabitatGS-LN tasks.

MethodTarget grounding and maintenanceWrong instanceCollision or blockageStopping timingTotal failures
Uni-NaVid (fine-tuned)2052128
StreamVLN (fine-tuned)17731138
ScaleNav w/o Internet videos705315
ScaleNav w/o anchor542415
ScaleNav20428

The main improvements are in grounding and stopping. ScaleNav has 2 grounding-related failures versus 25 for fine-tuned Uni-NaVid and 24 for StreamVLN, including 0 wrong-instance errors versus 5 and 7. Stopping errors fall from StreamVLN’s 11 to ScaleNav’s 2.

The ablations help interpret these gains: removing Internet-video training increases target-grounding and maintenance failures from 2 to 7. These results suggest that Internet-video training helps maintain correspondence between the instruction and the target as the viewpoint changes. In a task-wise comparison between the variant without the anchor and ScaleNav, 6 grounding failures and 3 premature stops become successes. With 2 reverse changes, this gives a net gain of 7 successes. We believe the anchor’s preserved initial visual context supports both instruction grounding and arrival judgment; zero-motion supervision directly trains termination after arrival.

The failure analysis links specific improvements to Internet-video training and the anchor. The proposed mechanisms are inferred from these results; a finer attribution to individual training stages remains for further study.

3. Contribution

We agree that several components build on prior work. Our contribution is to establish the value of Internet egocentric videos as a scalable source of navigation experience and provide a framework that uses the learned navigation priors to generate executable actions, without requiring action labels for those videos. ScaleNav performs strongly in simulation and navigates successfully on a real quadruped. CQ2 in our Common Questions response details this contribution.

I thank the authors' for the detailed explanation. After reading the rebuttal, it sounds like this work is a simplified version of SparseVideoNav. SparseVideoNav also leverages egocentric videos for pretraining to generate executable actions without requiring action labels, while handling more complex scenarios, even when the target object is initially out of view. Compared to SparseVideoNav, ScaleNav offers little methodological advantage and operates in a noticeably simpler setting.

Could the authors elaborate more edges compared to SparseVideoNav?

AuthorsOpenReview ↗

Differences from SparseVideoNav: Video Pretraining and Task Setting

Thank you for highlighting SparseVideoNav, which is closely related to our work and makes valuable contributions to beyond-the-view navigation. SparseVideoNav builds video prediction and inverse-dynamics action learning around a self-collected navigation corpus, with action generation dependent on the quality and consistency of predicted futures. ScaleNav instead learns navigation priors from independently sourced Internet videos and transfers them to robot control through a shared video-action backbone that continues to adapt under action supervision. Our focus is on scalable transfer from Internet video to executable navigation, together with a complementary practical objective: reliable execution of visible-target instructions for direct human-robot interaction and as a reusable low-level capability in agentic systems.

1. Data utilization and training framework

SparseVideoNav uses 140 hours of self-collected navigation videos and manually annotated instructions to train a history-conditioned model for long-horizon sparse future prediction and distill it for deployment. The purpose of this video pretraining is to provide long-horizon visual foresight for subsequent inverse-dynamics action generation. It then generates futures from observations in the same corpus and uses DA3-estimated camera motion from the generated frames to supervise a separate inverse-dynamics action predictor. (Sections 3.2-3.6)

The future predictor used by their inverse-dynamics model is learned during video pretraining and frozen during action learning. Video pretraining must therefore establish reliable scene and motion prediction for subsequent action generation. This places demanding requirements on pretraining video quality, as action supervision cannot further refine the predictor and its errors can propagate into control. Their own failure analysis reports future-prediction errors and visual-action mismatches, including incorrect turns and stops. The authors hypothesize that their decoupled design may limit generalization and identify joint video-action training as future work. (Section 3.6 and Appendix A.1)

Our central idea is to use existing Internet egocentric videos as an independent, scalable source of navigation priors, learning from diverse real-world scenes and the visual dynamics of human navigation.

Our video pretraining aims to learn navigation priors from real-world visual dynamics for subsequent transfer and adaptation to robot control. It therefore need not establish a fixed future predictor that is already reliable enough for action decoding. Videos with varied appearances, camera motions, and recording conditions can contribute transferable visual and motion knowledge. We believe this paradigm is better suited to scaling navigation learning with larger, more diverse Internet-video corpora. Our earlier no-Internet ablation and matched-budget video-source control examine the value of this transfer.

2. Task setting and practical relevance

SparseVideoNav evaluates both instruction-following and beyond-the-view navigation, emphasizing long-horizon foresight for initially unseen targets. Its deployed four-step model reports 25.0% BVN success across six real-world scenes. This demonstrates progress in search and route selection, while reliable completion remains challenging in that setting. (Table 1)

Our setting starts from a practical requirement: dependable execution of local goals in human-robot interaction and agentic systems. A person or high-level agent can specify a visible target through language without supplying metric coordinates or low-level motions. We view this as a useful interface for delegating local navigation.

For people and high-level agents, local navigation must provide dependable execution that subsequent actions and decisions can build on. When this capability is invoked repeatedly, a failed local command can disrupt the overall task, making robustness and repeatability important. Whereas SparseVideoNav emphasizes extending navigation to initially unseen targets through long-horizon foresight, our setting prioritizes reliable completion of specified local goals as a reusable execution capability.

In our evaluations, ScaleNav achieves 94.9% SR on HabitatGS-LN, 86.5% on single-view Short-Horizon OVON, and 83.3% across 60 real-robot trials. These results support ScaleNav's ability to execute local navigation instructions robustly across the evaluated simulation benchmarks and real-world tasks.

Reviewer_wDoqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present ScaleNav, a scalable pre-training pipeline and model for language-conditioned short-horizon navigation. It first learns from 2,000 hours of internet ego-centric videos. They use this pre-training data on DiT to predict executable action chunks. They also present a benchmark HabitatGS-LN built on HabitatGS. They evaluate ScaleNav and some baseline methods on their benchmark and Short-Horizon OVON. Then they test the model on real quadruped robots.

Critical strengths

Strong results such as in real robots and fast inference (for the real-world)

Critical weaknesses

The architecture without internet video training already outperforming all fine-tuned baselines, raises the question of missing baselines or exploring other reasons this happens (such as less information presented to other models). For baselines, the paper do not include latest VLN baselines such as NaVILA or OmniNav.

Also the contribution seems incremental as the model DiT is taken from previous works.

Questions for authors

For the HabitatGS-LN dataset why restrict yourself to 3D? HabitatGS can support 4D Gaussian Splatting and since you are intaking video seems you could evaluate on moving obstacles (like pedestrians)

Why is your architecture without internet video training already outperforming all fine-tuned baselines? Is it because baselines like StreamVLN and Uni-NaVid used discrete actions while yours was in smaller blocks?

2 comments
AuthorsOpenReview ↗

Response to Reviewer wDoq

Thank you for the constructive feedback. We added evaluations with moving pedestrians, an action-interface control, and comparisons with recent navigation models. We also discuss why the variant without Internet-video training performs well and clarify the contribution beyond the inherited DiT backbone.

1. Navigation with moving pedestrians

Our original evaluation focused on static indoor navigation. We have now extended it to moving pedestrians to examine whether Internet-video training also benefits navigation in dynamic environments.

Specifically, we add animated pedestrians to the same 157 HabitatGS-LN tasks across 45 scenes and compare ScaleNav with its variant without Internet-video training. Pedestrian paths are pre-generated by direction-biased random walks within navigable areas, followed by obstacle-clearance checks and corner smoothing. Both models use the same paths at walking speeds of 0.3, 0.4, and 0.6 m/s. Success requires reaching and stopping at the instructed target within the original execution budget without pedestrian contact. The recorded collision rate is the fraction of attempted episodes with at least one recorded pedestrian contact.

Success rate (%) ↑ · 157 tasks per speed

Model0.3 m/s0.4 m/s0.6 m/s
ScaleNav85.9984.0877.07
ScaleNav w/o Internet videos79.6280.2575.16

Recorded pedestrian collision rate (%) ↓

Model0.3 m/s0.4 m/s0.6 m/s
ScaleNav8.2810.8315.29
ScaleNav w/o Internet videos10.8312.1017.83

ScaleNav has higher SR and lower recorded pedestrian collision rates at all three speeds. For example, at 0.4 m/s, SR rises from 80.25% to 84.08%, while the recorded collision rate falls from 12.10% to 10.83%.

Our simulated action-labeled data contain no dynamic scenes, and dynamic obstacle avoidance remains a limitation. Nevertheless, these gains suggest that Internet-video training may provide useful priors for navigation around moving obstacles. We will investigate this mechanism and extend dynamic-environment training in future work.

2. Performance sources and action-interface control

The variant without Internet-video training still retains the pretrained video backbone and our local-navigation adaptations. Initial target visibility does not determine an obstacle-aware motion sequence or preserve the instruction reference as observations change. Joint video–action modeling connects visual-dynamics priors to motion generation, while the initial-observation anchor preserves that reference. We believe these properties help explain why this variant performs well even without the additional Internet-video training.

To test whether the advantage persists with a discrete action interface, we keep both trained checkpoints unchanged and quantize their outputs at inference. Each predicted nine-step action chunk is integrated into a relative pose sequence, then approximated with the StreamVLN interface: 0.25 m forward moves and 15° turns. The executor receives these discrete actions.

HabitatGS-LN: submitted baselines and ScaleNav with inference-only action quantization.

MethodSR (%) ↑SPL (%) ↑
LocateAnything + NavDP82.8061.00
LocateAnything + ApexNav75.8066.80
StreamVLN (zero-shot)48.4040.00
Uni-NaVid (zero-shot)8.908.20
StreamVLN (fine-tuned)75.8059.50
Uni-NaVid (fine-tuned)82.2071.60
ScaleNav (discrete)89.1783.18
ScaleNav w/o Internet videos (discrete)85.3580.62

The checkpoints receive no training or adaptation for this discrete interface, and quantizing their continuous predictions introduces appreciable trajectory approximation errors. Despite this mismatch, ScaleNav (discrete) achieves 89.17% SR and its variant without Internet-video training achieves 85.35%, both exceeding the baseline results in the original submission. These results support an advantage from the learned navigation model beyond its continuous action interface.

3. Recent baselines

To assess ScaleNav against recent navigation models, we added LightNav, NaVILA, and OmniNav. ScaleNav retains the highest SR among the evaluated single-view methods on both benchmarks. CQ1 in our Common Questions response reports the zero-shot and fine-tuned comparisons, including OmniNav’s native three-view setting.

4. Contribution beyond the inherited DiT

We agree that the DiT follows prior work. Our contribution is to demonstrate the value of Internet egocentric videos for local navigation and provide a framework that uses the learned navigation priors to generate executable actions. No action labels are required for these videos. CQ2 in our Common Questions response explains this contribution and the evidence supporting it.

Thanks for addressing my main concerns and running additional experiments to back your claims. I’m still concerned about the novelty of the method as it uses DiT backbone mostly, thus I’m upgrading my score to 3 (Weak accept)

Reviewer_YpKoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ScaleNav, which uses action-free Internet egocentric videos to improve language-conditioned local navigation with initially visible targets. It combines instruction-conditioned video pretraining with simulated action alignment and joint fine-tuning to predict future observations and actions within one model. Experiments on two simulation benchmarks and a Unitree Go2 demonstrate benefits from this training pipeline, alongside a human-checked HabitatGS-LN benchmark.

Critical strengths

S1. The evaluation design makes the results reasonably interpretable. HabitatGS-LN uses held-out scenes and is checked by two human reviewers. The paper distinguishes single-view Short-Horizon OVON from the original three-view protocol and reports both zero-shot and simulation-fine-tuned versions of StreamVLN and Uni-NaVid. These details reduce ambiguity from protocol differences.

S2. The benefits extend beyond simulation. The robot evaluation covers 12 tasks and 60 trials per method, with 83.3% SR for the full model versus 65.0% without Internet videos (App. E). The full model performs better on 8 tasks and ties on 4, so the aggregate improvement is not driven by a single task.

Critical weaknesses

W1. The interpretation of the Internet-video benefit goes beyond what the ablation isolates. The full model receives 10,000 additional Stage 1 updates, whereas “w/o Internet videos” starts from the Wan weights and runs only Stages 2 and 3. This comparison changes the video source, training budget, and adaptation to chunk-wise causal prediction together. It supports adding Stage 1, but does not establish how much of the benefit comes from Internet videos themselves. The most useful control would use the existing simulated videos with a matched Stage 1 budget and prediction objective. Without it, the interpretation should remain at the level of the complete training pipeline. The explanation in terms of goal-directed visual dynamics should also acknowledge the absence of an instruction-conditioning ablation.

W2. The pseudo-action control does not establish which approach is preferable. The pseudo-action branch achieves 73.9% SR on HabitatGS-LN, below the 90.4% of the no-Internet-video variant (Table 2). The paper specifies Depth Anything 3 pose estimation, equal per-clip mixing, and no confidence weighting, but does not report retained pseudo-label counts or speed/scale alignment between human video and robot motion. The lower result could reflect label error, the mixing ratio, or an action-distribution mismatch. These settings need clarification; until then, the conclusion should be restricted to the tested pseudo-action configuration.

Questions for authors
  1. Could you list the training budget for each stage of every variant and provide or discuss the simulated-video control with a matched budget and prediction objective described in W1?
  2. How many pseudo-action clips are retained, what are the actual sampling proportions, and how are speed and scale aligned? Are mixing-ratio sensitivity results available to assess whether this setting drives the performance drop?
  3. Could you provide paired episode results or uncertainty estimates for the key differences? The 94.9% versus 94.3% HabitatGS-LN results differ by about one episode, so that ranking should not be emphasized. For the robot results, please clarify the success-judgment procedure and qualify generalization by the 12 tasks, each evaluated with a fixed initial pose and instruction.
3 comments
AuthorsOpenReview ↗

Response to Reviewer YpKo (1/2)

Thank you for the constructive feedback. In Part 1, we report the stage-wise training budgets and simulated-video control, then clarify the pseudo-action data, motion scale and speed, and sampling-ratio results. Part 2 addresses uncertainty, real-robot evaluation, and instruction conditioning.

1. Training budgets and simulated-video control

We agree that the original comparison does not separate the benefit of Internet videos from that of additional training. Training budgets are reported below as optimizer updates. Global batch sizes are 64, 64, and 32 for Stages 1–3, respectively. The two numbers for Stage 1 denote initial video adaptation and subsequent causal chunk training.

ScaleNav variants: stage-wise training budgets (optimizer updates)

VariantStage 1Stage 2Stage 3
ScaleNav2,000 + 8,0002,5006,500
w/o Internet videos02,5006,500
w/o video pretraining02,5006,500
w/o replay2,000 + 8,0002,5006,500
w/o Stage 22,000 + 8,00006,500
w/o anchor2,000 + 8,0002,5006,500
Pseudo-action control02,5006,500
Simulated-video control2,000 + 8,0002,5006,500

As an additional control, we pretrained the model in Stage 1 on videos from our simulated training episodes, matching the Stage 1 budget and prediction objective, and then performed action alignment and joint fine-tuning on simulated data. This control achieves 92.36% SR on HabitatGS-LN and 77.29% on OVON, compared with 94.90% and 86.45% for ScaleNav.

The matched-budget control supports the value of Internet egocentric videos beyond additional Stage 1 adaptation. Pretraining on simulated videos improves HabitatGS-LN SR from 90.45% to 92.36%, but ScaleNav’s complete Internet-video training scheme, including pretraining and replay, performs better on both benchmarks under the same Stage 1 budget. The larger gain on OVON, whose scene distribution differs from our simulated training data, suggests that learning from these videos may improve transfer across scene distributions. We will investigate this effect further.

2. Pseudo-action data and sampling ratios

The pseudo-action training set contains 18,135 video segments and 34,400 action chunks; the simulated set contains 2,460 episodes and 12,038 action chunks. In the original experiment, pseudo-action data receive 88.06% of the sampling probability, with uniform chunk sampling within each domain.

For motion scale, we derive relative planar displacements and yaw changes from the metric-scale camera poses estimated by Depth Anything 3 (DA3NESTED-GIANT-LARGE). We do not additionally match the speed distributions of human and simulated robot motion. Estimated poses and metric scale remain subject to error, which can propagate into action supervision; motion-distribution differences provide another possible source of degradation.

We also extended the original experiment to 25%, 50%, 75%, and 100% pseudo-action sampling. The additional runs share the same initialization. The table includes the original 0% and 88.06% results and ScaleNav, which learns from Internet egocentric videos without pseudo-action labels.

Pseudo-action sampling ratios and ScaleNav: SR (%) † Same subset and evaluation protocol as in the paper.

Training settingHabitatGS-LN SR (%)Short-Horizon OVON† SR (%)
0% (original)90.4581.11
25%91.0857.06
50%87.9054.58
75%90.4560.88
88.06% (original)73.8954.20
100%31.2147.52
ScaleNav94.9086.45

No tested ratio improves SR on both benchmarks. At mixed ratios, the degradation is more pronounced on OVON, whose scene source and rendering differ from our simulated training data. This pattern suggests that imperfect motion supervision may impair cross-dataset transfer. At high pseudo-action ratios, performance deteriorates substantially on both benchmarks. Errors in pose and scale estimation, together with differences between human and robot motion distributions, may contribute to this performance degradation.

AuthorsOpenReview ↗

Response to Reviewer YpKo (2/2)

This part reports the paired uncertainty analysis, clarifies real-robot success judgment and the scope of the real-robot findings, and discusses the evidence for instruction conditioning during video pretraining.

3. Paired uncertainty estimates

We agree that the 94.9% versus 94.3% ranking should not be emphasized. We conducted five inference evaluations of all seven variants on the same 157 HabitatGS-LN tasks, giving 785 rollouts per variant. The table reports mean SR ± standard deviation and paired 95% bootstrap confidence intervals for the SR difference: ScaleNav minus each variant, in percentage points. Scenes and inference evaluations are resampled in matched pairs.

ScaleNav ablations on HabitatGS-LN: five inference evaluations; SR differences in percentage points

VariantSR (%), mean ± SDSR difference (pp) [95% CI]
ScaleNav94.27 ± 1.01—
w/o Internet videos91.08 ± 1.19+3.18 [0.00, 6.54]
w/o video pretraining90.06 ± 1.99+4.20 [0.12, 8.70]
Original pseudo-action configuration77.07 ± 2.66+17.20 [11.67, 22.82]
w/o Stage 293.63 ± 1.69+0.64 [-2.48, 4.18]
w/o replay90.45 ± 0.78+3.82 [0.50, 7.26]
w/o anchor90.83 ± 0.97+3.44 [0.58, 6.62]

Internet-video training, video pretraining, replay, and the anchor each improve mean SR over their corresponding ablations, with nonnegative lower bounds for the reported 95% confidence intervals. ScaleNav also outperforms the tested pseudo-action configuration. These results support the observed benefits of the training framework and Internet-video use.

Stage 2 has a smaller and less certain effect, with a confidence interval that includes both negative and positive differences. Its isolated contribution requires further evaluation.

4. Real-robot evaluation

We evaluate 12 tasks across three environments, with five trials per method per task. Each task has a fixed instruction, starting position, and heading. Before every trial, the operator returns the robot to the floor markers and aligns its heading.

During each trial, the robot executes the method’s commands until the method signals STOP or the execution-step limit is reached. Once it comes to rest, the operator measures the shortest horizontal gap between the robot’s main body and the instructed target object with a tape measure, using their projections onto the floor. A trial succeeds if the method signals STOP within the execution budget and the final gap to the specified target is at most 1.25 m. Otherwise, it is counted as a failure.

These results provide initial evidence of transfer to physical navigation in the evaluated environments. Broader generalization across environments, starting poses, and instructions remains to be evaluated.

5. Instruction conditioning

We agree that the current experiments do not isolate the contribution of instruction conditioning during video pretraining. Goal-directed prediction describes our training objective; its role in explaining the gain remains a hypothesis. We will clarify this distinction and further investigate instruction conditioning in future work.

Thank you for the thorough follow-up experiments and clarifications, which resolve my main methodological concerns; however, as the contribution primarily lies in a well-executed training pipeline with relatively modest conceptual novelty, I will maintain my Weak Accept score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Common Questions: Recent baselines and core contributions

We address the shared questions on recent baselines (CQ1) and core contributions (CQ2) below.

CQ1. Comparison with recent navigation models

We evaluated LightNav, NaVILA, and OmniNav on both benchmarks. LightNav is tested zero-shot because its official release does not provide training code; NaVILA and OmniNav are tested both zero-shot and after fine-tuning on our simulated navigation data. We provide OmniNav with three RGB views, following its native input setting. It therefore receives additional visual information compared with the single-view RGB input used by ScaleNav and the other baselines. We report the single-view comparison and the three-view reference separately below.

Single-view comparison: SR / SPL (%). Bold marks the best single-view results. † Same subset and evaluation protocol as in the paper.

MethodHabitatGS-LNShort-Horizon OVON†
LightNav (zero-shot)49.04 / 46.5360.11 / 53.60
NaVILA (zero-shot)39.49 / 35.9248.47 / 40.89
NaVILA (fine-tuned)87.26 / 73.9184.35 / 70.70
ScaleNav w/o Internet videos90.45 / 80.881.11 / 65.5
ScaleNav94.90 / 86.186.45 / 71.3

Three-view reference (additional RGB views): SR / SPL (%).

MethodHabitatGS-LNShort-Horizon OVON†
OmniNav (zero-shot, three-view)60.51 / 45.8679.20 / 64.49
OmniNav (fine-tuned, three-view)85.99 / 75.3387.40 / 76.32

ScaleNav maintains an advantage over the evaluated recent single-view methods, achieving the highest SR on both benchmarks. It also outperforms three-view OmniNav on HabitatGS-LN. On OVON, three-view OmniNav obtains 87.40% SR versus 86.45% for ScaleNav.

CQ2. Core contributions

We agree that the backbone, causal attention, and basic training objectives build on existing techniques. Our contribution is to establish the practical value of Internet egocentric videos for local navigation and provide a framework that uses navigation priors learned from these videos to generate executable robot actions. The task requires preserving the instruction reference across viewpoints, updating motion from new observations, and terminating at the target. The initial-observation anchor, causal action-chunk prediction, and zero-motion supervision support these capabilities within one framework.

Our experiments show that Internet egocentric videos improve executable local navigation without requiring action labels for those videos, providing a practical, scalable source of navigation experience. The staged training connects this data source to action generation, and the ablations and additional controls examine its benefits. The resulting ScaleNav model performs strongly in simulation and navigates successfully on a real quadruped robot; the added recent-baseline comparisons further assess its effectiveness for this task. We also systematically characterize failures in navigation to initially visible targets, showing that instruction grounding and arrival judgment remain challenging for the evaluated baselines.

Area_Chair_KY6DOpenReview ↗

Please review the authors’ rebuttal and update your assessment

Dear reviewers,

Thank you for your reviews, and thank you to the authors for their responses. Please read the authors’ rebuttal and discuss whether it addresses your main concerns. Please update your review and rating if warranted, and briefly explain your current assessment, including any concerns that remain unresolved. If your assessment is unchanged, please explain why.

Thank you for your continued engagement.

AuthorsOpenReview ↗

Update to CQ1: Real-Robot Comparisons with Recent Navigation Baselines

Following our Common Questions response, we have completed additional real-robot comparisons with LightNav and NaVILA on the Unitree Go2. We evaluate LightNav zero-shot and NaVILA both zero-shot and after fine-tuning on our simulated navigation data.

We use the same three environments, scene layouts, object placements, and 12 navigation tasks as in the paper. The instructions, marked starting positions and headings, execution budget, and success criterion are unchanged. Each method is evaluated five times per task, giving 60 trials per method. We report ScaleNav's original results alongside these new baseline evaluations.

Real-robot success rates by environment (%)

MethodHallLoungeOfficeOverall
LightNav (zero-shot)55.040.015.036.7
NaVILA (zero-shot)15.025.050.030.0
NaVILA (fine-tuned)45.050.040.045.0
ScaleNav85.085.080.083.3

ScaleNav achieves 83.3% SR, compared with 45.0% for fine-tuned NaVILA, the strongest newly added baseline. These results suggest that ScaleNav is more robust on the evaluated real-robot tasks.

During real-robot testing, the baseline methods exhibited incorrect target selection, collisions or blockage, and premature stopping or overshooting. In the second-door task, for example, LightNav and zero-shot NaVILA sometimes approached the first door instead.

These difficulties may partly reflect the baselines' original task emphases and training distributions. NaVILA was developed primarily for instruction following, while LightNav was trained on a mixture of instruction following, object-goal navigation, and tracking. Although we fine-tune NaVILA on our local-navigation data, its original design and pretrained representations may still influence its real-robot performance.

Results for each target are reported below as successes / trials and grouped by environment.

Results by target (successes / trials)

EnvironmentTargetLightNav (zero-shot)NaVILA (zero-shot)NaVILA (fine-tuned)ScaleNav
HallDisplay counter2 / 51 / 50 / 55 / 5
HallRocket model1 / 50 / 50 / 53 / 5
HallScreen on the rack5 / 52 / 55 / 55 / 5
HallSecond door3 / 50 / 54 / 54 / 5
LoungeWooden cabinet2 / 52 / 53 / 53 / 5
LoungeLeftmost white cylinder2 / 50 / 53 / 54 / 5
LoungeSecond table1 / 52 / 50 / 55 / 5
LoungeVending machine3 / 51 / 54 / 55 / 5
OfficeCabinet under the printer1 / 50 / 50 / 53 / 5
OfficeLeft trash can0 / 52 / 50 / 55 / 5
OfficeGlass door2 / 53 / 55 / 55 / 5
OfficeTripod phone stand0 / 55 / 53 / 53 / 5

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.