acceptodds
Withdrawn from ICLR 2027

WorldPixelNav: Deployment-Oriented Pixel-Goal Navigation with Future-State-Aware Trajectory Generation and Hierarchical Correction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pixel-goal navigation bridges high-level reasoning and low-level control in vision-language navigation by selecting an image pixel as the next local target. Existing imitation-based planners, however, treat the pixel goal as a static target and do not model the spatial consequences of their motion, leading to trajectory drift and collision. We propose WorldPixelNav, a deployment-oriented pixel-goal navigation framework with future-state-aware trajectory generation and hierarchical correction. WorldPixelNav converts the pixel goal into a robot-centric planar coordinate, encodes depth observations, motion history, this coordinate, and a fixed instruction into shared conditional memory, predicts a future spatial latent from this memory, and decodes the latent into continuous waypoints. Training combines trajectory imitation, future-state prediction, and deployment-oriented supervision. During execution, safety verification triggers trajectory reconnection, in-place re-observation, or historical-viewpoint recovery without requesting a new high-level pixel goal. WorldPixelNav achieves 96.48% success rate and 0.07% collision-episode rate in unseen simulated scenes, and 83.3% and 6.7% in unseen real-world scenes.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vln, pixel-goal navigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_f8jmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes WorldPixelNav, a deployment-oriented framework for pixel-goal navigation. The method combines future spatial-state prediction using Cosmos Predict 2.5, trajectory generation through a Transformer decoder, deployment-oriented training objectives, and hierarchical execution-time correction. Specifically, it predicts a future latent spatial representation to guide the generation of 32 waypoints, incorporates goal-proximity, progress, smoothness, and collision-avoidance losses, and introduces three recovery mechanisms: trajectory reconnection, in-place re-observation, and historical-viewpoint recovery.

Experiments on 12,000 simulated samples from 31 unseen scenes and a physical robot demonstrate promising results. The complete system achieves a reported simulation success rate of 96.48% and collision rate of 0.07%, with an average real-robot success rate of 83.3%.

Overall, the paper addresses an important practical problem and presents a comprehensive system. However, I have concerns about the novelty of the core modeling approach, the evaluation protocol, and whether the experimental evidence sufficiently supports the claims regarding generalization and real-world deployment.

Critical strengths

1.Practical relevance: The paper addresses important challenges in trajectory planning, collision avoidance, and reliable robotic deployment. 2.Comprehensive design: The integration of future-state prediction, multiple training objectives, and hierarchical correction is reasonable and supported by ablation studies. 3.Promising results: The method achieves strong simulation performance and provides preliminary validation on a real robot.

Critical weaknesses
  1. The methodological novelty is not sufficiently established. The future-state prediction module appears closer to auxiliary future-feature prediction than a genuine planning-oriented world model, since it is not explicitly conditioned on candidate actions or generated trajectories. The paper does not sufficiently establish its advantage over conventional future-feature prediction or multitask learning.
  2. The pixel-goal construction may limit the generality of the evaluation. Pixel goals are derived from expert trajectories, which may overestimate performance compared with arbitrary or imperfect goals generated by real high-level models. Robustness to target perturbations and invalid depth measurements remains unclear.
  3. The contribution of the core model is not fully isolated. The proposed method combines future-state prediction, multiple auxiliary losses, and hierarchical safety correction. Comparisons applying equivalent safety mechanisms to the baselines would better isolate the contribution of the core model.
  4. Real-robot validation is limited. Physical experiments cover only three scenarios with ten trials each. More diverse environments and statistical uncertainty estimates are needed to support claims of robust deployment.
Questions for authors
  1. How does the proposed future-state predictor differ fundamentally from conventional future-feature prediction, and what evidence establishes its independent contribution?
  2. Does the method generalize to pixel goals not derived from expert trajectories?
  3. Does its advantage persist when the baselines use equivalent safety-checking and recovery mechanisms?
Reviewer_yNwAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents WorldPixNav, a pixel-navigation method that is deployment-oriented and has a hierarchical correction stage. The work borrows ideas from future-aware vision-language navigation and hierarchical planning and control, and incorporates them in the pixel navigation setting. The proposed method shows significant performance gains relative to the baseline on a variant of the R2R dataset. The ablation studies demonstrate the contribution of the various loss components and the three-level hierarchy used for path correction.

Critical strengths

The future spatial-state prediction and the hierarchical correction are shown to be effective in the pixel navigation setting. The ablation study demonstrates the contribution of each of loss terms and the hierarchical correction stages to the overall performance. The performance of the proposed method on the InternVLA-CE-R2R dataset is significantly higher than competing techniques.

Critical weaknesses

The main weaknesses of this work are related to novelty, limited experimental validation, and importantly, missing important relevant references. These are elaborated below.

The core ideas of future spatial-state prediction and path correction are not novel. They have been applied to the pixel navigation setting in this work. A few relevant missing references are listed below:

Generating future-view semantics: https://openaccess.thecvf.com/content/CVPR2023/html/Li_Improving_Vision-and-Language_Navigation_by_Generating_Future-View_Image_Semantics_CVPR_2023_paper.html

DreamNav: https://ieeexplore.ieee.org/abstract/document/11697131

FutureNav (mentioned but not cited): https://github.com/linglingxiansen/FutureNav, https://arxiv.org/pdf/2606.30367

LatentPilot: https://link.springer.com/chapter/10.1007/978-3-032-37359-5_5. https://arxiv.org/pdf/2603.29165

VISTA: https://ieeexplore.ieee.org/abstract/document/11696563

Anticipate before acting: https://openreview.net/pdf?id=eQOczZdaL

A major miss is the total lack of literate coverage on path correction. Again, a few references are listed below. https://ieeexplore.ieee.org/abstract/document/11417380?casa_token=83M_mcSDQ-AAAAAA:ICPf0z5V1E0Gb-NUp2eZ-GKZi7uKR42BG0YBErLwguw4nXGF6arGkErJ3NyupZrZVmGBO_Eec_sLiw

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0341589

https://ieeexplore.ieee.org/abstract/document/10379540?casa_token=0_lgiBAuIxYAAAAA:HCfCB9CkhhIlDl5CvC6UGhn3n6CEZeTnvQ4pUzJ2EtTzXjXNAPup1ee12hPQObmjB_9Op8JAomVktQ

https://ieeexplore.ieee.org/abstract/document/10865457?casa_token=MezrMenYV1MAAAAA:GPKkGoCxtXQx6SBoAnbXag9RNS-VwfRPbObmLxcQ7ddwrSTXwUoprNWCHoEH6IUXT725LkAyLyV4Eg

https://advanced.onlinelibrary.wiley.com/doi/full/10.1002/adrr.202500072

Cross-dataset validation, a standard approach to demonstrate generalization in VLN, has not been done. The authors are encouraged to do this. Further, the current experimental validation is limited to one dataset.

The computational costs of the proposed approach has not been presented.

Another weakness is that there is no discussion on why the proposed method works well. The ablation studies present the contribution of the loss elements and the hierarchical correction stages. But this doesn’t give a reason for the success.

Questions for authors

First and foremost, why is the proposed method effective? Please present a clear justification.

Can you bring out the novelty of the method? Based on the current presentation, it appears to have reused ideas from the literature (and not all of them are cited).

Can you please revisit the literature coverage and improve it? A few relevant ones are listed above.

Can you cross-dataset validate the proposed method? The current validation is limited to one dataset.

Reviewer_8PAwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WorldPixelNav, a low-level pixel-goal navigation method with future-state-aware trajectory generation and hierarchical correction. It can convert a pixel goal into a robot-centric planar coordinate and predict a future spatial representation from depth and motion history.

Critical strengths
  1. The method maintains a persistent pixel goal. It updates the robot-centric coordinates of the pixel goal rather than regrounding the pixel goal again, which is more valuable and cleaner.
  2. The three-level hierarchical recovery is practical in close-loop design, where it first reconnects to a safe further waypoint, then replans or returns to the prior when necessary.
Critical weaknesses
  1. The benchmark's design favors the proposed method. For example, it is designed to reach the target with a long, 32-waypoint plan plus its own recovery system. However, methods such as DualVLN and NavDP are designed in a different way, where constant replanning is needed.
  2. World models usually refer to predicting future changes after the predicted action. Yet the world model in this work predicts the future latent conditioned on past motion history. Hence, this raises the concern of whether it gains from extra supervision of future depth frames rather than using Cosmos.
Questions for authors
  1. Can the author provide a fair comparison with other baselines, such as DualVLN and NavDP?
  2. Can the author provide more ablation on whether the gain is from extra supervision on future depth prediction, or using Cosmos?
Reviewer_UEEuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary
  1. WorldPixelNav predicts a future spatial latent from depth history, motion history, and a pixel goal, and decodes it into a continuous waypoint trajectory. It further uses deployment-oriented training with goal-proximity, motion-progress, smoothness, and collision constraints, together with a three-level closed-loop trajectory-correction mechanism.
  2. The method is evaluated in unseen simulated environments and a small-scale real robot. Ablation studies on the loss terms and correction modules are conducted to analyze the contribution of each component.
Critical strengths
  1. Compared with visual navigation methods conditioned on language instructions or target objects, WorldPixelNav introduces future spatial-state prediction from world models into continuous pixel-goal trajectory planning, addressing the limitation that existing methods primarily rely on current observations and lack explicit modeling of future states.
  2. WorldPixelNav goes beyond trajectory imitation by incorporating deployment-oriented supervision for goal proximity, motion progress, smoothness, and collision avoidance, together with a hierarchical trajectory-correction mechanism.
Critical weaknesses
  1. The method primarily relies on module-removal ablations and does not establish that the proposed future-state latent is superior to alternative designs, such as a current-state latent, an explicit geometric representation, or direct trajectory prediction.
  2. The future state in this paper is represented by features produced by a depth encoder and aligned with future depth features using a cosine loss. The trajectory decoder then directly conditions waypoint generation on this latent. However, the paper does not clearly characterize what future information the latent captures, nor demonstrate that it provides capabilities beyond a standard trajectory regressor conditioned on current observations. The method could therefore maybe interpreted as latent-conditioned trajectory regression based on current depth and goal information, rather than genuine future-state-aware world modeling.
  3. The real-robot evaluation covers only three scenes with ten trials per scenario, while the main baselines are evaluated under a fixed-goal, one-shot long-horizon protocol, even though some of them were originally designed for receding-horizon execution. The current results therefore do not clearly disentangle the method’s intrinsic advantages from those arising from protocol compatibility, nor do they establish robustness in real-world environments.
Questions for authors
  1. Once the initial pixel goal is converted into a fixed world-frame coordinate, how to determine whether this target remains valid under depth-estimation errors, pose drift, or target motion?
  2. After normalization and clipping, can the model still distinguish between distant targets at different ranges, and how does this representation affect long-range navigation? What is the range of the normalization scale sgs_g, and how was it selected?
  3. The latent is supervised using the cosine distance between predicted and future depth features, but the paper does not establish whether it preserves planning-relevant geometric information, such as target location, obstacle layout, or free-space structure. so future-state-aware appears more like a training constraint than an explicit spatial-modeling capability.
  4. The three-level correction mechanism in Sec3.4 introduces rotation, backtracking, and replanning, but the paper does not report their additional execution time, path length, or model-call frequency. unclear whether the improvement in success rate comes at a substantial execution cost.
  5. Although the paper reports a low collision rate, it does not sufficiently attribute the remaining failures to perception errors, pose errors, trajectory-generation errors, or failures of the correction mechanism.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.