- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the challenges of using JEPA-based world models for first-person navigation, including perceptual aliasing, obstacle avoidance, and generalization to unseen maps. The authors introduce PADoom, a benchmark that provides both first-person and top-down views of the same environments. They also propose JEPath, a world model that predicts future visual features and ego-motion. Instead of using visual similarity for planning, JEPath predicts future trajectories and selects actions based on their distance to the goal. Experiments show that JEPath outperforms existing world models and goal-conditioned policies on both seen and unseen maps.
Critical strengths
-
The relative ego-motion formulation shows promising results. The paper shows that predicting relative ego-motion can be more effective than directly predicting absolute positions. The ablation results provide some evidence for this design choice.
-
The proposed framework achieves competitive navigation performance. JEPath outperforms existing world models and goal-conditioned policies on both seen and unseen maps. In particular, its improvements over goal-conditioned policies that use the same spatial goal information suggest that the framework offers benefits beyond simply changing the goal representation.
Critical weaknesses
1. The task formulation limits the significance of addressing perceptual aliasing. The paper identifies perceptual aliasing as a major challenge in image-goal navigation. However, JEPath changes the setting to point-goal navigation, where the agent's current pose and goal coordinates are assumed to be known. I think this is essentially solving a different problem, rather than directly addressing the original challenge. In many real-world navigation tasks, such spatial information is not readily available. More importantly, perceptual aliasing is largely avoided by providing the relative spatial relationship between the agent and the goal, rather than being resolved by the proposed world model. This makes the claimed contribution to addressing perceptual aliasing less convincing.
2. The proposed planning objective does not fully address obstacle-aware navigation.
The paper argues that successful planning requires the cost function to reflect the geodesic distance to the goal. However, JEPath only minimizes the Euclidean distance between the predicted endpoint and the goal. Even with accurate motion predictions, this objective may fail when avoiding obstacles requires temporarily moving away from the goal. The method relies on searching for a complete detour within a limited planning horizon, which may not work for larger or more complex obstacles. I think this limitation is particularly important because obstacle avoidance is one of the main challenges the paper aims to address, yet the current design does not fully satisfy the planning conditions identified by the authors.
Questions for authors
- How does the method handle obstacles that require temporarily moving away from the goal, especially when the detour exceeds the planning horizon? Could the authors provide experiments on such challenging cases?
- The training trajectories are collected using A* and a path-following controller that tends to avoid collisions, while CEM may generate many actions that lead to obstacles. How does JEPath reliably predict motion for these potentially out-of-distribution actions?
- In the considered setting, actions are discrete and have fixed execution durations. I think it should be possible to estimate ego-motion directly from the action sequence, which may already be reasonably accurate when there are no collisions or obstacles. Could the authors provide a baseline that uses such action-based motion estimates to predict and evaluate candidate trajectories for navigation? This would help clarify how much the learned ego-motion prediction actually contributes to the performance.
- The evaluation mainly uses short navigation tasks, and the paper acknowledges that unseen-map episodes may be slightly easier than seen-map episodes. Could the authors provide difficulty-matched evaluations and more challenging long-horizon tasks to better support the generalization claims?