- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces OmniMapNavBench, a map-centric navigation benchmark pairing point, trajectory, and language goals with overhead RGB maps. The proposed OmniMapNav model uses separate agent-to-map and goal-to-map grounding branches to retrieve relevant map features. An action-conditioned predictor estimates next-observation semantic features, which refine action scores. Experiments on scene-disjoint simulated environments report improvements across goal conditions and distance thresholds. The paper also provides component ablations, spatial-grounding diagnostics, and real-robot evaluations following adaptation on a small set of collected trajectories.
Critical strengths
-
The benchmark addresses a useful navigation setting in which an overhead map supports both destination specification and spatial reasoning. Pairing different goal cues for the same scene, start, and target enables controlled comparisons of their contributions, while the scene-disjoint evaluation and varied multi-room layouts provide a meaningful test of navigation in unfamiliar environments.
-
The experimental analysis provides useful support for the proposed design. OmniMapNav improves average success over the strongest adapted baseline at both 3 m and 1 m. The spatially supervised dense-reader control and the future-loss ablation help separate structured retrieval and semantic supervision from auxiliary supervision and added capacity, respectively. The real-robot results provide additional evidence of practical feasibility.
Critical weaknesses
-
The interpretation of the predictor as action-conditioned semantic foresight would benefit from more direct validation. Only the demonstrated action receives future-feature supervision, and prediction accuracy for alternative actions is not reported. The loss-only ablation supports the usefulness of the auxiliary target, but the large degradation after disabling the learned scorer establishes reliance on that pathway rather than accurate counterfactual prediction. Measuring feature-prediction quality across actions and testing action-shuffled targets would help distinguish predictive semantics from auxiliary regularization.
-
The generalization evidence for intuitive multimodal goal specification remains limited. All test language grammar structures and vocabulary occur in training, and simulated trajectory cues are sparse anchors derived from expert routes rather than a collection of user-drawn sketches. Evaluation with free-form or compositionally held-out instructions and imperfect sketches would strengthen the contribution. The real-robot experiments are encouraging, but the separation between adaptation and evaluation environments should also be clarified before drawing broader deployment conclusions.
-
The results average three evaluation rounds, but training-seed variability and uncertainty intervals are not reported. This matters particularly for the approximately two-point gain from future-feature supervision, the smaller belief-propagation effect, and the very small advantage of Combined over Trajectory alone. Independent training runs and paired, scene-aware uncertainty estimates would help establish which component and cue-combination differences are reliable, beyond the larger overall improvement over the baselines.
Questions for authors
See weakness
Dear Reviewers,
Thank you for your efforts so far. As we are at the end of the discussion period, I would like to kindly encourage you to review the authors' rebuttals and engage in further discussions, particularly if the authors have addressed your questions or concerns.
Thank you for your time and contributions!
AC