acceptodds
Under review as a conference paper at ICLR 2027

OmniMapNav: Map-Guided Multimodal Goal Navigation via Dual-Grounding Map Reading and Lightweight Semantic World Modeling

Abstract

Visual navigation should support intuitive goal specification and efficient adaptation to unfamiliar environments. However, PointNav, image-goal navigation, and conventional route-following vision-language navigation require coordinates, reference images, or route descriptions that users may not readily provide; ObjectNav category labels leave ambiguity among target instances. Without a global spatial prior, inferring unfamiliar layouts from partial observations can require extensive exploration in large, multi-room environments. To bridge this gap, we introduce OmniMapNavBench, where one overhead RGB map serves as an interactive goal interface and a navigation prior. Its 50,241 episodes across 4,724 scenes pair map-selected points, hand-drawn route sketches, and language grounded in map–room, map–object, and object–object relations for evaluating individual cues and their combinations. Effective map utilization requires grounding local observations and heterogeneous goals in this common spatial frame. To this end, we propose OmniMapNav: its Dual-Grounding Map Reader estimates an agent pose-heading belief and a goal anchor field to retrieve agent- and goal-relevant map tokens. Furthermore, a Semantic Foresight Predictor estimates one-step, action-conditioned next-observation features; a semantic foresight scorer combines these with grounded context to refine action scores. OmniMapNav achieves an average geodesic success rate of 37.31% at 3 m versus 26.22% for the strongest baseline, with gains across all four goal conditions and stricter thresholds. Real-robot experiments demonstrate map-based goal specification and closed-loop navigation. Code is available at https://anonymous.4open.science/r/omnimapnav-9DDF.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied navigation, visual navigation, map-guided navigation, multimodal goal conditioning, spatial grounding, world models, robot learning, datasets and benchmarks

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_oyxHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OmniMapNavBench, a map-centric navigation benchmark pairing point, trajectory, and language goals with overhead RGB maps. The proposed OmniMapNav model uses separate agent-to-map and goal-to-map grounding branches to retrieve relevant map features. An action-conditioned predictor estimates next-observation semantic features, which refine action scores. Experiments on scene-disjoint simulated environments report improvements across goal conditions and distance thresholds. The paper also provides component ablations, spatial-grounding diagnostics, and real-robot evaluations following adaptation on a small set of collected trajectories.

Critical strengths
  1. The benchmark addresses a useful navigation setting in which an overhead map supports both destination specification and spatial reasoning. Pairing different goal cues for the same scene, start, and target enables controlled comparisons of their contributions, while the scene-disjoint evaluation and varied multi-room layouts provide a meaningful test of navigation in unfamiliar environments.

  2. The experimental analysis provides useful support for the proposed design. OmniMapNav improves average success over the strongest adapted baseline at both 3 m and 1 m. The spatially supervised dense-reader control and the future-loss ablation help separate structured retrieval and semantic supervision from auxiliary supervision and added capacity, respectively. The real-robot results provide additional evidence of practical feasibility.

Critical weaknesses
  1. The interpretation of the predictor as action-conditioned semantic foresight would benefit from more direct validation. Only the demonstrated action receives future-feature supervision, and prediction accuracy for alternative actions is not reported. The loss-only ablation supports the usefulness of the auxiliary target, but the large degradation after disabling the learned scorer establishes reliance on that pathway rather than accurate counterfactual prediction. Measuring feature-prediction quality across actions and testing action-shuffled targets would help distinguish predictive semantics from auxiliary regularization.

  2. The generalization evidence for intuitive multimodal goal specification remains limited. All test language grammar structures and vocabulary occur in training, and simulated trajectory cues are sparse anchors derived from expert routes rather than a collection of user-drawn sketches. Evaluation with free-form or compositionally held-out instructions and imperfect sketches would strengthen the contribution. The real-robot experiments are encouraging, but the separation between adaptation and evaluation environments should also be clarified before drawing broader deployment conclusions.

  3. The results average three evaluation rounds, but training-seed variability and uncertainty intervals are not reported. This matters particularly for the approximately two-point gain from future-feature supervision, the smaller belief-propagation effect, and the very small advantage of Combined over Trajectory alone. Independent training runs and paired, scene-aware uncertainty estimates would help establish which component and cue-combination differences are reliable, beyond the larger overall improvement over the baselines.

Questions for authors

See weakness

Reviewer_Gak2OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses a single overhead RGB map both to specify the goal and to give the agent the layout of the house. Users can click a point on the map, draw a rough route, or describe the target relative to the map. The authors build OmniMapNavBench in ProcTHOR, with 50,241 episodes in which every start–target pair comes with all three goal types. Their model, OmniMapNav, has two components. DGMR estimates where the agent is (ABQ) and where the goal is (GAQ) on the map and reads map features from those locations. SFP predicts the next frame's CLIP features for each candidate action and uses them to adjust the action scores. The model reaches 37.31% average GSR@3m, compared with 26.22% for ViNT, and succeeds in 54 of 60 real-robot trials.

Critical strengths
  1. Using the same map for goal input and for layout information is a well-motivated task setting. Every start–target pair has all three goal types, so the paper can compare them on identical episodes.

  2. The paper reports how accurately ABQ localizes the agent and how accurately GAQ localizes the goal, in meters (Table 4). It also includes a case where Language grounding fails (Fig. 10).

Critical weaknesses
  1. Only the prediction for the demonstrated action is supervised. The other five predictions may not look like the next observation at all. Three checks would help. First, compare those five predictions with frames actually rendered after each action, which is easy in AI2-THOR. Second, train with the future-feature loss but without the scorer. Third, score each action with the prediction from a different action.

  2. DGMR gives the policy 8 map tokens, while the dense reader gives it 1024. Fewer tokens alone can help behavior cloning. To attribute the gain to the location estimates, the paper should compare against the same 8 queries without the belief gating and without the goal-anchor bias.

  3. Adding goal types does not always help. In Table 9, T+L reaches 43.18% while All reaches 41.67%, and P+T reaches 40.37% while Trajectory alone reaches 41.56%. The paper says Combined achieves the highest GSR, which holds only among the four main conditions. The averaging of per-cue logits in Eq. 4 is a likely cause. Comparing the entropy of the anchor field with and without Point would test this.

Questions for authors
  1. Translation odometry is in map-aligned axes and is not rotated by θ₀, which leaks absolute heading after the first forward move. How much of ABQ's heading estimate comes from this?

  2. In the released code, _route_preference in omnimapnavbench.py and extract_goals in spoc_nav_utils.py take the four Trajectory anchors from the expert path at indices round(i·(n−1)/3). The first anchor is therefore the start position and the last is the goal. Combined with noise-free, start-relative odometry in map axes and a known map scale, this gives the agent its exact map position at every step. Is this intended? It seems to conflict with the statement that absolute position is not an input. It may also explain why Trajectory beats Point and Language, why Combined is close to Trajectory, and why odometry noise has little effect. How do the results change if the first anchor is removed or the anchors are taken only from the interior of the path?

  3. The paper reports 924 test episodes from 126 scenes. However, aggregate_online_shards.py and aggregate_offline_shards.py default to 1,035 expected episodes per goal type, and load_spoc_samples draws a scene-balanced subset by seed. How were the 924 episodes selected from the 1,035? Was the selection fixed before any model was evaluated?

Comments on the paper

1 comment

Dear Reviewers,

Thank you for your efforts so far. As we are at the end of the discussion period, I would like to kindly encourage you to review the authors' rebuttals and engage in further discussions, particularly if the authors have addressed your questions or concerns.

Thank you for your time and contributions!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.