acceptodds
Under review as a conference paper at ICLR 2027

NavMind: From Open-Ended Navigation Requests to Grounded Personalized Routes

Abstract

Conventional route planners optimize distance or time but rarely jointly consider visit needs and preferred street-level experience. Planning with large language models (LLMs) alone seems strong in open-ended requests, but may hallucinate places, misjudge street environments, or violate road-network constraints. We present NavMind, an LLM-guided personalized routing framework that grounds visit intent (e.g., a photogenic restaurant serving good steak) and preferred route experience (e.g., lively, visually engaging streets) in separate evidence sources using three modules. (1) Hybrid Point-of-Interest Retrieval-Augmented Generation (Hybrid POI-RAG) encodes POI locations and review-derived semantics to identify POIs that match the visit intent. (2) The perceptual-cost module scores street qualities with a fine-tuned vision-language model, while LLM-as-a-Jury reconciles different LLM estimates of each quality’s importance to the requested experience. (3) The route-optimization module casts visit intent as waypoint constraints and preferred street-level experience as edge costs in a generalized traveling salesman problem, yielding a network-valid route. Evaluation on Beijing data spans diverse personas and requests. Qualitative cases illustrate how NavMind adapts waypoints and routes to traveler preferences, generating diverse routes across intentions. At scale, simulations indicate that this adaptation requires only modest detours, with median detour rates below 20 for every persona. In user assessment, the resulting routes score 1.78 points higher than shortest paths on a seven-point scale and are preferred nearly four times as often. Component-level results further support these system-level findings: Hybrid POI-RAG achieves over five times the Precision@5 of semantic-only or spatial-only baselines; street-view scores agree with human ratings across four qualities (Pearson’s > 0.90); and Jury profiles align more closely with human judgments than single LLM. These results highlight NavMind’s potential for meaningful personalized navigation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Geospatial AI, Street View, Urban Perception

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7cSPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes NavMind, an LLM-guided framework that translates open-ended navigation requests into network-valid, perception-aware personalized routes. It separates local visit intent (flexible waypoints such as “a quiet cafe”) from global experiential preferences (e.g., “lively streets”) and grounds each in separate evidence via three modules. (1) Hybrid POI-RAG pipelines spatial corridor filtering, dense semantic retrieval over POI metadata and reviews, and evidence-aware listwise LLM reranking to form candidate anchor groups. (2) A perceptual-cost module scores road edges on four dimensions (Beautiful, Safe, Lively, Wealthy) with a LoRA-fine-tuned Qwen2.5-VL-7B trained on Place Pulse 2.0, and converts textual preferences into dimension weights via an LLM-as-a-Jury (four proposer LLMs plus an external judge), yielding edge costs that combine physical length with weighted cubic perception penalties. (3) An exact route optimizer casts joint anchor selection and ordering as an open generalized TSP, solved by Dijkstra metric closure plus Held-Karp subset dynamic programming, returning a globally optimal, network-valid route over the retained candidate space.

Validation on Beijing data (OSM road graph, 5,829 Amap POIs, 184k reviews, 7,083 Baidu street-view panoramas, 12 curated personas) spans four levels: component evaluation (POI retrieval Precision@5 0.359 vs ≤0.069 for single-modality baselines; VLM–human Pearson r > 0.90 on a held-out Place Pulse split; jury MAE 0.950, lower than any single proposer); an end-to-end comparison against ITINERA and LLMAP on 108 tasks (highest intent completion 66.2%, lowest extra distance 486.4 m, style win rates 66.67%/70.37% under an anonymized GPT-5 style evaluator); 36,000 simulated routes (overall median detour < 20%); and a blinded within-subject user study (36 participants, 324 paired comparisons) in which NavMind routes scored 5.26 vs 3.48 on a 7-point scale and were preferred roughly 4:1 over shortest paths.

Critical strengths

The problem formulation (separating local visit intent from global experiential preference) is well motivated, and the paper is relatively candid about method limitations (e.g., the Appendix A.1 caveat that solver optimality covers only the fixed candidate space). However, in light of the novelty and reproducibility deficits described below, I do not identify substantive strengths that could support acceptance.

Critical weaknesses
  1. Insufficient methodological novelty: the pipeline follows a routine cascade and leaves the essential coupling of point- and line-level semantics unaddressed. The method proceeds through the standard sequence for semantic route planning — POI semantic retrieval and reranking (spatial filtering + dense retrieval + listwise LLM reranking), link-level semantic/perceptual edge-cost scoring, and point-and-line route optimization — each assembled from established components. Beyond being a routine cascade, the three modules decide point-level semantics (which POIs to anchor) and line-level semantics (which streets to traverse) independently and in sequence: candidate POIs are generated without regard to the perceptual quality of the streets leading to them, and edge costs never feed back into POI selection, so their mutual influence is neither jointly optimized nor analyzed. This coupling is precisely the essential difficulty of point–line semantic planning, and prior work exists on both sides — cascaded realizations (keyword-aware route planning; perception-scoring-then-routing approaches such as pleasant/walkability routing; LLM itinerary + spatial optimization systems) and end-to-end realizations (MLLM-based route generation and navigation, and newly emerging route-planning benchmarks). The submission neither resolves this essential problem nor sharply differentiates itself from either line of work, leaving the methodological contribution limited to another competent cascade instance.

  2. Low reproducibility and non-solid conclusions: no public code or benchmark, no comparable baselines, and insufficient ablations. The paper introduces a new problem setting (open-ended request → perception-personalized network-valid routing) but releases neither the code nor the benchmark needed to evaluate it; moreover, platform terms prohibit redistributing the source reviews and street-view imagery (Appendices A.1, C), so third parties cannot reconstruct the POI-retrieval or human evaluations. Compounding this, the empirical comparison covers only two systems (ITINERA, LLMAP) with no stated justification for excluding other comparable methods — cascaded perception-aware routing or end-to-end semantic route generation — and the paper nowhere argues such comparisons are infeasible. The two deficits co-occur: no public benchmark to reproduce against, and no external anchor to calibrate against; the reported conclusions can therefore be neither reproduced nor cross-validated. The ablation design further weakens the conclusions: the human study’s only control is the shortest path, yet once any subjective semantic dimension is introduced and some detour budget is allowed, beating a pure efficiency baseline under subjective evaluation is trivially achievable — this comparison cannot establish the contribution of the perceptual scoring and weighting design itself. Missing controls include fixed-weight perception-aware baselines, a single-LLM weight variant, and the cost-form ablation (cubic vs. linear penalty). Secondary experimental gaps add to this: the scorer is fine-tuned and validated only on Place Pulse, never on the Beijing street views it actually scores; the impact of imputing unobserved edges is unquantified; and the default corridor width (an undisclosed magic number) is unreported. As the paper’s central claims are empirical, these gaps are decisive for my recommendation.

Questions for authors
  1. Point-level semantics (POI anchoring) and line-level semantics (perceptual edge costs) are decided sequentially in independent modules. How do you assess their coupling — e.g., do POI candidates generated without street-quality awareness propagate errors into the final route? Is a joint or feedback formulation feasible?

  2. What fraction of edges have direct street-view coverage, and how sensitive are final routes to the imputation of unobserved edges (Appendix A.2)?

  3. Can you validate the fine-tuned VLM on Beijing street views (e.g., a few hundred human-scored Baidu panoramas)? Given the abstract’s claim that street-view scores “agree with human ratings,” in-domain agreement seems necessary to support it.

  4. The human study’s only control is the shortest path. Would NavMind remain preferred against fixed-weight perception-aware baselines, a single-LLM-weight variant, or a linear-penalty cost form? What justifies excluding comparable cascaded or end-to-end methods from the empirical comparison?

  5. (Clarification) What default corridor width δ is used when no spatial scope is specified (Appendix A.1), and how sensitive are retrieval and routes to it?

  6. How does NavMind position itself against the broader set of recent end-to-end systems and benchmarks for query-to-route / travel planning (e.g., MLLM-based route generation, LLM–solver systems, benchmarks such as Map2Route)? Please sharpen the novelty positioning.

  7. Will the evaluation suite (127 route-aware requests with human Top-5 annotations, 12 personas, evaluation scripts) be released as a public benchmark, and can the released code run end-to-end given the redistribution restrictions?

Reviewer_cTNoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies personalized route planning under open-ended natural-language requests, where users may specify both flexible destinations and subjective route experiences. Its main contribution is a unified formulation that represents visit intentions as waypoint constraints and experiential preferences as perception-aware edge costs, combining language understanding with deterministic route optimization. Evaluations suggest that this formulation improves intent fulfillment and experiential alignment while keeping additional travel distance moderate.

Critical strengths
  1. The paper addresses an important gap between open-ended user intent and executable route planning. Existing navigation systems are poorly equipped to handle requests that combine desired destinations with subjective route-level preferences. NavMind turns such open-ended requests into geographically feasible routes rather than merely generating plausible textual recommendations. This is consequential because it extends personalized navigation from selecting among fixed routing options to interpreting and operationalizing users’ natural-language intentions.

  2. The paper introduces an elegant decomposition of complex routing preferences. By separating destination-related intent from route-level experiential preferences, the proposed framework converts an otherwise ambiguous language understanding problem into a structured and tractable routing problem. This design retains the flexibility of natural-language interaction while ensuring that the resulting routes remain grounded and feasible.

Critical weaknesses
  1. The method supports only a restricted representation of experiential preferences. Although NavMind is designed for open-ended requests, it ultimately projects route-experience preferences onto four predefined perceptual dimensions and combines them through a weighted cost function. Preferences that fall outside these dimensions, involve nonlinear interactions, or depend on contextual conditions cannot be represented naturally without introducing additional annotated attributes and modifying the cost formulation.

  2. The strongest end-to-end baseline comparisons focus on LLM itinerary/route planners (LLMAP, ITINERA), but do not include recent spatially grounded RAG for along-route POIs or perception-aware retrieval pipelines that are close in spirit to parts of the system.

Questions for authors

1.How does NavMind handle experiential requests that cannot be adequately expressed by the four predefined dimensions of Beautiful, Safe, Lively, and Wealthy, including preferences for lower or target values rather than simply higher values? Please clarify whether such requests can currently be represented without introducing new annotated attributes or manually modifying the cost function

2.How sensitive are routing outcomes to the cubic penalty in the perceptual cost function? Have you tried different exponents or learned calibrations, and how do these affect detour and user preferences?

Reviewer_j1uVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

NavMind turns an open-ended navigation request into a valid route on a road network. An LLM parses the request; candidate stops are found by spatially filtered, review-based retrieval with LLM reranking. Street quality is scored from street-view images by a fine-tuned VLM on four Place Pulse dimensions, weighted per user by an LLM jury. A GTSP solver then picks the stops and the path. The evaluation, in Beijing, covers component tests, a comparison with ITINERA and LLMAP, 36,000 simulated routes and a user study against shortest-path routing.

Critical strengths
  • S1: Clean division of labour: the LLM only interprets the request, while grounded tools make the decisions, so routes are always network-valid.
  • S2: Layered and transparent evaluation, with the questionnaire, per-case results and prompts in the appendix.
Critical weaknesses
  • W1: Inconsistency over the paper: The user study numbers do not match the stated design. With 36 participants per case, means should be multiples of 1/36, yet all 42 per-case means in Table 8 are multiples of 1/9, and every count in Tables 2, 8 and 9 is a multiple of 4. This suggests about 81 independent responses rather than 324. The only baseline is the shortest path, whose POI selection is undefined, so POI choice and street experience are confounded. Results are also odd: the efficiency-driven persona (Zhao Lei) shows the largest persona-level gain (+2.74), including 2.67 vs 5.33 in case C7, where the NavMind route is almost the shortest path (1.3% detour).

  • W2: The perception pathway is not validated where it is used. The r > 0.90 is on a Place Pulse split, not on the Beijing imagery that drives routing, and edge coverage is not reported. The simulation reports only detour, not any perceptual gain over the shortest path; a low detour is trivially achieved by not deviating.

  • W3: The paper is essentially an assembly of existing components, in a crowded line of work, with no new method. Each module closely follows prior work: geo-RAG, Place Pulse scoring, perception-weighted routing (Quercia et al. 2014), LLM juries, and LLM + solver planners (ITINERA, LLMAP). The corridor-based POI search corresponds to in-route nearest neighbour queries (Yoo & Shekhar 2005), and the solver is essentially the classic trip planning / optimal sequenced route problem (Li et al. 2005; Sharifzadeh et al. 2008); none of this spatial-database literature is cited. No system-level ablation shows that the combination matters.

Reference

Li, F., Cheng, D., Hadjieleftheriou, M., Kollios, G., & Teng, S. H. (2005, August). On trip planning queries in spatial databases. In International symposium on spatial and temporal databases (pp. 273-290). Berlin, Heidelberg: Springer Berlin Heidelberg.

Sharifzadeh, M., Kolahdouzan, M., & Shahabi, C. (2008). The optimal sequenced route query. The VLDB journal, 17(4), 765-787.

Shekhar, S., & Yoo, J. S. (2003, November). Processing in-route nearest neighbor queries: a comparison of alternative approaches. In Proceedings of the 11th ACM international symposium on Advances in geographic information systems (pp. 9-16).

Questions for authors
  • Q1: How many independent responses per case does the user study actually contain? Please explain why all per-case means are multiples of 1/9 and all counts multiples of 4.

  • Q2: How does the shortest-path baseline select POIs in waypoint cases?

  • Q3: What is the route-level perceptual gain over the shortest path in the 36,000 simulated routes, and what fraction of routes differ from the shortest path at all?

  • Q4: Is there any validation of the perception scores on the Beijing imagery, and what fraction of edges has direct street-view coverage?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.