- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
In this manuscript, the authors propose a destination-agnostic Graph Neural Network (GNN) to solve the potential-field navigation problem without target conditioning. In particular, their model maps each cell of an occupancy grid to an embedding z_i, aiming to approximate the shortest-path distance for any possible goal. The paper’s main claim is that evaluating and training such models relying only on MSE (regression accuracy) is not a suitable proxy for greedy-descent navigability. The authors further support their claims in the experimental section with a map-level study and argue that the regression objective itself does not enforce the local monotonicity that greedy descent requires.
Critical strengths
- The experimental setup and training details are carefully reported throughout the paper, and the attached supplementary material is easy to follow.
- Overall, the research question about single-forward-pass embedding generation to solve the proposed problem in a conditioning-free setup is reasonable.
Critical weaknesses
-
Clarity, style, and writing concerns: overall, the presentation is poor and below the venue’s standards. Manuscript sections are poorly written and below the expected level of clarity. From the beginning, the paper’s scope is not clearly framed, and the writing style seems to continuously overstate the work’s merit without getting straight to the point, leaving a sense of vagueness (e.g., lines 314-315 for limited experimental evidence). The paper’s narrative is oddly interrupted by experimental setup description, implementation or file formatting details, or inline math describing the model’s architecture; while reproducibility and clarity are a fundamental part of a manuscript, I think that these details belong in the appendix. Finally, the figures are not ready for publication and only drafted (Figure 1 has matplotlib's colorbar only attached on the rightmost panel, and Figure 2 subplots titles are barely readable). I strongly suggest that the authors thoroughly revise the manuscript’s style before publication.
-
Limited related work section. The bibliography is scarce and omits whole lines of research in the field, ranging from relevant research on neural planning on grids (e.g., Yonetani et al., Path Planning using Neural A* Search, ICML 2021) to an analysis of current GNN applications in this area (e.g., Veličković et al., Neural Execution of Graph Algorithms, ICLR 2020). Engaging with the relevant literature would strengthen the novelty and better position the paper.
-
Limited novelty and weak experimental setup. Although an original application of GNNs could potentially be of relevance and interest to the community, this manuscript only presents an incremental contribution, proposing a method that only “assembles” existing components in a naive way: GATv2 as the GNN backbone, P-GNN-inspired distance supervision, and a standard contrastive loss (pairwise ranking). Beyond the incremental formulation of the methodology, the main claim that “regression accuracy is not planning success” has already been established in the literature and recognized by the community (see, for example, Chrestien et al., Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goal, NeurIPS 2023, for a broad treatment of the problem). Here, the authors do not propose a novel theoretical or methodological perspective on this known problem, strongly limiting the novelty of the overall work. Finally, the experimental setup is narrow, testing the authors' method only in a synthetic scenario, without statistical analysis (30x30 maps, 50 test maps) and providing no external baselines to support the paper's claims.
Questions for authors
-
Novelty: How does this work contribute beyond existing literature?
-
Statistics: Can authors provide a significant statistical assessment of their methodology?
-
Baselines: After having addressed 1 and 2, can the authors provide and compare against multiple external baselines?
2 comments
I thank the authors for the detailed response and revised experiments, that addressed some of my concerns. However, the revision still presents the incremental contribution I highlighted in my initial review. I therefore cannot recommend the acceptance of the manuscript in its current form.
Response to Reviewer 6sDV
Thank you for the detailed review. We answer the three questions first and then the writing, figure and related-work points.
Novelty. We agree that the broad point that cost-to-goal regression is not what a planner needs is known: Chrestien et al. (NeurIPS 2023) argue this for learned heuristics in A* and greedy best-first search and propose ranking losses for them. We do not claim this point as new, and the revised introduction and Related Work say so. Our setting differs in two ways. (a) We learn a goal-agnostic embedding that is computed once per map, and the potential field for any goal is obtained afterwards as an embedding distance; greedy descent on this field then requires local monotonicity at every cell on the path, not a good expansion order. (b) We quantify, for this setting, how far regression error and descent success are coupled (Spearman -0.35 over 20 runs, with differences between loss weights and sampling densities falling inside seed noise) and localise where descent fails. We have added the Neural A* line of work (Yonetani et al., ICML 2021) and Chrestien et al. to Related Work, and Neural Execution of Graph Algorithms (Velickovic et al., ICLR 2020) as a neighbouring use of GNNs on graph algorithms; we state plainly that our architecture is an assembly of existing components and that the contribution is the empirical analysis, not the architecture.
Statistics. The submitted version reported one seed. We have since trained the main configuration with 11 seeds and report mean and standard deviation: completion 92.46 +/- 0.57%, MSE 826 +/- 63, ranking violation 5.97 +/- 0.12% (seed 42, as in the paper: 93.07%, 774, 5.77%). For the comparison configurations (rank_weight 1; 32x128 sampling) we trained seeds 0-4 and report seed-paired differences with t-statistics and map-level bootstrap intervals. The conclusion changes: differences between configurations in completion rate are within seed noise (for example -0.20 pp, map-level 95% interval [-0.82, +0.26], for rank_weight 1 vs 10), and we have removed the claim that completion peaks at an intermediate weight. Per-map completion distributions are reported as well (10 of 50 maps always solved; the worst 10 maps carry 66% of failures).
Baselines. The in-framework baseline is the goal-conditioned regressor (3 seeds); we report its best setting (mono_weight 3) and make no claim about a sweep. We have not added external planners in this revision. Neural A* and similar methods solve a different problem (they learn a guidance map for a search from a given start and goal), so they are not directly comparable to a precomputed goal-agnostic potential field evaluated by greedy descent. A fair comparison would use our learned field as the heuristic inside A* or GBFS, which we list as future work together with landmark-based non-learned embeddings. We have added this to the limitations section.
Clarity. We moved implementation and file-format details from Sections 4-5 into the appendix, shortened the overstated phrasing in the introduction and conclusion, and replaced the failure-map figure with one that has a single labelled colourbar and no baked-in titles. We also corrected the metric definitions in Section 5.3 so they match the evaluation code, and replaced phrases that referred to the submission history with plain statements of the current results.
Figure 1 of the submitted version (Figure 2 in the appendix of the revised paper): the train and test map panels. The train-map panel shows a colour field only because the same plotting routine renders both map files, and every map file stores a goal and a field. Training maps are used only through anchor-target pairs; the stored goal and field are never read, so the colour on the train map carries no information and has no scale to share. Only its obstacle layout matters. We have corrected the caption to say this, so the two panels are not read as sharing a meaningful colour scale; the colour and colourbar apply to the test map, whose goal and field are the evaluation target.