acceptodds
Withdrawn from ICLR 2027

NAV3R: BENCHMARKING NAVIGATION-CONSTRAINED RGB CAPTURE FOR 3D RECONSTRUCTION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied 3D reconstruction requires an agent to decide not only how to reconstruct a scene, but also where to move and which observations to retain. Yet exploration and reconstruction are typically evaluated separately, leaving unclear whether better exploration actually produces better inputs for reconstruction. We introduce NAV3R, a benchmark for navigation-constrained RGB capture and 3D reconstruction. Agents explore unfamiliar indoor scenes under movement and frame budgets, and their retained observations are evaluated jointly by exploration coverage, pose-free geometric reconstruction, and novel-view synthesis. Across 51 scenes from three indoor environments, we benchmark diverse navigation and view-selection policies with multiple RGB reconstruction models. Our results reveal a consistent trade-off: broader exploration improves whole-scene completion and generally benefits novel-view synthesis, but is associated with higher reconstruction error over the observed region. We further find that reconstruction quality depends strongly on acquisition structure, including view overlap, spatial spread, and frame selection. These results show that coverage and reconstruction quality are not interchangeable objectives, motivating joint evaluation of exploration, geometry, and rendering for embodied 3D reconstruction.

Primary area:
datasets and benchmarks
Keywords:
3D resonstruction, Site Navigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vTZgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a benchmark that evaluates embodied exploration by the 3D reconstruction its captured images support. Agents explore indoor scenes from Matterport3D, iGibson, and ProcTHOR on an offline waypoint graph under movement and frame budgets, and their retained RGB frames are evaluated by coverage, pose-free reconstruction with feed-forward models, and novel-view synthesis. Benchmarking prompted VLMs, VLN policies, and scripted baselines, the paper reports a trade-off in which broader exploration improves whole-scene completion but increases error over the observed region, and identifies view overlap, spatial spread, and frame selection as key factors for reconstruction quality.

Critical strengths
  1. Problem formulation. Evaluating exploration by the downstream reconstruction of its captured frames is a well-motivated and under-studied interface between embodied navigation and 3D reconstruction.

  2. Design. Decoupling capture from reconstruction allows the same capture set to be replayed through multiple reconstruction backends, enabling controlled comparisons across both policies and reconstructors.

  3. Analysis. The controlled studies of co-visibility, spatial spread, frame selection along fixed walks, and frame ordering provide useful evidence that acquisition structure, not just frame count, drives reconstruction quality.

Critical weaknesses
  1. Scenes. All scenes are indoor, so it is unclear whether the conclusions hold for outdoor or large-scale environments; evaluation on outdoor data such as Wanderland [1] is needed.

  2. Interface. The discrete waypoint graph is justified only by simulation speed, yet it handicaps policies trained in continuous environments, as the paper itself acknowledges that rankings reflect the action interface. A continuous-environment setting is needed for the benchmark to measure navigation ability rather than adaptation artifacts.

  3. Fairness. VLN policies are trained for efficient instruction following, so their low coverage is expected rather than a meaningful finding, and they also follow a different capture rule from the VLM planners. Comparing them on coverage against policies explicitly prompted to explore is not a fair comparison.

  4. Reconstruction. Only feed-forward models are evaluated, and key findings, such as error increasing with more frames along a walk and sensitivity to frame order, are tied to drift in streaming models. Conventional SfM with global bundle adjustment would likely behave differently, so the conclusions may not generalize beyond feed-forward reconstruction.

  5. Confound. Observed-region error is computed over a region that grows with exploration, so the central coverage–error trade-off is partly a consequence of the metric definition. Moreover, meter-level completion errors and very low PSNR suggest that many reconstructions fail outright, making policy comparisons in this regime hard to interpret.

  6. Insights. Sec. 4.2 reports phenomena without explaining them; for example, it does not analyze why many VLM planners cover less than a random walk beyond attributing it to stalling.

  7. Baselines. Although the paper identifies co-visibility and frame selection as key factors, it does not propose a baseline that exploits these findings or offer guidance on how to perform better on the benchmark.

  8. Positioning. The claim that exploration and reconstruction are typically evaluated separately overlooks active reconstruction work that evaluates both jointly, such as ANM [2], NARUTO [3], and ActiveSplat [4]. The paper should clarify how its pose-free, offline capture-set formulation differs from these.

[1] Liu, Xinhao, et al. "Wanderland: Geometrically grounded simulation for open-world embodied AI." CVPR. 2026.
[2] Yan, Zike, et al.. "Active neural mapping." CVPR. 2023.
[3] Feng, Ziyue, et al. "NARUTO: Neural active reconstruction from uncertain target observations." CVPR. 2024.
[4] Li, Yuetao, et al. "ActiveSplat: High-fidelity scene reconstruction through active Gaussian splatting." RA-L. 2025.

I actually lean towards a borderline rating of the paper at current state despite all these weaknesses. If all my concerns can be addressed during rebuttal, I'm happy to give a positive rating.

Questions for authors
  1. How exactly is a stall defined? In particular, does a ROTATE action, which changes the heading but neither the waypoint nor the retained frame set, count toward termination?

  2. What instruction is given to the VLN policies during exploration?

Reviewer_KEzdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

NAV3R evaluates the RGB observations collected by an embodied policy under navigation and retained-frame budgets. It combines 51 indoor scenes from Matterport3D, iGibson, and ProcTHOR-10K in an offline waypoint-and-heading graph, and assesses capture sets using waypoint coverage, geometric reconstruction, and held-out novel-view synthesis. The experiments compare navigation policies and, separately, unconstrained view selectors, with four RGB reconstruction backends. The principal empirical finding is that broader capture coverage is associated with better whole-scene completion and rendering, but often with worse error on the surfaces observed by each capture set. Further analyses examine co-visibility, spatial extent, frame selection, and input ordering.

Critical strengths

Strengths

  1. A useful acquisition-to-reconstruction interface. Separating capture generation from downstream reconstruction permits the same observations to be evaluated by multiple backends. This makes the benchmark potentially useful beyond the particular policies tested here.
  2. Complementary evaluation objectives. Reporting both observed-region and whole-scene completion, together with held-out rendering, exposes the limitations of evaluating only a small, well-reconstructed region. The paper already includes whole-scene results; this is an important strength .
  3. Substantive diagnostic experiments. The fixed-budget spatial-extent experiment, fixed-walk oracle selection, and fixed-set ordering perturbation address different acquisition factors rather than simply comparing model averages .
Critical weaknesses
  1. The central geometric trade-off needs a fixed-support diagnostic Observed-region completion is measured against surfaces visible in the retained frames. Consequently, changing the policy or extending a trajectory changes both the reconstruction and the set of reference surfaces. A larger error can reflect worse reconstruction of previously observed surfaces, newly included difficult surfaces, changes in alignment, or a combination of these effects. The manuscript correctly acknowledges this issue, and the whole-scene, accuracy, and depth results add useful evidence. Nevertheless, they do not directly establish that additional observations degrade reconstruction on an unchanged target region. The fixed-budget spatial-spread experiment also changes observed area, as the authors acknowledge.

  2. The action adapter and stall rule confound the policy-family comparison

JanusVLN emits 25 cm actions and reportedly needs 6–19 model calls to cross an edge; NaVILA also uses decomposed short actions and stalls on every evaluated trajectory. Yet the common stall rule checks whether consecutive calls change the waypoint or retained-frame set. Meaningful low-level motion or orientation changes can therefore fail to register as progress under this definition. Additionally, VLN policies capture automatically at newly reached waypoints, while VLM planners choose captures. Coverage AUC does not remove these control and capture-interface differences.

The authors recognize that the rankings depend on adaptation, which is welcome. However, the conclusion that VLN methods help only at waypoint level is not adequately separated from the evaluator’s preference for waypoint-level actions.

  1. Clarify what coverage and the resource budgets actually measure

Coverage counts waypoints within 2 m of a captured waypoint. It is therefore a capture-location proximity measure, not directly the fraction of visited space or visible scene surface. It is insensitive to camera heading; whether the distance is Euclidean, geodesic, or graph-based also matters around walls and across floors. Further, frame-indexed AUC does not measure the motion or observation effort needed to obtain those frames. Equal hop limits are well-defined graph budgets, but need not imply equal physical travel distance, rotation cost, or model-call budget.

  1. The evaluator must be versioned and sufficiently specified to interpret the results

The available main paper delegates crucial geometry-alignment and Gaussian-rendering details to Appendix A.5. I could not inspect that appendix, so I cannot determine whether these details are already fully specified. In particular, the evaluator should identify the similarity/rigid alignment procedure, its fitting data, point filtering and sampling, held-out camera transformations, and whether any pose or intrinsic optimization occurs during Gaussian fitting. It should also make the train/held-out image split, fitting schedule, and invalid-pixel treatment explicit. The fixed per-scene held-out set is a good design choice, but these additional details determine what the low absolute rendering scores and policy differences mean.

Checkpoint identity is also material. The current official VGGT-Ω repository documents an August 2026 retraining after a potential training-data contamination risk and recommends the reproduced checkpoint for benchmarking [R1]. This is not evidence that NAV3R used the earlier checkpoint or that any NAV3R scene was leaked. Please identify the exact checkpoint, preprocessing resolution, and evaluation commit. If an earlier checkpoint was used, explain the choice and consider a focused sensitivity check. The timing of the public notice relative to this submission has not been established here.

Uncertainty and aggregation. Repeated trajectories and scene-bootstrap intervals already exist. Please extend scene-paired uncertainty reporting to the main policy differences in Tables 2, 4, and 5. Clarify how five-run stochastic policies and one-run deterministic policies are weighted in within-scene correlations. Figure 3’s standard deviation across three datasets is not a substitute for uncertainty across scenes.

Questions for authors

see the above

Reviewer_o6exOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces NAV3R, a benchmark for navigation-constrained RGB capture and downstream 3D reconstruction across 51 indoor scenes from Matterport3D, iGibson, and ProcTHOR-10K. Agents explore under movement and frame budgets, and their retained images are evaluated through exploration coverage, pose-free geometric reconstruction, and novel-view synthesis. The benchmark compares scripted baselines, vision-language planners, navigation specialists, and active view selectors using four reconstruction backends. The results reveal a trade-off: broader exploration generally improves whole-scene completion and held-out rendering, but is associated with larger geometric errors over observed surfaces. Additional analyses examine how view overlap, spatial spread, frame selection, and input ordering affect reconstruction quality, motivating joint evaluation of exploration and downstream reconstruction.

Critical strengths

The paper’s main contribution is a benchmark connecting embodied image acquisition with downstream 3D reconstruction, rather than evaluating exploration and reconstruction separately. The evaluation is broad, covering 51 scenes across three datasets, 19 capture policies, and four reconstruction backends. This diversity, together with complementary metrics for exploration coverage, geometric reconstruction, and novel-view synthesis, provides a systematic assessment of how acquisition policies affect downstream outcomes. The benchmark could serve as a useful reference for researchers in navigation, active reconstruction, and potentially world modeling when comparing and selecting image acquisition policies.

Critical weaknesses

My key concern is that the empirical findings offer limited new insight beyond established observations in active mapping and view planning: broader visual coverage generally benefits scene completion and novel-view synthesis, while informative viewpoints and sufficient overlap improve reconstruction. The potentially more distinctive finding—that broader exploration is associated with worse observed-region geometry—remains difficult to interpret. Coverage measures proximity to capture locations rather than actual surface visibility, ignoring orientation and occlusion, while observed-region completion evaluates different reference surfaces for different capture sets. Consequently, the reported trade-off may partly reflect spatial dispersion and differences in evaluation difficulty rather than reconstruction degradation caused by broader exploration. Furthermore, the frame-order experiment diagnoses reconstruction-backend sensitivity through offline replay of a fixed image set; it does not establish the role of acquisition order in a closed-loop active policy, where intermediate observations influence subsequent decisions. Visibility-based coverage and controlled comparisons on fixed reference surfaces would strengthen the central claim and clarify what new understanding the benchmark provides. Also, the experimental analysis is largely descriptive: many paragraphs report numerical differences and then restate their direction as a conclusion, without sufficiently investigating the mechanisms behind them. Although the paper includes controlled analyses of acquisition properties, it provides limited diagnosis of policy-specific failures or explanations of where individual policies excel. For example, low coverage could arise from ineffective planning, action-interface mismatches, premature stalling, or poor capture decisions, each implying a different limitation. Representative trajectory analyses and targeted ablations separating these factors would make the benchmark more informative and actionable, helping readers understand when and why to choose a particular policy rather than primarily providing performance rankings.

Questions for authors
  1. Why do you define coverage as the fraction of waypoints within 2 m of a capture location, rather than using visibility-based surface coverage? The current metric ignores camera orientation and occlusion, so identical coverage scores can correspond to substantially different observed surfaces. What motivates this choice, and does the reported coverage–reconstruction trade-off persist when coverage is measured by the proportion of reference surfaces actually visible in the captured images?
  2. Section 4.5 is confused. On iGibson with CUT3R, Qwen2.5-VL-7B achieves lower observed-region completion error than the random walk (0.982 vs. 1.144 m), but also lower held-out PSNR (9.26 vs. 10.20). Because these metrics differ in both evaluation domain and target—observed-region 3D geometry versus held-out 2D appearance—this discrepancy alone does not establish a meaningful trade-off between geometric accuracy and rendering quality. 3.Could you clarify what constitutes a “weak explorer” in your evaluation (Line 483-485)? Capture-based Coverage AUC measures progress per captured frame, but does not directly assess efficiency in terms of travel distance, runtime, or computational cost. For navigation specialists, performance may also depend on their adaptation from goal-directed navigation to exploration, particularly under the shared action and stall interfaces. Would the reported rankings persist under distance-normalized coverage and controls for these interface adaptations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.