- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a robust 3D reconstruction pipeline that relies on combining a frozen 3D foundation model (by default VGGT-Ω b) with lightweight dense matching and efficient bundle adjustment. The backbone supplies initial camera and depth estimates, while a hierarchical matching decoder with a looped transformer adapter produces multi-view features reused across image pairs. This shared computation reduces the marginal cost per pair to coarse descriptor matching, local coordinate decoding, and convolutional refinement.
The convolutional refiner, based on the RoMa v2 architecture, uses local image features to iteratively adjust the coarse correspondence map, producing more precise sub-pixel matches. The camera poses and per-pixel depths are jointly refined through dense bundle adjustment over regular pixel grids, regularized by confidence-weighted log-depth priors. This is done efficiently with GPU fused kernels that compute residuals and derivatives in one pass, reducing the normal equations via the Schur complement in parallel. The solver supports self-calibration of focal length, principal point, and radial distortion, with a robust Cauchy loss for reprojection errors.
Critical strengths
Main arguments for acceptance:
- DMap combines an elegant matching architecture with efficient dense bundle adjustment, achieving strong accuracy and speed through shared multi-view features and GPU-friendly optimization. -Flexible camera modeling, particularly effective rolling-shutter correction, adds practical value, while extensive ablations substantiate the contributions of its main components.
Paper's strengths in more detail:
-
The main idea is not revolutionary, but it is elegant and interesting. In particular, the hierarchical matching decoder with a looped transformer adapter producing multi-view features that can be reused across image pairs, and hence significantly reducing the computational cost, nicely illustrated in Figure 3 and Figure 4.
-
DMap obtains more accurate results and is faster than recent SOTA methods (AMB3R, BLASt3R, VidMap). The high precision and the high speed they obtain are due to the usage of a regular pixel grid and fixed view graph enabling aligned memory access and the construction of the optimization problem memory-bound with fused GPU kernels. This yields a dense BA with millions of observations at a low cost.
-
DMap supports self-calibration of focal length, principal point, and radial distortion, as well as rotation-based rolling-shutter correction. Explicitly accounting for rolling shutter is a valuable strength for visual SLAM, where camera motion during image acquisition can introduce geometric errors. Table 3 demonstrates this benefit on TUM RGB-D: rolling-shutter correction reduces average trajectory error from 2.3 to 1.1 cm in the calibrated setting and from 2.2 to 1.0 cm in the uncalibrated setting. Moreover, the uncalibrated rolling-shutter variant slightly outperforms its calibrated counterpart, showing that accurate reconstruction does not require supplied camera intrinsics.
-
The paper provides extensive and informative ablation studies including backbone choices (Table 5), multi-view context and the number of adapter iterations (Figure 6), and the separate contributions of matching and optimization (Table 8). The component-swap experiment, which evaluates DMap and BLASt3R matchers with both BA backends under a common initialization is also interesting showing that each component independently improves accuracy and that their combination performs best.
Critical weaknesses
I did not identify any critical weaknesses that would support rejection, my main concerns being:
-
The technical novelty is somewhat limited. The contribution lies primarily in the integration and efficient implementation of ideas with established precedents, including dense source-anchored BA, Schur elimination, depth priors, and hierarchical matching. Nevertheless, reusing foundation-model features for multi-view matching and combining them with an efficient dense BA solver constitute meaningful technical contributions.
-
The experimental presentation lacks clarity about which DMap configuration is used in each evaluation. Table 1 distinguishes fast and HR variants, Table 3 separates global- and rolling-shutter settings, and the reconstruction experiments distinguish direct BA from fusion, whereas other tables simply report “DMap (ours).” Please explicitly specifying the settings for each result and consistently including a shared reference configuration across applicable tables. This would make the comparisons easier to interpret and help isolate the contribution of each component.
-
The paper’s organization could be significantly improved. Depth fusion appears to contribute significantly to reconstruction quality, yet its description is confined to the appendix. I think part of section C should be in the main paper. Space could be recovered by moving Figures 3, 4 and/or 6 to the appendix, keeping only their main findings in the text. Also some tables can be made more compact.
Questions for authors
- It would be good to clarify how DMap chunking maintains consistency across long sequences and provide a scaling study of runtime and memory usage, particularly for global bundle adjustment as the number of views increases?
2 comments
Satisfied with the author's answers
I have read the authors’ responses to the reviewers’ concerns and find them satisfactory. I appreciate the ported clarifications, the new sections in the updated appendix and I see no reason to lower my initial clear accept score.
Reply to Reviewer hreS
We thank the reviewer for the positive assessment.
Novelty. We thank the reviewer for recognizing that our contributions are meaningful. What unifies them is the design insight: dense matching as a simple, efficient, and general interface between a feed-forward 3D model and classical optimization. Each component operates at the abstraction it is suited for: the foundation model provides global context and initialization, the decoder translates it into geometric evidence, and a standalone solver optimizes it efficiently, without the need for joint training. Much as a language model calls a calculator for arithmetic, a 3D foundation model can call an external optimizer through dense matching. We believe this paradigm enables future exploration of 3D foundation models beyond scaling the backbone alone.
Presentation: which configuration each table uses (revised PDF, Appendix K, Table 14). Most tables use the same settings. Input images are at VGGT-Ω's default size, about 512² pixels, including ETH3D fast. Only ETH3D HR adds a second refinement round on 2× higher-resolution input, because its images are high resolution. All datasets assume global shutter, except the TUM rows that are marked as rolling shutter. The revised PDF adds a table with the configuration of the main experiments (image resolution, high-resolution refinement, shutter model). We will remove the redundant fast/global shutter annotation for the default, to avoid confusion.
Paper organization. We thank the reviewer for the thoughtful suggestions. We will improve the organization and move the core of the depth fusion from the appendix into the main text in the final version.
Consistency over long sequences. For long sequences, drift can be reduced by proper loop closure. However, developing an effective image retrieval strategy to find loop candidates efficiently is out of scope for this work. For simplicity, as described in Section 3.3 (Edge selection), we sample 30 keyframe anchors at a uniform stride for long sequences, and they match each other. Every other frame also matches its 8 nearest keyframes. Together with the edges between temporal neighbors, the view graph is well connected and our global BA can keep the sequence consistent. The feed-forward model runs in windows of around 100 frames that share the 30 keyframes and overlap by 5 frames. Each window is aligned to the first by a Sim(3) on the shared frames, and then one global BA refines all frames.
Scaling of runtime and memory with the number of views (revised PDF, Appendix J, Figure 17). We measured our BA on subsets of the Eyeful Tower kitchen, from 500 to 6,042 views. Runtime and peak GPU memory grow about linearly with the number of views, to 27.3 s and 19.0 GiB at 6,042 views.