DMap: Efficient Dense Matching and Bundle Adjustment for Accurate 3D reconstruction

Deep 3D models provide remarkable global robustness, but direct regression acts as an information bottleneck that discards sub-pixel geometric constraints. Conversely, classical optimization achieves sub-pixel precision but fails to scale to dense correspondences and relies on heuristic, discrete track building. We present DMap, a framework that bridges frozen foundation representations with efficient dense optimization. First, DMap extracts dense multi-view correspondences from the latent features of a frozen multi-view backbone, amortizing per-view representation computation so that per-pair decoding reduces to lightweight coarse feature matching and local convolutional refinement. Second, we resolve the efficiency bottleneck of non-linear optimization by formulating dense bundle adjustment on a regular view graph over regular 2D pixel grids rather than irregular point graphs. By anchoring tracks to source rays, dense warps enter the optimization directly without keypoint quantization or track merging, and millions of depth variables are marginalized in parallel within milliseconds. DMap achieves state-of-the-art accuracy across unordered (ETH3D, Texture-Poor) and sequential (TUM RGB-D, ETH3D-SLAM) benchmarks, improving relative-pose AUC@1◦ to 88.6% on ETH3D while running 13–17× faster than existing dense optimization baselines.