acceptodds
Under review as a conference paper at ICLR 2027

3DR-Reg: Rethinking Color Point Cloud Registration as 3D Reconstruction

Abstract

Color point cloud registration still largely follows the feature-matching paradigm, in which point cloud features are matched before estimating a rigid transformation. Recent 3D reconstruction foundation models offer a different way to organize cross-view information by directly inferring structured scene geometry. We introduce 3DR-Reg, which reformulates color point cloud registration as 3D reconstruction. Rather than matching independently learned descriptors, 3DR-Reg reconstructs a shared 3D scene from the color information of two input point clouds using a pretrained 3D reconstruction foundation model. These predictions enable registration through camera poses, cross-view correspondences, or a shared reference frame. To improve correspondence-based registration, we further introduce a geometry-guided correspondence refinement module that resolves ambiguous correspondences through geometric consistency and global context. Experiments on 3DMatch and 3DLoMatch demonstrate state-of-the-art registration performance, with mean rotation/translation errors of /2.14 cm and /13.15 cm, respectively. Across multiple frozen 3D reconstruction models, reconstruction predictions support accurate registration without learning registration-specific representations. For the first time, we demonstrate that a 3D reconstruction foundation model can directly serve as the basis for color point cloud registration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Point cloud registrtation, 3D reconstrction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3GtzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new framework for color point cloud registration. Traditional registration methods face challenges when overlap is limited or different regions exhibit similar geometry and color. To address this challenge, the paper formulates color point cloud registration through the predictions of a pretrained 3D reconstruction model, and leverage geometric consistency and camera-pose cues to resolve ambiguous correspondences. Experiments on 3DMatch and 3DLoMatch demonstrate strong registration performance.

Critical strengths
  1. The paper explores an interesting perspective that directly exploiting the structured geometric predictions of pretrained 3D reconstruction models for point cloud registration.
  2. The proposed framework supports multiple registration routes and demonstrates strong performance.
  3. The experiment is relatively comprehensive, including comparisons across multiple reconstruction backbones, with component-level ablations.
Critical weaknesses
  1. The geometry-guided refinement module could be better clarified, since geometric consistency and correspondence reweighting have been broadly explored in prior registration methods.
  2. Although the proposed framework requires relatively few trainable parameters, it relies on large pretrained reconstruction models.
  3. The current pipeline assumes pixel-organized RGB-D observations and correspondence between image pixels and 3D points, the applicability of the method to general color point clouds could be discussed more clearly.
Questions for authors

Please refer to the weakness.

Reviewer_xFN8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

3DR-Reg turns color point-cloud registration into a two-view 3D reconstruction problem. It arranges the color of two RGB-D point clouds in their original pixel layout and feeds them to a frozen reconstruction model. Camera poses, point maps, and 3D tracks are then converted into a rigid transform by relative pose, mutual nearest neighbors, or alignment in a shared reference frame. A lightweight Geometry-Guided Correspondence Refinement module reranks candidate correspondences with rigid consistency. On 3DMatch and 3DLoMatch, the refined model reports mean errors of 0.69°/2.14cm and 3.77°/13.15cm. The same routes are tested on multiple frozen reconstruction models.

Critical strengths
  1. Using a frozen 3D reconstruction model to register an RGB-D pair is a sensible idea, and the results show that it works. In Table 4, training-free routes already reach about 99.7–100% registration recall on 3DMatch. On 3DLoMatch, Pi3 camera poses reach 86.6% recall and VGGT alignment in the shared frame reaches 91.3%.
  2. The paper is clearly written. Figure 2 and Section 3 cleanly separate the three registration routes from the refinement module. Table 4 evaluates nine frozen models, including DUSt3R, MASt3R, VGGT, and Pi3, so the result is not tied to one backbone.
Critical weaknesses
  1. The reported performance mainly comes from reading geometry that frozen reconstruction models already output, and the paper's own module adds little. The camera route takes the predicted relative pose and rescales its translation with the input depth. Point-map matching and shared-frame alignment likewise transfer the model's point maps back to the input clouds. These training-free routes already account for most of the accuracy.
Questions for authors
  1. The pipeline requires color in its original pixel layout. How does registration degrade when that layout is removed, for example on fused point clouds that no longer correspond to one camera image?
Reviewer_RhiCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes 3DR-Reg, which reformulates RGB-D point cloud registration using pretrained 3D reconstruction models. The framework leverages predicted camera poses, point maps, and 3D tracks for registration, and introduces a geometry-guided correspondence refinement module. Experiments on 3DMatch and 3DLoMatch demonstrate promising registration performance across multiple reconstruction backbones.

Critical strengths
  • The paper explores a promising alternative to conventional feature-based registration by directly leveraging geometric predictions from pretrained 3D foundation models.

  • The method achieves strong registration performance on both 3DMatch and 3DLoMatch, particularly under low-overlap conditions. Its applicability across multiple frozen reconstruction models is also encouraging.

Critical weaknesses
  1. Limited methodological novelty. Overall, this work is primarily engineering-oriented, with limited methodological novelty. The proposed registration routes largely convert existing reconstruction outputs into rigid transformations using standard geometric operations, including relative camera-pose composition, nearest-neighbor matching, and SVD-based alignment. Furthermore, the correspondence refinement relies on established geometric consistency principles, closely related to PointDSC and MAC. While integrating these components with pretrained 3D reconstruction models is practically meaningful, the main contribution appears to be a combination of existing techniques rather than a fundamentally new registration methodology.
  • Insufficient experimental evidence to support the proposed refinement and overall practical advantages. In Table 5, removing geometric consistency reduces registration recall on 3DLoMatch only marginally (91.0% to 90.7%), raising questions about its independent contribution. Finally, the framework relies on large reconstruction models (0.4–1.3B parameters), yet no inference-time or memory comparisons are provided to establish its practical efficiency relative to existing registration methods.
Questions for authors
  • Could the authors clarify the key methodological novelty beyond directly converting pretrained reconstruction outputs into registration estimates and applying established geometric consistency refinement?

  • Given the limited performance difference after removing geometric consistency in Table 5, what is its independent contribution to registration accuracy?

  • Can the authors provide comparisons of inference time and peak GPU memory consumption against existing registration methods under identical hardware and evaluation settings, including the computational overhead of the pretrained reconstruction backbone?

Comments on the paper

1 comment
Area_Chair_vnT2OpenReview ↗

Looking for reviewers' reply !!!

Dear Reviewers,

The authors have posted their rebuttal. Please read it and reply in the discussion, covering:

  1. Whether each of your questions and concerns has been addressed.
  2. Whether your rating and recommendation change, and why.
  3. Any remaining concern that must be settled before the meta-review.

Best.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.