- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents IHF-ReLoc, an instance-aware framework for image-to-point-cloud camera relocalization. It establishes instance-level associations through scene graph matching, progressively refines point-to-patch correspondences, and uses a Candidate Graph Reasoner to incorporate local geometric consistency into candidate selection. Experiments on ScanNet and 3RScan evaluate localization accuracy and robustness to scene changes.
Critical strengths
-
The progression from instance-level association to fine-grained matching provides a coherent approach to reducing cross-modal ambiguity.
-
Retaining multiple correspondence candidates and using geometric consistency to select among them is a reasonable design for ambiguous regions.
Critical weaknesses
The novelty of the second contribution is limited. Scene graphs and object relationships have already been used for cross-modal association: GOReloc establishes correspondences between 2D and 3D object graphs for camera pose estimation [1], while SceneGraphLoc matches query images against 3D scene graphs for coarse localization [2]. Using scene graphs and instance topology to support matching is therefore not new. Moreover, filtering map points using positive-depth, viewing-frustum, and instance-mask constraints constitutes standard geometric and visibility checking, rather than a new occlusion model or visibility reasoning method. The combination presented here does not establish sufficient methodological novelty to justify a major contribution.
The experimental evaluation is insufficient. The ScanNet v2 comparison omits Top-I2P [3], a relevant method that also exploits topology for image-to-point-cloud registration and reports ScanNet results. More importantly, the 3RScan experiments compare only different training configurations of the proposed method, without external localization baselines. They therefore do not establish an advantage over existing methods in changing scenes. PnP success rate also cannot substitute for localization recall under specified pose-error thresholds.
References
[1] Yutong Wang, Chaoyang Jiang, and Xieyuanli Chen. GOReloc: Graph-based Object-Level Relocalization for Visual SLAM. IEEE Robotics and Automation Letters, 2024.
[2] Yang Miao et al. SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs. ECCV, 2024.
[3] Pei An et al. Top-I2P: Explore Open-Domain Image-to-Point Cloud Registration Using Topology Relationship. IJCAI, 2025.
Questions for authors
The appendix describes three training stages. Which stages use each loss in Section 3.5, and which network outputs do these losses supervise? Which modules are frozen or jointly trained at each stage? Please also specify the CGR architecture and its training procedure.
Table 1 reports a median translation error of 0.32 m and recall of 60.4% at 0.5 m / 15° for PlanaReLoc, compared with 0.20 m and 73.1% in its original paper. Given the different evaluation protocols, could the authors explain the differences in query selection, model training, map construction, and post-refinement, and confirm whether the complete PlanaReLoc pipeline was used?
Could the authors provide a comparison with Top-I2P on ScanNet under the same evaluation protocol?
Could the authors provide comparisons with external localization baselines on 3RScan under comparable input conditions, including localization recall at explicit translation and rotation thresholds?
A1: Our main contribution lies in coupling instance association, local verification, and pose-guided correspondence refinement. Scene-graph context retrieves candidate map instances, and instance features are injected into local matching to reduce ambiguity among similar objects. Local point-to-patch evidence then verifies the candidate instance pairs, providing instance associations for progressive refinement.
A2: Top-I2P is added to Related Work. Its source code was unavailable for our comparison; Hg-I2P provides a related graph-based baseline.
We have added external baseline comparisons in Section 4.2. All methods are evaluated on validation rescans using reference-scan maps. Baselines and ours* train on training reference scans and rescans; ours uses reference scans only.
Results and analysis are provided in Section 4.2 (Table 3).
A3: Training consists of segmentation pretraining, SGM training, and joint matching training. Appendix C.2 specifies the stages and which parameters are frozen or optimized. Appendix C.1 details CGR architecture and message passing, and Appendix C.4 defines the supervision and loss weights. CGR is trained jointly with matching, without separate pretraining.
A4: We implemented PlanaReLoc from the paper and use its basic pipeline without subsequent refinement, as stated in Table 1.
1. Query selection. We follow our ScanNet split and frame sampling, additionally removing frames with fewer than three visible map planes.
2. Training. Frozen MoGe-2 predicts query depth; sequential RANSAC extracts planes and visual features are pooled within each region. Two PointNet branches encode individual map-plane shape and global layout, followed by self-/cross-attention plane matching.
Stage one uses GT planes with normal/offset noise standard deviations of 5°/5 cm and query-plane dropout probability 0.1. Stage two uses planes recovered online by MoGe-2 and RANSAC. Predicted and GT regions are associated at IoU 0.3; annotated map-plane IDs provide labels for correct and unmatched planes. The visual-feature head, map encoders, and matcher are trained with AdamW (learning rate and weight decay 10⁻⁴), batch 2 accumulated to 16. The optimizer and scheduler are reset for stage two.
3. Map construction. Sequential RANSAC extracts planes within plane-supporting semantic instance regions, followed by merging compatible planes and removing fragments.
4. Pose estimation. Query plane extraction uses a 10 cm distance threshold, 0.9 normal dot product, and at most 16 planes, stopping when the current plane has fewer inliers than 1% of pixels. Matches are filtered by mutual nearest neighbors and confidence 0.1. Rotation RANSAC samples two nonparallel plane correspondences, uses a 10° normal-angle threshold, and re-estimates rotation from all inliers. No subsequent refinement is applied.