- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents UniReg, a feed-forward network for image-to-point-cloud (I2P) registration across different sensing setups, spanning outdoor LiDAR, indoor RGB-D fragments, and scene-level reconstructions.
The framework is built on three core designs:
-
Token-space conditioning: Employs foundation models (DINOv2 for images and PTv3 for point clouds) with a camera token and learnable scene tokens to process cross-modal interleaved attention.
-
Ray-centric multi-geometry prediction: An MLP predicts the initial 6-DoF pose from the camera token, while two DPT heads predict dense depth and a Plücker ray map in the point-cloud frame.
-
Confidence-aware fusion: Uses predicted dense ray and depth confidence maps to update the camera token via cross-attention, regressing the final refined pose.
UniReg is jointly trained on six sources using domain-indexed scene-token banks and coordinate scale factors. The authors report competitive or superior performance compared to per-dataset specialist models across standard benchmarks.
Critical strengths
I found two main strengths in this work:
First, the paper provides a clear and coherent framing of cross-sensor degradation in I2P registration. Instead of treating sensor differences as a domain shift, the authors categorize the underlying bottlenecks into three coupled failure modes: entangled representations, sensor-sensitive geometric pathways, and negative interference in naive joint training. The proposed framework maintains a clean, one-to-one mapping between these identified challenges and its architectural components, making the methodological narrative intuitive and well-structured.
Second, joint ray and depth supervision is conceptually appealing. Table 3 provides useful empirical evidence that the two supervision signals are complementary and using both yields the lowest reported translation and rotation errors across the evaluated datasets.
Critical weaknesses
Despite the clear framing, several major issues weaken the paper's core claims and empirical reliability:
First, the baseline numbers and evaluation protocols appear misaligned. While comparing this work with recent I2P paper (e.g., Table 5 of arXiv:2603.26262), the baseline entries in Table 2 (formatted as "A ± B", such as for 2D3D-MATR) is the same as the Mean and Median values reported in that work. The manuscript never defines what "±" signifies. If "±" connects the mean and the median, this is non-standard and misleading. It is also unclear whether UniReg’s own entries use the same convention, mean ± standard deviation across test pairs, or variability across training runs. Furthermore, the documented evaluation setup differs from prior work, including the 30% overlap threshold on 7-Scenes (compared to the standard 50% cutoff in baselines such as CA-I2P), making it unclear whether all entries were evaluated on identical test pairs.
Second, the paper omits a direct comparison with RayI2P (Li et al., 2026), one of the most directly related prior methods. RayI2P already uses ray prediction and ray-guided pose regression to avoid explicit 2D–3D matching. Section 2 mischaracterizes RayI2P as a correspondence method, and Table 1 omits it entirely despite reporting competitive results on the same benchmark datasets (e.g., 0.09 m RTE on KITTI). Without a benchmarked comparison under a common evaluation protocol, the technical progress attributable to UniReg remains unsubstantiated.
Third, cross-sensor generalization and feature disentanglement are overclaimed. The model relies on explicit domain metadata via separate indoor/outdoor token banks and manual coordinate scalings. This does not invalidate the single-checkpoint setting, but the scope of generalization should be stated more precisely. Moreover, while Table 4 confirms scene tokens improve accuracy, it does not prove that sensor-specific factors are truly disentangled from geometry.
Finally, the mechanism of confidence-aware fusion lacks isolated validation. The strong claim that confidence weighting prevents noisy depth from corrupting the pose is not theoretically guaranteed by formulation alone. Controlled ablations such as removing fusion entirely or using uniform weights without confidence are required to substantiate this benefit.
Questions for authors
Could the authors clarify the following points during the rebuttal?
Clarification of Table 2:
What do the values after "±" represent in Table 2? Please clarify whether all methods, including UniReg, follow the same statistical definition and aggregation procedure.
Consistency in Evaluation:
Are the baseline numbers in Table 1 and Table 2 re-evaluated under UniReg's exact test protocol (including 6-DoF pitch/roll perturbations on outdoor sets and the 30% overlap threshold on 7-Scenes), or were they compiled from prior papers? If compiled, could you report baselines evaluated on the exact same test pairs and perturbations?
Relation and Comparison to RayI2P:
Could you clarify the conceptual differences between your ray formulation and RayI2P? Additionally, please include RayI2P's reported benchmark numbers in Table 1 to provide a direct and fair comparison.
Shared vs. Domain-Specific Token Banks:
How does UniReg perform with a single shared scene-token bank, while keeping the current normalization and training settings unchanged? Separately, how sensitive is the model to the domain-dependent scale normalization? To support the broader transfer claim, could you evaluate a sensor configuration or data source excluded from training, or clarify that the current evidence concerns acquisition regimes represented in training?
Ablation of Confidence-Aware Fusion:
Could you compare the proposed fusion against the initial pose from Eq. 5 and against the same fusion architecture with both confidence maps fixed to one (with learned attention weights preserved and ray/depth supervision unchanged)? A comparison of predicted confidence against actual geometric error would also help assess the reliability claim.
I am open to revising my rating if the authors address the concerns raised above.
2 comments
Response to Reviewer oA39 (2/2)
W3 & Q4 — Unified model, token banks, and normalization
Why this remains one model. A single checkpoint and shared registration pathway handle every source: all encoders, eight attention layers, ray/depth heads, fusion, and pose heads are shared. There are no dataset-specific backbones or output heads. The indoor/outdoor factors are reversible preprocessing that place roughly 50 m outdoor and 10 m indoor coordinates/targets in comparable ranges; translations are converted back to meters. Relative to one shared bank, acquisition conditioning adds only 16 tokens (16,384 parameters).
No sensor metadata. A fixed 30 m point-cloud-span threshold selects the normalization and token bank. Thus, routing uses only observed input geometry, not sensor or dataset identity.
Separate sensitivity controls. We isolate token sharing, normalization, and generic token capacity while holding all other settings fixed:
| Token design | Normalization | KITTI RTE/RRE | 7-Scenes RTE/RRE | ScanNet RTE/RRE |
|---|---|---|---|---|
| No tokens | outdoor /5, indoor /1 | 0.23/1.34 | 0.123/2.962 | 0.103/2.288 |
| 16 shared register tokens | outdoor /5, indoor /1 | 0.13/0.74 | 0.084/2.105 | 0.058/1.402 |
| Two randomly routed banks | outdoor /5, indoor /1 | 0.14/0.77 | 0.086/2.142 | 0.060/1.437 |
| One shared bank | norm=1 for all | 0.17/0.86 | 0.085/2.118 | 0.059/1.421 |
| Conditioned banks | norm=1 for all | 0.15/0.79 | 0.073/1.861 | 0.048/1.192 |
| Conditioned banks | outdoor /5, indoor /1 | 0.10/0.59 | 0.071/1.816 | 0.045/1.116 |
No tokens vs. shared registers measures generic capacity; random vs. conditioned banks isolates acquisition-aligned routing with identical parameters and computation; norm=1 vs. outdoor /5, indoor /1 measures normalization sensitivity. Most importantly, one shared bank with norm=1 still registers every benchmark. Conditioning improves stability and accuracy but is not required for unification. All rows are reported in Table 4 of the revision, with the analysis in Sec. 4.6.
Scope of generalization. The original tables demonstrate one-checkpoint performance across regimes represented during joint training. We now additionally hold out complete CARLA configurations:
| Training configuration | Unseen test configuration | RTE (m) ↓ | RRE (°) ↓ |
|---|---|---|---|
| FoVs 60°, 90°, 120° | FoV 75° (interpolation) | 0.15 | 0.73 |
| FoVs 60°, 75°, 90° | FoV 120° (extrapolation) | 0.23 | 1.12 |
On the fully held-out CARLA Town05, the same checkpoint reaches 0.14 m RTE, 0.71° RRE, and 99.32% Acc. Both results are added to Appendix A.3 (Tables 8 and 9) of the revision.
Intrinsics and ray consistency. The network receives only the image and point cloud; is used during training to construct ray/depth targets but is not an inference input. Ray directions are unit-normalized, and each training ray field is generated consistently from one physical camera. The held-out-FoV results above show that registration remains accurate on unseen projective geometry without supplying at inference.
W4 & Q5 — Confidence-aware fusion
Table 3 establishes complementary ray/depth supervision but does not isolate fusion or confidence. We therefore retain identical dense heads and supervision and vary only fusion:
| Fusion | KITTI RTE/RRE | 7-Scenes RTE/RRE | ScanNet RTE/RRE |
|---|---|---|---|
| Initial pose, Eq. (5) | 0.18/1.02 | 0.104/2.583 | 0.078/1.802 |
| Unweighted fusion () | 0.13/0.76 | 0.084/2.137 | 0.058/1.394 |
| Learned-confidence fusion | 0.10/0.59 | 0.071/1.816 | 0.045/1.116 |
The first difference measures dense multi-geometry fusion; the second isolates learned confidence from the attention architecture. We have added this ablation as Table 5, with its analysis in Sec. 4.6 of the revision.
Thank you again for your detailed and constructive feedback. We hope the corrected evaluation, RayI2P comparison, and controlled experiments resolve your concerns and support a more favorable assessment.
Sincerely,
The Authors
Response to Reviewer oA39 (1/2)
Dear Reviewer oA39,
We sincerely thank you for recognizing UniReg's coherent framing and complementary ray/depth design. We also appreciate your precise identification of the evaluation and positioning issues. We address each concern below and indicate where the manuscript has been revised.
W1, Q1 & Q2 — Statistics and evaluation consistency
Meaning of "±". The manuscript omitted its definition and mixed two benchmark-specific conventions. Table 1 follows outdoor I2P work and reports mean ± standard deviation over test pairs. Table 2 follows the indoor convention and reports mean and median; using "±" there was inappropriate. In the revision, Sec. 4.3 now defines "±" as mean ± standard deviation, and Table 2 reports separate Mean and Median columns, as stated in its caption.
7-Scenes overlap. The stated 30% threshold is a typographical error. Our data-generation code uses the standard 50% threshold, official split, and the same image–point-cloud pairs as 2D3D-MATR and CA-I2P. No additional 30%-overlap pairs were evaluated. We have corrected this in Sec. 4.2, Appendix A.1.5, and the caption of Table 2.
Source of baseline numbers. On RGB-D Scenes V2 and 7-Scenes, Table 2 values are compiled from prior papers, which use the same indoor pairs, metrics, and Mean/Median aggregation.
Table 1 values are also quoted from the corresponding papers and were not rerun under our exact perturbations. Most compared methods estimate pose from scratch through correspondences and PnP-RANSAC; our perturbation is not an initialization supplied to their solver. UniReg further uses a broader 6-DoF range, including large pitch and roll (App. A.2.3), so it is not evaluated under an easier pose distribution. We now state in the outdoor results of Sec. 4.4 that these baseline values are reported from the original papers, and we interpret small numerical gaps as competitive rather than as a strict ranking.
W2 & Q3 — Relation and comparison to RayI2P
We agree that RayI2P is the closest ray-based prior. It predicts patch-wise ray bundles and performs ray-guided pose regression without explicit 2D–3D matching. We add a fair benchmark-level comparison using the same datasets, official splits, and metrics (all test pairs retained; Acc.: RTE < 2 m and RRE < 5°):
Our contribution is not the first use of rays, but ray-centric multi-geometry: joint pose, dense depth, and per-pixel cloud-frame Plücker-ray prediction. Rays provide sensor-robust projective geometry, depth supplies metric range, and confidence-aware fusion combines them within one model spanning LiDAR, RGB-D fragments, and complete scenes. In the revision, we discuss RayI2P in the Related Work (Sec. 2), add it to Table 1, and compare against it in the outdoor results of Sec. 4.4.