acceptodds
Under review as a conference paper at ICLR 2027

REVAR3R: GAUGE-AWARE PERTURBATION UNCERTAINTY FOR FEED-FORWARD 3D RECONSTRUCTION

Abstract

A correctly reconstructed distant point appears uncertain even when a frozen 3D model processes equivalent inputs because its output frame rotates fractionally. This exposes a weakness of training-free perturbation uncertainty: when outputs contain an unobserved symmetry, run-to-run variation potentially reflects symmetry rather than error. Existing alternatives have trade-offs: built-in confidence is outperformed in most evaluated conditions, while trained evidential heads require model-specific supervision. For point maps, this research derives a closed-form, error-independent variance term that grows with scene extent and potentially overwhelms the desired signal. Simulation reproduces the effect; all 30 real VGGT view-sets tested exhibit its predicted signature. ReVar3R robustly registers predictions to a common similarity frame before computing per-point variance, without retraining or modifying the frozen model. Optional calibration and fusion use a held-out split. Across VGGT, , and MASt3R on six datasets, the same estimator on every backbone lowers AUSE below built-in confidence in 15 of 18 conditions. The staged evaluation yields 11 of 18 wins for the label-free core, 12/18 for label-free equal-weight fusion, 14/18 with held-out weights, and 15/18 when the built-in signal is included. Against a trained evidential head, the result is a trade-off: the head calibrates magnitude better and leads in its training domain, whereas ReVar3R transfers across backbones without adaptation. Its ranking improves point filtering, but it does not detect stable systematic bias, aid novel-view synthesis, or transfer calibration across domains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D reconstruction, uncertainty estimation, feed-forward 3D models, perturbation-based uncertainty, gauge ambiguity, similarity alignment, uncertainty calibration, point maps

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_qvqiOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ReVar3R estimates uncertainty from repeated predictions under input perturbations, using robust similarity alignment before computing per-point variance. The paper derives a distance-dependent variance term induced by scale and rotation changes and validates it through simulations and real-data experiments. The label-free core outperforms built-in confidence in 11 of 18 model–dataset conditions; fusion with weights fitted on held-out labels, including the built-in signal, increases this to 15.

Critical strengths

1. Clear analysis of gauge contamination in perturbation variance. The paper distinguishes predictive variation from apparent changes caused by scale, rotation, and translation. Its theoretical decomposition and testable distance-dependent predictions provide a clear account of this confound.

2. Applicability across frozen reconstruction models. The core estimator operates only on output point maps, requiring neither network modification nor retraining. Experiments on VGGT, π3, and MASt3R support its applicability across architectures.

3. Extensive ablations and transparent reporting of limitations. The paper examines alignment scope, reference selection, perturbation families, and sample count. It also distinguishes label-free estimation from supervised fusion and reports failures involving systematic bias, cross-domain calibration, and novel-view synthesis.

Critical weaknesses

1. Limited evidence of practical relevance and downstream benefits

The paper could better explain how unreliable 3D points affect practical systems and which problems improved reliability estimation can address. Point filtering closely reflects the selective prediction behavior measured by AUSE, while the novel-view synthesis experiment shows no improvement. Evaluation on representative tasks such as multi-view fusion, surface reconstruction, or localization would strengthen the evidence of practical utility.

2. Substantial inference overhead and an unclear cost–benefit trade-off

ReVar3R reportedly incurs approximately ten times the inference cost of a single-pass uncertainty head, but whether its benefits justify this overhead remains unclear. The cost reporting also needs clarification, including nine forward passes in the main protocol versus approximately ten to eleven in the cost table, and latency reported per set versus per pair. Comparisons under matched workloads and fixed computational budgets would better establish the efficiency–performance trade-off.

3. Limited independent scene coverage

The distance-effect experiment includes 30 view-sets from only eight scenes, while the Trust3R comparison covers only one KITTI sequence and two TUM sequences. View-sets from the same scene are correlated, so additional view-sets do not substitute for broader independent scene coverage. More independent scenes and scene-level statistical analyses would strengthen the generalization claims.

4. Inconsistent cross-domain performance and unclear applicability

ReVar3R performs well on KITTI, but its AUSE on the expanded ETH3D evaluation is 0.667, compared with 0.585 for built-in confidence, and magnitude calibration transfers poorly across domains. The estimated contamination strength also shows little correlation with alignment gains, providing limited guidance on when the method is likely to help. The paper would benefit from clearer applicability criteria and guidance on choosing between ReVar3R and built-in confidence without target-domain labels.

5. Predictive stability does not necessarily indicate geometric correctness

Alignment reduces variance caused by coordinate-frame changes, but the remaining variance need not reliably reflect geometric error. Appendix D.6 shows that increasing the fraction of stably biased points from 0 to 0.6 reduces the Spearman correlation between variance and error from +0.39 to −0.67, indicating that consistently incorrect predictions can receive low uncertainty scores. Quantifying the prevalence and impact of such errors in real scenes would better establish the limits of the proposed reliability measure.

Questions for authors

All questions are detailed in the weaknesses above.

4 comments

In short. We agree with your main points and changed the paper: (1) per-view filtering improves only modestly and merging views does not, and the paper says so; (2) our cost numbers were inconsistent and are re-measured; (3) the main results now rest on 70 independent scene groups instead of 12 scenes; (4) the ETH3D loss came from an incomplete dataset copy, and on all 13 scenes Core and EQ are clearly better than built-in confidence for all three models; (5) stable errors remain a limitation, now measured on real scenes.

W1: downstream use.

  • Per-view filtering (benchmark; Appendix D.9, Table 29): removing each view-set's 50% most uncertain points, EQ keeps lower-error points than built-in confidence in 12 of 18 conditions and is never clearly worse, but closes only a median 13% of the gap to a perfect ordering. You are right that this is one point of the AUSE curve; the paper now says so.
  • Merging views into one point cloud (3 models, 7-Scenes and DTU; Table 30), each method removing its 25% least reliable points: no uncertainty signal, ours or built-in, beats keeping every point or deleting at random. Deletion improves accuracy (VGGT on 7-Scenes, EQ: 2.6 to 1.6 cm) but loses completeness (3.3 to 5.1 cm), and even deleting by the true error raises the F-score only by 0.005 to 0.033, because errors cluster. Weighted merging, surface reconstruction and localization remain untested, and the paper says so.

W2: cost. You are right that our reporting was inconsistent: the "about 10–11 passes" counted the shared run more than once, and the "about 9.9×" divided a time per set of 5 images by Trust3R's 0.919 s per image pair (our new measurement: 0.17 s per pair). We withdraw both. Our method uses 9 model runs (1 + 4 + 4). New timings (Appendix D.8, Table 27; same GPU, the same ten 5-view sets, median of 3 synchronized repeats):

ModelBuilt-in (one run)Our methodRatio
VGGT0.90 s9.71 s10.8
π³1.14 s12.04 s10.6
MASt3R (512 px)1.51 s15.20 s10.1
MASt3R (224 px)0.54 s5.62 s10.4

About 70% of the time is the 8 extra runs; alignment takes 5 to 10%. At 224 px on the same sets Trust3R takes 0.66 s, so ours costs about 8.5 times more, the price of needing no trained head per model. At an equal number of runs, EQ is never worse than spending all runs on one perturbation type and clearly better in 3 to 5 of 6 conditions (reply to mWSG, Q5); a 5-run EQ costs about 6 times one run (estimated from stage timings).

W3 is answered in the next comment.

W3: independent scenes. We agree, and the problem was larger than stated: our main results came from only 12 independent scenes. We redid both experiments you name, and our main comparison, on 157 view-sets per model from 70 independent units (Appendix D.2; 64 units for the Trust3R comparison), each unit counting once.

1. Against built-in confidence (18 conditions): EQ is clearly better in 15, worse in 1 and unclear in 2; LW in 13 and never worse; Core alone in 9 and worse in 1, never on DTU or KITTI.

2. The distance test (you noted 30 view-sets from 8 scenes; Appendix B, Table 3 and Figure 4). For each view-set, the Spearman correlation ρ\rho between the frame-motion part of the variance, uunaligned(p)2−u(p)2u_{\text{unaligned}}(p)^2-u(p)^2, and ∥xp∥2\lVert x_p\rVert^2; the theory predicts ρ>0\rho>0.

ModelImage sets with ρ>0\rho>0Median ρ\rhoDatasets where the interval is above 0
VGGT146/157+0.616/6
π³153/157+0.766/6
MASt3R146/157+0.456/6

The effect holds on average in every dataset for all three models (the paper had tested VGGT only). Our "positive in 30/30 view-sets" does not hold: 26 of 471 view-sets show no positive trend, most often DTU objects (12 of 26).

3. Against Trust3R (you noted 1 KITTI drive and 2 TUM sequences; Appendix D.6, Table 23), both at 224 px on identical images and points, after checking that both pipelines reproduce their stored AUSE from our submission exactly on a fixed ETH3D listing. EQ's AUSE minus Trust3R's, averaged over units:

DatasetUnitsDifference [95% interval]Units where ours is better
DTU30−0.233 [−0.274, −0.193]30
KITTI9−0.089 [−0.116, −0.062]9
7-Scenes7−0.031 [−0.099, +0.020]5
ETH3D13−0.002 [−0.064, +0.046]4
TUM5+0.085 [+0.038, +0.134]0

Counting each unit once, ours is clearly better on DTU and KITTI, Trust3R on TUM, and neither on 7-Scenes or ETH3D. Pooled over all points, Trust3R leads on 7-Scenes, TUM and ETH3D (ETH3D: 0.498 vs 0.654), because pooled AUSE also rewards putting whole high-error scenes first; we report both. At 224 px, EQ is also clearly better than MASt3R's built-in confidence on all five datasets.

W4: uneven results across datasets, and when to use the method without labels.

ETH3D. Our paper's 0.667 vs 0.585 for built-in came from 9 scenes, because our copy of ETH3D was incomplete. On all 13 scenes, Core beats built-in confidence for all three models:

ModelCore minus built-in, per-unit AUSEScenes where ours is betterp
VGGT−0.16710 of 130.012
π³−0.13211 of 130.008
MASt3R−0.08410 of 130.036

EQ is clearly better on ETH3D for all three models too, including the paper's own setting (MASt3R at 224 px: lower than built-in by 0.137 [0.035, 0.246] per unit, 10 of 13 scenes; pooled 0.654 against 0.801).

Calibration does not carry over. Median ECEq\mathrm{ECE}_q for EQ is 0.002 with calibration data from the same images, 0.118 from an unseen scene and 0.382 from another dataset (all values in our reply to frTA, W2). Given the same labelled data, Trust3R's calibration transfers better (on the 64-unit, 224 px Trust3R comparison: 0.094 against 0.103 for EQ on unseen scenes; Table 25), and it needs no labels for an error size. Sorting points, our main use, needs no labelled fusion data.

Deciding without labels. We fixed three label-free signals in advance (Appendix D.5, Table 22) and rank-correlated each with the gain over built-in confidence across the 18 conditions (values for Core, EQ and LW; negative means a larger signal goes with a larger gain): share of the spread that alignment removes −0.05, +0.21, +0.16 (unrelated to the gain, as you observed); agreement of the two perturbation types −0.38, −0.43, −0.48; agreement with built-in confidence −0.29, −0.04, −0.22. None is reliable enough to decide on its own, and the paper no longer says alignment helps most where the frame moves. EQ is our fixed default, but no label-free rule tells when it will beat built-in confidence.

W5: being stable is not the same as being right. We agree. With WW the 10% of points with the largest error, the share of WW in a score's "confident" half (0.5 is chance) is share=∣{p∈W:u(p)≤median⁡(u)}∣∣W∣.\text{share}=\frac{\lvert\{p\in W: u(p)\le \operatorname{median}(u)\}\rvert}{\lvert W\rvert}. On the benchmark (Appendix D.10, Table 31) the median share is 0.13 for EQ against 0.25 for built-in confidence, and EQ is lower in 17 of 18 conditions, but no score finds every stable error. In the 12 conditions with stored 3D points, a median 98.5% of points have an error more than 3 times their perturbation spread: repeated predictions often agree even when wrong. Our score sorts points; it is not an error bar, and the paper says so.

Presentation. The configurations are named Core, EQ and LW; Table 6 separates what needs labelled data from what does not; exact definitions are in Appendix D.1, with a pointer from Section 5.

Thank you for the substantial additional experiments and clarifications. The expanded scene coverage, corrected cost analysis, and transparent discussion of limitations strengthen the paper. While concerns about practical benefits and generalization remain, the response meaningfully addresses my main concerns, and I am raising my rating.

Reviewer_frTAOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a confound in perturbation-based uncertainty estimation for feed-forward 3D reconstruction: repeated predictions may differ because their coordinate frames change, even when their underlying geometry is consistent. Under a small-perturbation similarity-gauge model, the paper derives an additive variance term whose rotation and scale components grow with squared distance from the origin. ReVar3R addresses this issue by robustly aligning perturbed point maps before computing per-point variance. The label-free core can optionally be extended with held-out rank fusion and magnitude calibration. Experiments cover VGGT, π3, and MASt3R across six datasets. The core improves AUSE over built-in confidence in 11/18 conditions, while the full fused estimator improves it in 15/18. Additional experiments examine alignment choices, sampling baselines, a trained evidential head, point filtering, and failure cases.

Critical strengths
  1. The paper identifies a concrete confound and connects its mathematical explanation to targeted experiments. The decomposition explains why scale and rotation variation introduce distance-dependent variance unrelated to the assumed predictive noise. Simulation reproduces this behavior, while real-data analyses provide supporting evidence for the predicted spatial signature. The translation, rigid, and similarity alignment ablations further connect the proposed correction to the mechanism. This constitutes a useful methodological contribution for uncertainty estimation from frozen geometric predictors.
  2. The evaluation separates several sources of improvement and reports important limitations. The paper distinguishes the label-free core, equal-weight fusion, held-out weighting, and inclusion of built-in confidence. It also routes an input-noise ensemble through the same alignment and scoring procedure, demonstrating that alignment contributes beyond the particular perturbation recipe. The comparison with Trust3R includes calibration and computational cost, while negative results on stable bias, ETH3D, calibration transfer, and novel-view synthesis appropriately qualify the method’s usefulness.
Critical weaknesses
  1. The default estimator addresses a broader set of transformations than the global gauge analyzed by the proposition, which limits its interpretation for multi-view reconstruction. Independent per-view Sim(3) alignment removes view-dependent changes that can represent genuine cross-view inconsistency rather than a single arbitrary scene coordinate frame. The evaluation likewise aligns each view independently to ground truth, excluding these discrepancies from the target error. The authors explicitly acknowledge this distinction, so the reported results are meaningful for the chosen per-view target. However, they do not establish equivalent reliability for a jointly reconstructed scene. Table 7 illustrates the importance of this choice: scene-level alignment beats built-in confidence in 7/18 conditions, compared with 11/18 for per-view alignment. Evaluating both the estimator and target under one shared scene-level similarity would better establish applicability to multi-view fusion and globally consistent geometry.
  2. The held-out supervision supporting the full estimator is not specified sufficiently to assess reproducibility and generalization. The manuscript repeatedly refers to a small held-out labelled split, but does not clearly provide its size, scene or sequence composition, partitioning unit, or whether fusion weights and calibration maps are fitted separately for each backbone–dataset condition. These details matter because the full estimator’s 15/18 result depends partly on labelled fusion, whereas the label-free core achieves 11/18. Some evaluation datasets contain few independent scenes and overlapping view-sets, making scene-level separation particularly important. The exact AUSE normalization and regression-ECE construction also require more explicit definitions. This uncertainty limits interpretation of the supervised extensions without invalidating the label-free alignment result.
Questions for authors
  1. How do the estimator variants compare when both run-to-run registration and ground-truth error evaluation use one shared scene-level Sim(3), preserving relative inconsistencies between views? Does the ranking benefit remain useful under that target?
  2. Please specify the calibration and fusion split sizes, scene/sequence identities, partitioning unit, and separation from evaluation data. Are weights and calibration maps domain-specific, and how much labelled data does each condition require?
  3. Please provide the exact AUSE normalization and regression-ECE definitions. How sensitive are the principal conclusions to these choices, particularly where pooled and per-view-set results differ?
3 comments

W1 / Q1: what happens if all views are aligned together, for both our uncertainty and the measured error? We ran this test (specified before running; each scene group counts once). We align the repeated predictions either per view (our default) or with one transform for all views, and measure the error either per view (eAe_A) or with one fit for the whole scene (eBe_B, which keeps mismatches between views as error).

VGGT pilot on 7-Scenes (7 rooms) and DTU (30 objects). Mean difference in per-scene AUSE, first-named minus second-named (negative means the first sorts errors better; scenes where it is better):

7-ScenesDTU
Root-mean-square error, shared-scene relative to per-view1.004×1.018×
Core vs built-in, per-view error−0.130 (6/7)+0.028 (15/30)
Core vs built-in, shared-scene error−0.132 (6/7)+0.024 (17/30)
Scene-level vs per-view alignment of Core, shared-scene error+0.013 (2/7)−0.007 (22/30)

For VGGT the per-point change between the two errors (0.78 cm, 2.2 mm) is about the disagreement between the datasets' own ground-truth camera positions (0.64 cm, 2.6 mm), so this pilot is a weak test of cross-view inconsistency.

All 3 models on 6 datasets (18 conditions), shared-scene error:

  • Aligning all views together vs per view: for Core never clearly better and worse in 7; for EQ better in 1 and worse in 4; for LW better in none and worse in 4. The largest loss is Core on MASt3R/TUM, where the shared-scene error rose most among the TUM and KITTI conditions (worse in all 5 TUM groups).
  • The ranking benefit survives: against built-in confidence, EQ is clearly better in 14 of 18 and worse in 1, Core in 7 and 2, LW in 10 and 0 (per-view error: 15 and 1, 9 and 1, 13 and 0).
  • One global transform per run supplies 68% of the alignment gain under the per-view error (mean AUSE 0.473, 0.383 with one transform, 0.340 per view) and 83% under the shared-scene error (12 conditions where all three settings are scored). The submitted 65% came from the small ablation set-up.

The revised paper says the method measures per-view reliability, not that of a jointly reconstructed scene, and reports both targets and both scopes (Appendix D.3, Table 19).

W2 / Q2: how the labelled data was used, and whether it was separated from the test data.

The submission (Appendix C): learned weights wk∝1/AUSEk2w_k\propto1/\mathrm{AUSE}_k^2 were fit on alternating halves of the view-sets of each model and dataset (about five view-sets per half), and calibration on 20,000 points from a random half of the points. The scenes were 7-Scenes chess, DTU scan100, one KITTI drive, two TUM sequences, four ETH3D scenes and three BlendedMVS scenes. Neither split kept scenes apart: in 7-Scenes, DTU and KITTI both halves came from one scene, and in DTU they shared 15 of 18 source images. So our 14/18 and 15/18 described seen scenes. The paper now calls this "cross-fitted fusion" and labels it as the small ablation set-up.

New tests on the benchmark (70 units; Appendix D.4, Tables 20 and 21): weights learned only on the other units of the same dataset (LW), on just 1, 2 or 4 units, or on another dataset, against equal weights (EQ).

  • Learned weights help little: LW is clearly better than EQ in only 2 of 18 conditions and clearly worse in 6; weights from another dataset are better in 7 of 90 cases and worse in 38; 1, 2 or 4 labelled units do almost as well as all of them.
  • Against built-in confidence: EQ is clearly better in 15, worse in 1 (π³ on BlendedMVS) and unclear in 2; LW is clearly better in 13 and never worse.
  • The overlapping split made little difference: on the new units it gives practically the same AUSE as the strict one (differences of at most 0.006).

So the default is EQ, which fits no weights to labelled data, with LW reported beside it. EQ was not evaluated in the submission: we chose it after a VGGT pilot on 7-Scenes and DTU, in which equal weights were clearly better than built-in confidence on both datasets and learned weights only on 7-Scenes. Without those two conditions, EQ is clearly better in 13 of the remaining 16, worse in 1 and unclear in 2 (Appendix D.2); datasets are shared, so this is a sensitivity check, not an independent confirmation.

Calibrating error sizes works well only with labels from the same scenes. Median ECEq\mathrm{ECE}_q for EQ, by where the calibration data come from: 0.002 from the same images, 0.031 from other view-sets of the same scenes, 0.118 from an unseen scene, 0.382 from another dataset. LW and built-in confidence behave the same (Section 7; Table 21).

Q3: exact definitions, and a mistake we found in our calibration measure.

The definitions are now in Appendix D.1 (Eqs. 9 and 10), with a pointer from Section 5.

AUSE. Remove the most uncertain 0%, 1%, …, 99% of points and record the remaining root-mean-square error (curve SS); doing the same in order of the true error gives the best curve OO: AUSE=1S(0)∫00.99∣S(f)−O(f)∣ df.\mathrm{AUSE}=\frac{1}{S(0)}\int_0^{0.99}\lvert S(f)-O(f)\rvert\,df . We divide by the error of the group scored: a whole condition for "pooled", one independent unit for "per-unit" (our primary statistic). The phrase "after per-scene normalization" was inaccurate and is removed.

Our ECE was wrong. For a predicted size σi\sigma_i (an estimate of the average error) it counted errors below Φ−1(1+p2)σi\Phi^{-1}(\tfrac{1+p}{2})\sigma_i, treating a 3D distance as a one-dimensional bell curve whose width is the average error. For Gaussian 3D errors ∥e∥\lVert e\rVert follows a χ3\chi_3 distribution, so a perfectly calibrated method scores 0.150 instead of 0, and a less accurate method can score better. The correction: ECEq=111∑p∣1n∑i1 ⁣[∥ei∥≤q^p(ui)]−p∣,\mathrm{ECE}_q=\frac1{11}\sum_p\left\lvert \frac1n\sum_i \mathbf 1\!\left[\lVert e_i\rVert\le \hat q_p(u_i)\right]-p\right\rvert , where q^p(u)\hat q_p(u) is the pp-quantile of the error given score uu, learned on calibration data. It makes no bell-curve assumption; a correctly calibrated method scores about 0.003.

Against Trust3R (64 units, both at 224 px, the same calibration data, each scored against the error of its own prediction; medians, lower is better; Appendix D.6, Table 25):

Calibration map fitted onEQTrust3RClearly lower
Same view-sets, other points0.0040.010EQ (overall; DTU, KITTI)
Other scenes of the same dataset0.1030.094Trust3R (overall; DTU, ETH3D)
Another dataset0.4480.407Trust3R (overall; 4 of 5 datasets); EQ on DTU

Within the same view-sets our calibration is better; on unseen scenes Trust3R's transfers better, and it degrades less across datasets, where both largely fail. Trust3R also outputs an error size with no calibration data; ours always needs labelled data. The revised paper states this trade-off, removes the figures based on the old measure, and shows its bias (Appendix D.1, Table 13).

Pooled vs per-set AUSE. Pooled AUSE also rewards putting the worst view-sets first: on the small set-up Core wins 11/18 pooled but 5/18 per set. The benchmark uses per-unit AUSE as primary and reports pooled AUSE separately.

Comments on the paper

7 comments

Summary of changes in the revised PDF Unchanged: the theory (Proposition 1), the alignment procedure, the perturbation families and the rank-fusion formulation.

1. Configurations. ReVar3R-Core: the aligned photometric spread. ReVar3R-EQ (the new default): the submitted rank fusion of Core, the aligned view-order spread and built-in confidence, with equal instead of learned weights. ReVar3R-LW: the same fusion with weights learned on labelled scenes other than the test scenes. EQ was not evaluated in the submission; we chose it after a VGGT pilot on 7-Scenes and DTU and fixed it before the other 16 benchmark conditions ran. Figure 1 now shows the three configurations.

2. Main evidence: a scene-diverse benchmark of 157 five-view sets per backbone, from 73 scenes in 70 independent units, with intervals over units and 5 seeds. EQ is clearly better than built-in confidence in 15 of 18 conditions (1 loss), and in 13 of the 16 conditions outside the pilot; LW in 13 (no loss). The earlier 8 to 10 overlapping view-sets per condition remain as labelled ablations.

3. Corrections.

  • Win counts: the submitted 11/18 (Core) and 15/18 (fusion) compared pooled AUSE on overlapping view-sets from few scenes. A win now needs a 95% interval over independent units that excludes 0: Core 9/18, EQ 15/18.
  • Distance signature: not "30/30"; positive in 445 of 471 view-sets, and on average in all 18 conditions.
  • Calibration: the interval-based ECE was biased for 3D error norms; it is replaced by quantile coverage (ECEq\mathrm{ECE}_q), and the figures built on it are removed. With the same labelled data, EQ is better within the same view-sets, but Trust3R's calibration transfers better to unseen scenes and degrades less across datasets, where both largely fail.
  • Per-view filtering: a modest gain (median 13% of the gap to a perfect ordering), not 30 to 57%.
  • Labelled splits: the submitted weights and calibration were fit within the same scenes; now labelled "cross-fitted", with new tests on unseen scenes.
  • Cost: 10.1 to 10.8 times one backbone pass, 8.5 times Trust3R.
  • ETH3D: all 13 scenes (an incomplete local copy had dropped 4); Core and EQ clearly beat built-in confidence for all three backbones.
  • Eq. (4) now matches the code (RMS spread); the AUSE normalization is stated exactly.
  • Old Table 8 (now 11) took the best of three noise strengths per cell; at the default strength, aligned noise beats built-in confidence in 8 of 9 conditions, not 9.
  • Removed or corrected: "alignment helps most where the frame moves", "robust alignment is at least as good as a plain fit on every backbone", "rotation is one of the two largest alignment steps", "the fit is insensitive to the Cauchy scale" (the scale cannot change which points are trimmed).
  • Calibration: the interval-based ECE was biased for 3D error norms; it is replaced by quantile coverage (ECEq\mathrm{ECE}_q), and the figures built on it are removed. With the same labelled data, EQ is better within the same view-sets, but Trust3R's calibration transfers better to unseen scenes and degrades less across datasets, where both largely fail.

4. New analyses: both error targets and registration scopes; weights and calibration on unseen scenes and datasets; Trust3R per scene, pooled and for calibration; per-view filtering; merged point clouds; the size of the gauge term; stable errors; label-free indicators; matched-run comparisons; seed stability.

5. Scope. The score ranks per-view reliability. It is not a calibrated error bound, does not establish reliability for a jointly reconstructed scene, and does not improve merged-cloud F-score.

Where to find what you cited: old Table 7 is now Table 10; old Table 8 is Table 11; old Appendix D.6 is Appendix D.10 (Table 31); ETH3D is in Section 6.4 and Table 16.

To all reviewers: what we checked, what we got wrong, and what we added

Thank you for careful and constructive reviews. Before new experiments we re-checked our pipeline (re-running it reproduces every number in the submitted result tables exactly) and found five problems:

  1. Few independent scenes. Our 18 headline comparisons rest on only 12 independent scenes (7-Scenes one room, DTU one object, KITTI one drive), and the three models share the same view-sets.
  2. The labelled split was not scene-separated, so the submitted 14/18 and 15/18 describe scenes whose labels the method had seen.
  3. Our ECE was wrong: it treated 3D error distances as a one-dimensional bell curve, so a perfectly calibrated method scores 0.150 instead of 0. We replace it with quantile coverage (next comment).
  4. Our cost numbers mixed units: "~9.9×" divided a time per set of 5 images by the competitor's time per image pair. We withdraw it; new timings are in the reply to qvqi.
  5. Some local dataset copies were incomplete, and our loader silently skipped view-sets with missing files (ETH3D: at most 9 of 13 scenes). New experiments use re-downloaded data.

New experiments (each specified before it ran, except two comparisons labelled as added later: the size of the frame-motion effect and a 5-run cost comparison on all 18 conditions; each independent scene group counts once): a benchmark of 157 view-sets per model from 70 independent units with 5 seeds; error measured after one alignment for the whole scene; weights and calibration on unseen scenes and datasets; timings and equal-run comparisons; per-view filtering and merged point clouds; stable errors and label-free indicators.

Names. Core: our aligned photometric spread. EQ (new default): the submitted rank fusion of Core, the aligned view-order spread and built-in confidence, with equal weights, so it needs no labelled fusion data. LW: the same fusion with weights learned on other scenes. We chose EQ after a VGGT pilot on 7-Scenes and DTU and fixed it before the other 16 benchmark conditions ran.

Main results:

  • Against built-in confidence, EQ is clearly better in 15 of 18 conditions, worse in 1 and unclear in 2, with the same classification on all 5 seeds; LW is clearly better in 13 and never worse.
  • Alignment is the largest single step (clearly better than no alignment in 17 of 18).
  • Against Trust3R, counting each scene once: EQ is better on DTU and KITTI, Trust3R on TUM, neither on 7-Scenes or ETH3D; Trust3R's calibration transfers better to unseen scenes.
  • Per-view filtering improves modestly; merging views into one cloud does not.
  • Our spread sorts points; it is not an error bar.

Terms and formulas used in our replies (formal definitions: Appendix D.1, Eqs. 8 to 10)

  • Error of a point: eA(p)=∥svRvxp+tv−yp∥e_A(p)=\lVert s_vR_vx_p+t_v-y_p\rVert, with a similarity (sv,Rv,tv)(s_v,R_v,t_v) fitted to ground truth ypy_p separately for each view vv. The shared-scene version eB(p)=∥sRxp+t−yp∥e_B(p)=\lVert sRx_p+t-y_p\rVert uses one fit for all views, so mismatches between views count as error.
  • AUSE (lower is better): with S(f)S(f) the RMS error left after removing the most uncertain fraction ff, and O(f)O(f) the same when removing by true error, AUSE=1S(0)∫00.99∣S(f)−O(f)∣ df.\mathrm{AUSE}=\frac{1}{S(0)}\int_0^{0.99}\lvert S(f)-O(f)\rvert\,df . "Per-unit" scores one independent unit; "pooled" scores all view-sets of a condition together.
  • ECEq\mathrm{ECE}_q (0 is perfect): with q^p(u)\hat q_p(u) the pp-quantile of the error given score uu, learned on labelled data, for p∈{0.05,…,0.95}p\in\{0.05,\dots,0.95\}, ECEq=111∑p∣1n∑i1[∥ei∥≤q^p(ui)]−p∣.\mathrm{ECE}_q=\frac1{11}\sum_p\Big\lvert\frac1n\sum_i\mathbf 1\big[\lVert e_i\rVert\le\hat q_p(u_i)\big]-p\Big\rvert .
  • Core (the submission's "label-free core"): over N=5N=5 photometric runs aligned per view to run 0, u(p)=(1N∑n∥T^n(xn(p))−xˉ(p)∥2)1/2.u(p)=\Big(\tfrac1N\textstyle\sum_{n}\lVert \hat T_n(x_n(p))-\bar x(p)\rVert^2\Big)^{1/2}.
  • 2F: equal-weight rank fusion of Core and the view-order spread.
  • EQ (new; not the submission's "label-free equal-weight fusion", which averaged the two perturbation types without built-in confidence): ufuse(p)=13∑krk(p)u_{\text{fuse}}(p)=\frac13\sum_k r_k(p), the ranks of pp within its view-set under Core, the view-order spread and built-in confidence (low confidence ranks as uncertain). No labels are used to fit it, although built-in confidence comes from a trained model.
  • LW (new): ufuse(p)=∑kwkrk(p)/∑kwku_{\text{fuse}}(p)=\sum_kw_kr_k(p)/\sum_kw_k with wk∝1/AUSEk2w_k\propto1/\mathrm{AUSE}_k^2 measured on labelled scenes other than the test scene.
  • Cross-fitted fusion: the submission's "full estimator", with weights fit on other view-sets of the same scenes (small ablation set-up only).
  • Unit: an independent scene group; 70 in the benchmark (7 rooms, 30 DTU objects, 13 ETH3D scenes, 6 BlendedMVS scenes, 5 TUM groups, 9 KITTI drives).
  • Interval: a 95% range for the mean difference over units, from 10,000 resamples of whole units; "clearly better" means it lies below 0. Paired sign-flip tests are reported only with at least 6 units.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.