- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies whether confidence ranking is an appropriate way to allocate a fixed reconstruction budget. It distinguishes confidence-based filtering from subsequent budget allocation and introduces a matched-pool experiment: confidence-ranked and uniformly sampled candidate pools have identical sizes and pass through the same sampler, output budget, and initialization. The paper also derives a coverage ceiling, U(C)=2r_C/(1+r_C), where r_C measures the reference surface supported by a candidate pool. This bounds the F-score achievable by any coordinate-preserving subset under the specified evaluation conditions. Experiments show substantial support loss from narrow confidence-ranked pools across several reconstruction backbones and datasets. Mixed and confidence-proportional pools recover most of the observed loss. The paper also identifies conditions where the advantage diminishes or reverses.
Critical strengths
- The matched-pool design isolates the proposed failure mechanism convincingly. Holding candidate count, sampler, output budget, and initialization fixed makes the comparison substantially more informative than a conventional confidence-threshold sweep. On DTU, the narrow ranked pool achieves an F-score of 23.52, compared with 77.62 for the equal-sized random pool. Its coverage ceiling is only 24.23, demonstrating that most of the deficit is already imposed by pool membership. This provides direct evidence that changing the downstream coordinate-preserving sampler cannot recover the lost performance.
- The analysis includes meaningful controls and explicitly characterizes the limits of the finding. The paper examines multiple backbones, pool widths, retained fractions, and random seeds, with scene-level paired uncertainty estimates. It separately demonstrates that confidence filtering helps on raw predictions, reports the ranking reversal at wider BlendedMVS retention, and includes the low-reach VGGT-Omega counterexample. Mixed and confidence-proportional pools provide constructive checks. These results support a qualified conclusion about allocation under tight budgets rather than a blanket recommendation to discard confidence.
Critical weaknesses
- The practical impact beyond the controlled selection setting is less established. Appendix B notes that the released VGGT COLMAP exporter already combines confidence thresholding with uniform budgeted sampling, which largely follows the paper’s recommendation. The motivating examples involving per-view opacity selection or pose estimation also differ from the specific global confidence-ranked-pool protocol studied here. Consequently, it remains unclear which established reconstruction pipelines would materially benefit from changing their current allocation rule. The downstream registration experiment provides limited additional evidence: success is defined relative to a transform fitted to ALL, so it measures agreement with that reference rather than independent registration accuracy. Likewise, the coverage ceiling does not directly establish consequences for pipelines that subsequently move points, refine geometry, or optimize rendering primitives. The authors acknowledge these restrictions, but they limit the demonstrated application-level significance of the otherwise convincing selection result.
Questions for authors
- Which existing reconstruction pipelines use the specific budgeted confidence-ranking rule examined here? Can the observed failure be connected to a concrete pipeline configuration, particularly given that the released VGGT exporter already uses thresholding followed by uniform sampling?
- Can the registration comparison use a reference transform independent of the ALL point set, or report absolute geometric accuracy alongside agreement with ALL? This would help determine whether the observed agreement advantage translates into better registration.
- How sensitive are the ranked–random contrast and the retention crossover to the initial eligibility threshold and evaluation tolerance? This would help establish how readily the practical recommendations transfer beyond the selected settings.
2 comments
Follow-up (18 Nov): a released configuration
Thank you again for your review. This follows up our 15 Nov reply with a new result. WorldMirror's optional confidence cap, off by default, is a concrete released configuration of the kind you asked about. In a new test where held-out images never reach the network, uniform selection gives higher held-out PSNR than confidence-ranked selection on all ten ETH3D scenes, at the same Gaussian count. We would be glad to hear whether this answers your question.
The failure connects to one concrete released configuration, WorldMirror's optional global cap. The released VGGT exporter already thresholds and then draws uniformly, as we recommend. The closest released match to global confidence ranking is WorldMirror's cap, off by default. We now run it end to end, at shipped settings with voxel merging on and no optimization step. Our uniform selection renders held-out views better than ranked selection by +3.27 dB (95% range 2.89 to 3.66, 21 DTU scenes). With nothing fitted per scene, the splats score 11.73 dB (ranked) and 15.00 dB (uniform), so the finding is the paired difference. The splats come from the remaining views, though WorldMirror's camera pass sees the held-out images, and these are our development scenes. After merging, the ranked set keeps 1.24M Gaussians on average against 2.35M for uniform. At equal count, uniform still leads by +1.25 dB (0.87 to 1.61, positive in 18 of 21 scenes). On point centers with merging off, uniform also leads by +44.27 points at tight budgets. At the shipped cap, which keeps about half of the points, it leads by +2.29 (1.01 to 3.52; Table 28, Appendix J.6).
The random pool still leads after densification. You asked about pipelines that "subsequently move points, refine geometry, or optimize rendering primitives", and we ran one: a fuller Gaussian fit of 30,000 steps with densification and equal final Gaussian counts, on 28 scenes with fixed cameras. Random leads by +1.20 dB PSNR (95% range 0.67 to 1.72), and DTU accounts for the gain. The seven BlendedMVS scenes alone give +0.29 dB, from -0.93 to +1.53, too close to call (Appendix J.2). The VGGT-Omega counterexample you credited came from an error in one of our scripts, and with that corrected random leads there too (general comment, Appendix E). Registration measured without the full eligible set favors random on disturbed DTU. We measure the errors against a camera-derived reference, which we fit from predicted cameras and benchmark calibration and which does not use ALL. Under disturbance on DTU, rotation error is 1.05 degrees for random, 1.06 for ALL and 4.95 for ranked, and translation error shows the same pattern. Our fixed success test of 2 degrees and 1% of radius gives 0% for every DTU arm, including ALL, so on DTU the error values are what separate the arms. On BlendedMVS, disturbed success is higher for random and undisturbed success is higher for ranked (Table 27, Appendix J.5).
Random's lead with a narrow pool excludes zero in 22 of 24 eligibility and tolerance settings. A narrow pool has four candidates per output point. On DTU the random pool leads at all six eligibility cuts we tried, and at the main cut the ceiling excludes recovery at all three tolerances (Appendix J.1). BlendedMVS is more sensitive to tolerance. At the main cut the ceiling rules out 28 of 28 conditions at native and double tolerance and 8 of 28 at half, where random's narrow-pool lead is +10.10 (95% range 3.94 to 19.29). Both exceptions are BlendedMVS at half tolerance, one with the confidence cut at 1 and one with percentile-mask eligibility, and in both the interval spans zero.
The retention crossover, where the random-minus-ranked scene mean first becomes negative, moves with both settings. On BlendedMVS at the main cut it falls at 10% retention at half tolerance, 50% at native and 75% at double. These are scene means, and at native tolerance the interval at 25% retention includes zero. A higher confidence cut also moves it up, and on DTU ranking first overtakes random at 50% retention only with the cut at 1. So we tie the advice to tight budgets, where random's narrow-pool lead stays above zero in 22 of these 24 settings, rather than to any one crossover point.