acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Dense Risk Prediction for Consistent Feed-Forward 3D Reconstruction

Abstract

Recent feed-forward 3D reconstruction models reconstruct dense 3D geometry from unconstrained multi-view images in a single forward pass. As a 3D foundation model, VGGT is increasingly used as a geometric prior in visual SLAM, dynamic scene reconstruction, and novel view synthesis, where VGGT confidence is crucial for point filtering and optimization weighting. However, VGGT confidence is learned jointly with geometric regression rather than directly supervised for depth accuracy, allowing high-confidence regions to contain severe depth errors. Such errors can survive filtering, receive large weights in optimization, and destabilize downstream tasks. We observe that the failures can be captured from VGGT outputs. Depth and point maps can reveal internal self-consistency of VGGT, and multi-view depth maps can capture geometric consistency across views. Motivated by this observation, we propose VERA, a VGGT ERror-Aware dense risk prediction network that adaptively fuses two confidence maps and two consistency signals to predict a dense risk map of depth errors. Trained on nine datasets and evaluated on five sequence-disjoint test sets, VERA substantially improves depth error prediction metrics, reducing AUSE by 30.1% while improving AUROC by 28.6% and AUPR-Error by 81.3% over the best raw signal baseline. When used as an optimization weight, VERA reduces VGGT-Long odometry ATE RMSE by 13.7% and reduces Chamfer distance after confidence-weighted pose refinement by 16.6%, all without retraining or modifying the frozen foundation model.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
3D reconstruction, risk prediction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9AwbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose a post-process module that predict a risk map from cues from depth confidence, point confidence, D2P confidence, MV geometric residuals provided by VGGT. The module is directly trained by depth risk supervision, which can filter unreliable depth for downstream applications such as visual odometry and reconstruction.

Critical strengths
  • The problem is clearly identified and well motivated.
  • Improvement on multiple tasks.
  • Technical soundness of the network architechture.
Critical weaknesses
  • The propose module is tied to VGGT outputs.
Questions for authors
  • requests. The improvement is not obviously shown in Fig 6, do you consider hightlight or replace it with more clear examples?
1 comment
AuthorsOpenReview ↗

Response to Question

Thank you for your review and for suggesting that we make the qualitative improvements in Figure 6 easier to see.

In the revised manuscript, we have added red boxes to Figure 6 and expanded the caption to explain what to look for in each example. In the first example, VERA reduces the overlapping surfaces around the chair. In the second, the window frame and wooden cabinet appear sharper and better aligned with VERA.

Reviewer_EYxTOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VERA, a framework that predicts the risk map of VGGT output. It uses four signals derived from VGGT outputs: depth confidence maps, point confidence maps, self-consistency maps from depth-to-point residual, and multi-view consistency maps from multi-view depth residual. The authors first use pyramid encoders to extract multi-scale features from each signal and use self-attention and spatial gating to combine the coarse features. Finally, the authors use a multi-scale decoder to produce the final risk map. Experiments show that VERA improves depth-error prediction, VGGT-Long odometry, and confidence-weighted pose refinement.

Critical strengths
  1. The research problem is important. Identifying unreliable predictions is useful for downstream tasks and applications like robotics and SLAM.

  2. The evaluation is solid. The authors consider depth-error prediction, long-sequence odometry accuracy, and reconstruction quality after pose refinement.

  3. The paper is well organized and easy to follow. The qualitative comparisons in Figure 6 and the ablations in Figure 7 make the effectiveness of each component clear.

Critical weaknesses
  1. VERA relies on signals derived from VGGT outputs. As also acknowledged in Section 5, errors that remain confident and geometrically consistent may be difficult to detect. It is also unclear how much VERA helps when the original confidence maps are already reliable.

  2. The evaluation is limited to VGGT as the 3D backbone. It is unclear how the method would generalize to other models, such as DA3 [1] and Pi3 [2]. These models may have different levels of cross-view consistency and confidence reliability. More experimental results would help.

  3. The paper already provides qualitative results, but more comparisons with the Simple MLP and Simple CNN baselines would help show the advantages of VERA over simpler learned predictors.

Questions for authors
  1. Could the authors provide more analysis on separate contributions of the two consistency signals? In Table 1, adding D2P to the confidence-only model changes AUSE only slightly, from 0.209 to 0.207. The full model improves all three metrics, but an additional confidence + multi-view consistency maps ablation would help separate the benefits of the two residuals.

  2. VERA does not consistently improve reconstruction quality. In Table 2, it sometimes underperforms raw point confidence and uniform weighting baselines. More analysis of these cases would help explain when the predicted risk is useful and when it can hurt downstream optimization.

  3. Could the authors clarify key parameter choices? The authors use binary targets that label the 20% of valid pixels with the highest absolute depth errors in each image as positive. Comparisons with continuous error targets and different thresholds would help justify the choice.

[1] Lin, Haotong, et al. "Depth anything 3: Recovering the visual space from any views."

[2] Wang, Yifan, et al. "π3\pi^ 3: Permutation-Equivariant Visual Geometry Learning."

8 comments
AuthorsOpenReview ↗

Response to Question 1

Thank you for suggesting this ablation. We trained a Confidence + MV variant using the depth and point confidence maps and the multi-view (MV) residual, with the depth-to-point (D2P) residual removed. We used the same training objective and evaluated the variant on the same five test datasets as in Table 1(b).

Input signalsAUSE ↓AUROC ↑AUPR-Error ↑
Confidence only0.2090.8470.613
Confidence + D2P0.2070.8650.665
Confidence + MV (new)0.2000.8610.651
Confidence + D2P + MV (VERA)0.1900.8820.700

When added separately to the confidence-only model, MV reduced AUSE by 0.009, compared with 0.002 for D2P. D2P yielded larger gains in AUROC and AUPR-Error. By combining the complementary signals from D2P and MV, VERA achieved the best performance across all three metrics.

AuthorsOpenReview ↗

Response to Quesetion 2

Thank you for pointing this out. VERA reduced absolute trajectory error (ATE) relative to raw point confidence on 9 of 11 sequences. ATE increased on sequences 03 and 04, the two shortest sequences in the benchmark. When the initial alignment is already accurate and few incorrect correspondences receive high confidence, VERA-based weighting may have less room to improve alignment and can reduce the influence of reliable correspondences.

On sequence 08, ATE increased from 55.90 m with uniform weighting to 57.52 m with raw point confidence. When overconfident failures degrade alignment, VERA can limit their influence by downweighting high-risk correspondences. With VERA-based weighting, ATE decreased by 0.97 m to 56.55 m.

AuthorsOpenReview ↗

Response to Question 3

Thank you for suggesting the comparisons with various target settings. In the paper, we define a binary target by labeling the top 20% of valid pixels by absolute depth error in each image as positives. We also trained VERA with positive ratios of 10% and 30%. To compare with continuous error targets, we sampled pixel pairs within each image and trained VERA to assign a higher risk score to the pixel with the larger absolute depth error.

Target typesAUSE ↓AUROC ↑AUPR-Error ↑
Continuous target0.16700.8480.641
Binary target, 10% positives0.20910.9120.629
Binary target, 20% positives (paper)0.19040.8820.700
Binary target, 30% positives0.18470.8540.739

The 20% setting achieved a lower AUSE than the 10% setting, with only a 0.0057 difference from the 30% setting. The 20% setting achieved higher AUROC than the 30% setting and higher AUPR-Error than the 10% setting. Given these trade-offs, we selected a positive ratio of 20% for the binary target.

Pairwise ranking based on continuous absolute depth errors achieved a lower AUSE than training with the binary target, but binary target achieved higher AUROC and AUPR-Error. We use the binary target because it directly trains VERA to distinguish high-error pixels from the remaining pixels.

Reviewer_8vnrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VERA, a small network that predicts depth error risk from VGGT confidence maps and geometric consistency signals. VGGT remains frozen. The predicted risk improves error detection and is used to weight downstream odometry and pose refinement.

Critical strengths

The paper addresses a practical limitation of using raw confidence for geometric optimization.

The MLP and CNN baselines use the same inputs and supervision.

The experiments include two downstream tasks and evaluation on KITTI with KITTI excluded from training.

Critical weaknesses

Labeling the highest error 20% of pixels in each image does not establish that predicted risks are comparable across images.

The experiments do not separately establish the benefits of attention and spatial gating.

Appendix A does not separately report VERA inference time, leaving the stated overhead unclear.

Questions for authors

How sensitive are the results to the 20% positive ratio? Have the authors considered absolute or relative depth error thresholds?

Can the authors report ablations removing attention and gating separately?

Does the reported runtime include the VERA forward pass? Please provide the complete timing breakdown.

3 comments
AuthorsOpenReview ↗

Response to Question 1

To assess the effect of the positive ratio, we trained VERA by labeling the 10%, 20%, or 30% of valid pixels with the largest absolute depth errors in each image as positive. We also evaluated a variant trained on the top 20% of pixels ranked by relative depth error.

Per-image positive labelsAUSE ↓AUROC ↑AUPR-Error ↑
Top 10% by absolute depth error0.20910.9120.629
Top 20% by absolute depth error (paper)0.19040.8820.700
Top 30% by absolute depth error0.18470.8540.739
Top 20% by relative depth error0.23120.8430.621

The results are averaged across five test datasets. For AUROC and AUPR-Error, each model was evaluated using the same label definition as in training. AUROC was highest in the absolute-error top-10% setting (0.912), where the task was to distinguish pixels with the most severe errors from the remaining pixels. AUPR-Error was highest in the top-30% setting (0.739), where the higher positive ratio also raises the random-ranking AUPR baseline from 0.20 to 0.30. AUSE improved slightly from 0.1904 at 20% to 0.1847 at 30%, but worsened to 0.2091 at 10%.

At a fixed positive ratio of 20%, the relative error setting yielded lower AUROC and AUPR-Error and increased AUSE from 0.1904 to 0.2312, indicating that absolute depth-error labels were more effective for the error-ranking task.

AuthorsOpenReview ↗

Reponse to Question 3

Thank you for pointing out that Appendix A omitted the runtime of the VERA forward pass. The originally reported total did not include this step. For the CO3D scene with 30 input views, the VERA forward pass takes 0.0419 s. The complete timing breakdown is given below.

Pipeline stageRuntime (s)
VGGT forward pass0.8277
Depth-to-point (D2P) residual computation0.0144
Multi-view (MV) residual computation0.0424
VERA forward pass0.0419
Total0.9264

Including the VERA forward pass, the end-to-end runtime is 0.9264 s, or 1.12× the runtime of VGGT alone, corresponding to an 11.9% overhead.

AuthorsOpenReview ↗

Response to Question 2

Thank you for suggesting these ablations. We trained two VERA variants, removing attention in one and spatial gating in the other.

ModelAUSE ↓AUROC ↑AUPR-Error ↑
Full VERA0.1900.8820.700
Without attention0.2000.8450.638
Without spatial gating0.1830.8610.665

Removing attention degraded performance on all three metrics. Removing spatial gating reduced AUROC and AUPR-Error but decreased AUSE from 0.190 to 0.183. Thus, spatial gating improved the detection of high-risk pixels, while risk-based sparsification deviated slightly more from the oracle curve. Overall, both components contributed to high-risk pixel detection, with larger decreases in AUROC and AUPR-Error when attention was removed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.