- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VPGGT, building on OmniVGGT to explore feed-forward geometry estimation with optional camera and depth priors. The core methods include: introducing synthetic pose perturbations inspired by sensor noise during training; incorporating aligned camera priors into the iterative state of the camera prediction head via parameter-free Prior Residual Connections (PRC); and predicting a unified physical scale through Metric Global Attention (MGA). Matched-training experiments across four datasets demonstrate that PRC improves translation direction accuracy and joint pose AUC when all views have camera priors. However, performance gains are less consistent under partial prior coverage, and the full model shows noticeable performance variance across different datasets.
Critical strengths
Reveals a valuable failure mode and proposes a lightweight intervention. The "prior dilution" analysis shows that even when given helpful camera estimates, the model can output poses worse than the input priors. Figure 2 further illustrates that simply feeding accurate priors into the existing iterative process is insufficient; the process itself must adapt to prior inputs. PRC addresses this without introducing additional parameters, offering a clean design with a transparent mechanism.
Core PRC results are backed by convincing matched-training experiments. The baseline and PRC variants share identical initializations, training configurations, and training budgets. When noisy camera priors are provided for all views without depth, PRC improves RTA@5° by 8.11 and 12.84 percentage points on TUM RGB-D and C3VD, respectively (Table 1). The appendix also provides results for partial coverage, directional translation metrics, and predictions on views without depth, which helps clarify the scope and limits of the approach.
Critical weaknesses
The isolated contribution of Metric Global Attention remains unverified. Table 5 reports only the full model without an MGA ablation under identical training conditions, and lacks comparisons against external physical scale reconstruction baselines. Additionally, because the full model underwent two extra epochs of fine-tuning, system-level differences cannot be cleanly attributed to MGA. While Appendix A.4 defines a baseline that directly fits scale using input camera priors, no experimental results are reported for it. This comparison is critical: on Sewerage, even with accurate camera and depth priors, the scale-aligned depth AbsRel is 0.0098, whereas the physical-scale AbsRel remains 0.1258 (Tables 4–5). As a result, it is currently unclear whether the learned scaling mechanism outperforms directly recovering scale from existing physical scale observations.
Gains over matched-trained baselines do not always translate to improvements over the raw input estimates themselves. For example, under full noisy camera priors, the input priors on TUM achieve an AUC of 81.07, whereas PRC-only achieves 76.37 (Table 1). Partial prior coverage can also degrade performance relative to matched baselines: on TartanAir-V2 at 30% accurate camera coverage, AUC drops from 72.91 to 70.92 (Table 8). In full-system comparisons, VPGGT achieves a lower AUC than the published OmniVGGT in 11 out of 12 settings on TartanAir-V2 (Table 4). While the authors explicitly acknowledge these findings, combined with the small evaluation subsets and synthetic noise setups, the practical utility of the method is not yet fully established.
Questions for authors
-
Under the same training budget, could you compare MGA against a scale token that does not condition on prior scale, as well as the prior scale fitting baseline defined in Eq. (13)?
-
In addition to the existing input prior reference scores, could you evaluate a baseline that retains provided poses and only predicts missing ones, reporting image pair results separately for pairs with priors on both views, one view, and neither view?
-
Holding all other factors constant, could you contrast models trained strictly on clean priors against those trained on noisy priors, and test them across varying noise intensities or unobserved perturbation types?
4 comments
W2. Do gains over C1 imply gains over the supplied poses or across domains?
No. On TUM RGB-D with noisy cameras on every view and no depth input, AUC@30° rises from 70.89 for C1 to 76.37 for C2, still below the supplied prior's 81.07. On the same inputs, C2 exceeds the prior on RTA@5°, 67.00 versus 66.00. C3VD shows the reverse ordering across metrics: C2 exceeds the prior on AUC, 72.81 versus 71.73, but not on RTA@5°, 55.46 versus 57.73. We distinguish improvement over matched fine-tuning from improvement over the supplied pose and do not claim uniform cross-domain gains.
Q2. What happens if supplied poses are kept unchanged?
Using the same C2 checkpoint, we first predict all poses from RGB alone. We fit one similarity transform from the predicted to the supplied camera centers on the provided views, transform the full predicted trajectory, and copy the supplied poses back unchanged. The table gives joint AUC@30° as keep-provided / PRC on the same windows. AA, AU, and UU denote pairs with two, one, and zero supplied poses, respectively.
| Dataset | Poses | Coverage | Windows | AA | AU | UU |
|---|---|---|---|---|---|---|
| TartanAir-V2 | Exact | 30% | 660 | 100.00 / 83.04 | 28.64 / 67.34 | 71.80 / 72.76 |
| TartanAir-V2 | Exact | 70% | 660 | 100.00 / 85.86 | 51.98 / 66.62 | 65.07 / 67.32 |
| C3VD | Exact | 30% | 45 | 100.00 / 90.57 | 4.86 / 58.90 | 58.73 / 58.65 |
| C3VD | Noisy | 70% | 45 | 67.76 / 69.80 | 7.18 / 54.14 | 55.16 / 54.99 |
Copying preserves exact AA poses by construction. PRC gives higher AU AUC in all four examples, whereas UU AUC changes little on C3VD. On TartanAir-V2 at 70% exact coverage, these differences nearly cancel in all-pair AUC: 75.26 for keep-provided versus 75.65 for PRC. The appendix gives the four-domain breakdown.
Q3. Does training with corruptions improve robustness to test noise?
We have not isolated that training effect. C2 was trained with a mixture of clean and synthetically corrupted poses. At full camera coverage without depth input, its joint AUC changes from 80.46 with exact poses to 76.37 with the tested corruption on TUM RGB-D, and from 88.48 to 72.81 on C3VD. These comparisons measure test-time sensitivity. Without matched clean-only training, they do not establish how much robustness is caused by including corruptions during training.
Thank you for highlighting the prior-dilution diagnosis and the matched PRC results. We also appreciate your distinction between predicting geometry and reproducing the supplied inputs. Your questions help separate three outcomes, which are beating the trained baseline, retaining the input pose, and recovering physical scale.
W1/Q1. Does direct scale conditioning outperform an unconditioned token or supplied-pose fitting?
On the 20 Sewerage windows in your example, supplied-pose fitting outperformed learned scale. We held the Full predictions fixed and fitted one scale per window to the exact camera baselines supplied as input. Median-aligned depth AbsRel remained 0.0098, whereas metric depth AbsRel fell from 0.1258 to 0.0773 and point-radius error from 12.05% to 3.96%. Thus median alignment concealed a substantial difference in physical units. This test does not establish an advantage over an equally trained unconditioned token or the same ordering when exact cameras are unavailable.