- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents Feed-Forward Bundle Adjustment (FFBA), which replaces per-scene iterative optimization with a single forward pass through a learned refinement network. Given initial camera parameters and 3D points from a geometric foundation model, together with fixed multi-view correspondences, FFBA constructs a heterogeneous graph comprising camera, 3D point, observation, and global nodes. Eight geometric graph aggregation layers jointly refine camera poses, focal lengths, and 3D coordinates. The framework is trained without manual annotations using a confidence-weighted reprojection loss. The authors construct approximately 0.9 million samples using three foundation models. Experiments on in-domain and cross-domain datasets demonstrate improvements over the initial geometry and substantially reduced refinement time, while ablation studies assess the contributions of the key components.
Critical strengths
-
To my knowledge, this is the first work I have encountered that explores a feed-forward approach to bundle adjustment. The method design and experimental results demonstrate the feasibility of replacing per-scene iterative optimization with learned feed-forward refinement. This provides a promising alternative to conventional BA and is a substantive contribution supporting acceptance.
-
bottlenThe paper addresses a meaningfuleck in pipelines built on geometric foundation models: their predictions retain geometric errors, while iterative refinement through conventional or differentiable BA can erode their speed advantage. FFBA substantially reduces refinement time while maintaining competitive accuracy. Although the results do not show a dramatic leap in accuracy, the approach offers an insightful direction for future research on efficient geometric refinement.
Critical weaknesses
The cross-domain experiments demonstrate FFBA’s ability to improve the initial geometry estimates, but its accuracy advantage over conventional BA is not consistent. FFBA achieves lower relative pose AUC than BA on three of the four cross-domain datasets, suggesting limitations in the transferability of the learned update rules across data distributions. Further analysis of the scene conditions associated with these accuracy gaps would help clarify when FFBA is a suitable alternative to conventional BA.
Questions for authors
FFBA improves the initial estimates across all evaluated cross-domain datasets, but its camera accuracy does not consistently surpass that of BA and EPO. This raises questions about the applicability of the learned update rules: to what extent does performance depend on the scale and distribution of the training data, as well as the GFMs used during training? Would FFBA require retraining or fine-tuning if the upstream GFM architecture or its error distribution changed? Furthermore, as GFMs increasingly incorporate geometric consistency constraints and potentially optimization mechanisms, how do the authors envision FFBA’s role as a standalone refinement module or as an integral component of a geometric foundation model?
2 comments
4. Future role: standalone refinement, training-time supervision, or integration with GFMs
We appreciate your perspective on the future role of learned geometric refinement. We see learned geometric refinement as both a standalone module and a potential component of GFM training. Enforcing geometric consistency during training does not guarantee that predictions for every test scene agree with the observed image correspondences. A standalone refinement module therefore remains useful for correcting residual errors, particularly on new scenes or domains, as demonstrated by our unseen-GFM experiments.
The low refinement cost of FFBA also makes training-time geometric supervision a promising direction. Refined reconstructions could provide additional teacher targets for an upstream GFM, allowing it to learn corrections that would otherwise be applied after inference. Alternatively, refinement could be integrated into the GFM and share its feature representations. These approaches could reduce the need for a separate refinement stage at deployment.
An outstanding practical challenge is to acquire correspondences efficiently and reliably. Whether tracks come from an external matcher or a GFM, they should provide constraints supported by image evidence and avoid reinforcing errors in the predicted geometry. We therefore view FFBA as a step toward inexpensive geometric correction. Efficient correspondence construction is central to both its current standalone use and future integration into GFM training, and will be a major focus of our future work.
We appreciate your time and remain available to answer any further questions.
We thank the reviewer for recommending acceptance and recognizing the value of FFBA. Below, we discuss the dependence on training data, present results on unseen GFMs, and outline future integration with GFMs.
1. When does FFBA fall short of BA?
We agree that FFBA does not consistently match BA's camera accuracy: BA achieves higher relative-pose AUC on ETH3D, LLFF, and Mip360, whereas FFBA performs better on T&T (Tab. 2). Our experience suggests that difficult initializations and weak geometric constraints, such as limited parallax or sparse view overlap, can challenge refinement with a fixed update budget; iterative BA can instead continue adapting to each scene.
2. Dependence on training data and initialization sources
FFBA learns geometric corrections from data, so its performance depends on how well training covers the scene structures and initialization errors encountered at test time. We train on three datasets and three GFMs with shared underlying tracks to broaden this coverage (Sec. 4). Our experience motivates balancing initialization quality and error patterns, rather than increasing sample count alone. We have not isolated the effects of training scale, data distribution, and initializer diversity, so we cannot quantify their individual contributions. Nevertheless, the unseen-GFM experiments below show that useful transfer is possible beyond the training initializers.
3. Does changing the upstream GFM require retraining?
Thank you for suggesting this evaluation, which directly tests transfer to unseen initializers. We evaluated MapAnything and DA3-Large, neither used during FFBA training, on three in-domain and four cross-domain datasets, using the same pretrained FFBA weights without retraining or fine-tuning.
In-domain
Cross-domain
FFBA improves all reported aggregate metrics for both models, including when both the initializer and scene domain are unseen. These results demonstrate strong transfer across the tested GFMs and scene domains, showing that changing the upstream GFM does not necessarily require retraining. Full comparisons with BA and EPO are provided in App. H, Tabs. 11–12.
Unfinished. Please keep reading the comments.