acceptodds
Under review as a conference paper at ICLR 2027

QuacamFM: Quaternion-Constrained Flow Matching for Camera Pose Estimation

Abstract

Camera pose estimation from multi-view images remains a challenge in computer vision. Traditional methods often address this problem using Structure-from-Motion (SfM) with bundle adjustment. However, camera poses estimated from sparse views are inherently ambiguous due to insufficient geometric constraints. Recent work leverages probabilistic models, such as diffusion models, to generate multiple camera pose hypotheses and therefore capture this uncertainty better. Most of these methods represent camera rotations using unit quaternions, but treat them as unconstrained 4D vectors during the generative processes, thereby ignoring the unit-norm constraint of quaternions. Unconstrained quaternions create non-smooth and suboptimal generation trajectories. To this end, we propose *QuacamFM*, a quaternion-constrained flow matching framework for camera pose estimation that preserves unit quaternion representations throughout the entire flow trajectory. We design the optimal transport of the quaternion flows using smooth spherical linear interpolation. Experiments on CO3Dv2 demonstrate our method's advantage in camera pose accuracy over the diffusion-based quaternion methods and classical SfM approaches. We further show that our quaternion-constrained formulation outperforms the naive application of standard flow matching to 4D quaternion vectors on sparse-view camera pose estimation. Finally, it is observed that QuacamFM generalizes well across datasets and in-the-wild examples. The code is released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Sparse-view, camera poses, flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_Nr38OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies camera pose estimation from sparse multi-view inputs: given N unordered images, it predicts the rotation and translation of each image relative to a randomly selected reference frame.

Critical strengths
  1. The constraint that a quaternion must lie on the unit sphere is enforced at every step of the generative process, rather than being imposed only as a loss term.
  2. A controlled variant shares the same architecture and hyperparameters, so the central claim is supported by direct numerical evidence.
  3. The appendix provides ablations over the backbone and the number of integration steps, which is useful reference material for practical deployment.
Critical weaknesses
  1. The translation branch produces a confusing result. The translation branch of QuacamFM and QuacamFM-lerp is identical in both design and formulation, yet translation accuracy (RTA) differs substantially in Table 1. A modification confined to the rotation branch should not directly alter the generation path or the supervision signal for translation. This result is difficult to interpret.
  2. The reported numbers do not match the authors' claims. PoseDiffusion's RTA@15 (79.8) and AUC@15 (47.5) both exceed the proposed method's (74.2 and 45.1), and RelPose++'s RRA@15 (84.5) also exceeds it (82.0). This is inconsistent with the paper's stated "advantage in camera pose accuracy over diffusion-based methods" and with the claim of "significant improvements over previous methods, including classic SfM and diffusion-based approaches".
  3. The baseline coverage is dated. The most recent methods compared against are PoseDiffusion (2023) and RelPose++ (2024). More recent feed-forward geometry methods — DUSt3R, MASt3R, VGGT, FLARE — as well as Camera-as-Rays and DiffusionSfM, are discussed in the related work but none of them appear in the experiment tables.
Questions for authors
  1. The translation branch is identical in QuacamFM and QuacamFM-lerp. Why does RTA@15 in Table 1 then improve from 60.4 to 74.2?
  2. On the quantitative results the proposed method is not uniformly ahead. What, specifically, is its advantage over PoseDiffusion and RelPose++?
  3. What is the advantage of the proposed method compared with current feed-forward geometry methods such as DUSt3R, MASt3R, VGGT, and FLARE?
3 comments

Thank you for your thoughtful feedback and recommendations. We provide the answers to your questions below:

Q1: Why does the translation metric differ in QuacamFM and QuacamFM-lerp?

During training, the model predicts camera translations in the camera-to-world representation. This allows the model to learn translations independently of camera orientations, providing a consistent training translation target.

During evaluation, we convert the predicted poses into the world-to-camera representation to maintain consistency with the evaluation protocol in PoseDiffusion. Since this conversion of translations involves estimated rotations, rotation inaccuracies can affect RTA, explaining why the metric can differ despite identical translation branches.

Q2: What are the advantages over PoseDiffusion and RelPose++?

Our method shows advantages in all metrics compared to PoseDiffusion w/o GGS, which has the same model capability and does not either use correspondences or post optimisation.

Comparing with PoseDiffusion and RelPose++, which require explicit correspondence or post-optimisation, we achieve higher accuracy at threshold of 3030, and comparable at threshold of 1515, while reducing inference time by approximately 19×19 \times and 10×10 \times, respectively.

Q3: What is the advantage of the proposed method compared with feed-forward methods?

Inspired by probabilistic modeling, our method can generate a distribution of plausible camera poses, capturing multiple valid hypotheses under ambiguous image observations. Please see Figure 7 in our uploaded revision version for a demonstration.

In challenging cases, such as symmetric objects, multiple camera poses may be equally plausible. In contrast, deterministic feed-forward models, e.g. VGGT, DUSt3R, can only produce one single pose estimate, which may fail to capture the correct solution when there is ambiguity.

In Figure 7, in section 4.4 in our uploaded revision version, we discuss how our method generates plausible hypotheses with input objects that are highly symmetric and have repeated structures.

We also added a comparison against RayDiffusion and DiffusionSfM, please see updated Table 1. RayDiffusion and DiffusionSfM use a dense-ray representation to parameterize the cameras, hence, we observed that they achieves higher accuracy but take 20×20 \times and 5×5 \times times longer inference than our method.

MethodRTA@30RRA@30AUC@30RTA@15RRA@15AUC@15s
parameterized-as-(R,t)
QuacamFM85.5 ± 1.2591.4 ± 0.9167.1 ± 1.0074.2 ± 1.5183.2 ± 1.2245.1 ± 0.770.46
parameterized-as-rays
RayDiffusion88.393.574.482.191.252.111.8
DiffusionSfM89.694.474.987.493.756.52.2

Thank you for the detailed responses.

The clarifications in Q1 and Q2 are helpful. But my concern in Weakness 3 remains. I pointed out that the original baselines are relatively outdated and asked for a comparison with recent feed-forward methods such as VGGT-omega, Pi3 and DUSt3R. The rebuttal still does not provide quantitative evidence that the proposed method is stronger than these approaches.

The discussion of pose ambiguity in Figure 7 and Section 4.4 is interesting: the method can sample multiple plausible hypotheses for symmetric or repeated structures, whereas VGGT and DUSt3R return a single estimate. This remains a qualitative argument. It does not show that the proposed method is more accurate, or more useful in practice, than current feed-forward estimators under the same evaluation protocol.

I therefore maintain my score of 2.

We thank the reviewer for further feedback.

We would like to further discuss the remaining concern.

The overall benefit of probabilistic methods, including ours and other baselines, over feed-forward methods is their ability to generate multiple plausible hypotheses. However, since only a single ground-truth pose is available for each ambiguous input, rather than the underlying pose distribution, we cannot reliably evaluate uncertainty for any probabilistic method.

The key is not simply which method achieves higher accuracy, but which can capture a meaningful distribution of plausible solutions. For example, with symmetric objects, multiple camera poses can produce identical observations.

Figure 7 demonstrates our method's ability to capture such plausible distributions. The selection of a particular hypothesis depends on the downstream task.

We do not compare against current feed-forward methods, e.g. VGGT, for two reasons:

  1. Methodological focus: Our approach is probabilistic and is therefore more directly comparable to probabilistic baselines such as Ray Diffusion and DiffusionSfM. Under this comparison, our method achieves substantially faster inference.

  2. Fairness of training settings: VGGT and VGGT-Omega are trained on many massive datasets, making comparable training infeasible. Our reported results, including results in submission and rebuttal phases, follow consistent experimental settings to ensure fair comparisons.

Reviewer_jwSHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents QuacamFM, a conditional flow-matching method for sparse multi-view camera pose estimation. Its key idea is to model rotations on the unit-quaternion manifold rather than as unconstrained four-dimensional vectors. The method constructs a spherical interpolation path between a sampled source rotation and the target rotation, predicts an angular-velocity field conditioned on image features, and integrates this field through multiplicative quaternion updates. Translations are modeled with conventional Euclidean flow matching. The paper evaluates the method on CO3Dv2, held-out CO3Dv2 categories, RealEstate10K, and several in-the-wild examples.

Critical strengths

Geometrically well-motivated formulation. The paper identifies a meaningful mismatch in prior pose-generation methods: unit quaternions represent valid rotations only on S3S^3, whereas additive Euclidean updates can leave this manifold. QuacamFM addresses this consistently through a slerp-based conditional path, angular-velocity prediction, and Hamilton-product integration. The ablation against QuacamFM-lerp provides evidence that this design improves rotation estimation.

Practical accuracy–efficiency trade-off. The method achieves strong performance on CO3Dv2 while maintaining inference time comparable to PoseDiffusion without geometric guidance and substantially lower than methods requiring correspondence-based guidance or iterative post-optimization. The evaluation also includes held-out object categories, cross-dataset testing on RealEstate10K, and qualitative in-the-wild examples.

Critical weaknesses

The main empirical evidence for the paper’s central claim is the comparison with QuacamFM-lerp. However, this ablation does not fully isolate the contribution of the proposed components. In particular, the paper appears inconsistent about whether QuacamFM-lerp normalizes the quaternion after each integration step: this is stated in the introduction, but is not explicit in the additive update in Section 3.3. Since projection back to S3S^3 can itself materially affect performance, this ambiguity weakens the causal interpretation of the reported gains. A stronger ablation should separately evaluate: (i) lerp with no normalization, (ii) lerp with per-step normalization, (iii) slerp with additive integration plus normalization, and (iv) slerp with Hamilton-product integration.

Some empirical claims should be more carefully qualified. While QuacamFM performs strongly on CO3Dv2, its cross-dataset results on RealEstate10K are mixed: for example, its AUC@30 is below PixSfM and full PoseDiffusion. The claim of strong generalization would be more convincing with additional domains, multiple random seeds and uncertainty estimates, and more controlled comparisons under matched backbone, training-data, and compute-budget settings.

Questions for authors

1.How do you handle the quaternion double-cover ambiguity, i.e., the fact that qq and −q-q represent the same rotation? In particular, do you enforce q0⊤q1≥0q_0^\top q_1 \geq 0 before applying slerp or use another sign-canonicalization procedure?

2.The paper describes the slerp trajectory as an “optimal transport” path. Could the authors clarify in what sense this path is optimal? Is optimality formally established for the conditional distributions used here, or is “optimal transport” used more informally to denote the chosen interpolation path?

3.Could you report mean and standard deviation over multiple random seeds, especially for the comparison between QuacamFM and QuacamFM-lerp? This would help assess the statistical robustness of the claimed improvement from quaternion-constrained flows.

4.Why does QuacamFM obtain lower AUC@30 than PixSfM and full PoseDiffusion on RealEstate10K despite its strong CO3Dv2 results? Do you believe this is mainly due to domain shift, the lack of explicit correspondence constraints, or another factor?

5.Could the authors provide a more granular ablation that separately isolates the contributions of: (i) the slerp conditional path, (ii) angular-velocity prediction, (iii) Hamilton-product integration, and (iv) unit-norm enforcement? The current QuacamFM versus QuacamFM-lerp comparison changes several components simultaneously.

6.The performance is non-monotonic with the number of integration steps: 3030 steps outperform both 5050 and 8080 steps. Could the authors explain this behavior? Does it reflect numerical error, accumulated velocity-prediction error, or a mismatch between training and inference discretization?

7.Can the method handle image sets with severe occlusion, repeated structures, low overlap, or highly symmetric objects? A failure-case analysis would clarify when the learned quaternion prior compensates for missing geometric correspondences and when explicit geometry remains necessary.

4 comments

Thank you for your thoughtful feedback and recommendations. We provide the answers to your questions below:

Q1: How do you handle the quaternion double-cover ambiguity?

Our slerp implementation ensures that q0q_0 and q1q_1 are less than 90∘90^\circ apart, by flipping the sign of q1q_1 if ⟨q0,q1⟩<0\langle q_0, q_1 \rangle < 0. Therefore, the interpolated qtq_t at all t∈[0,1]t\in [0,1] span the shortest great arc on the manifold.

In QuacamFM-lerp, we do not perform this procedure since we want to treat the quaternions completely as 4D Euclidean vectors, where vector sign should not matter. The quaternion construction does however ensure that the 4D vectors have a non-negative scalar part, which may help to some extent.

Q2: What sense this slerp trajectory is optimal?

The optimal transport is to describe the chosen interpolation trajectory between the quaternion q0q_0 and q1q_1. Through comparative experiments with Quacam-lerp, we found that this trajectory is also optimal for learning under our camera pose estimation setting. Using lerp requires normalization at each step during both training and inference. This introduces an additional challenge for the model, as it must learn to compensate for the effects of normalization.

Q3 & Q5: Could you report mean and standard deviation? Could the authors provide a more granular ablation?

We thank reviewer for suggesting further ablations to isolate the contributions. However due to the limited rebuttal time frame, we choose to only ablate the unit-norm enforcement as suggested. This highlight the importance of properly using the quaternion algebra for learning in our work.

To this end, we provide QuacamFM-lerp w/o norm, which does not perform normalization at any intermediate step, but only the final step to convert to valid rotations.

We provide the mean and standard deviation over multiple random seeds as below:

MethodRTA@30RRA@30AUC@30RTA@15RRA@15AUC@15
QuacamFM-lerp77.6 ± 1.1976.5 ± 0.6551.5 ± 0.7861.3 ± 2.1067.8 ± 1.6541.6 ± 0.70
QuacamFM-lerp w/o norm78.1 ± 1.6377.9 ± 1.0453.3 ± 0.8663.2 ± 0.3570.8 ± 1.6244.4 ± 1.12
QuacamFM85.5 ± 1.2591.4 ± 0.9167.1 ± 1.0074.2 ± 1.5183.2 ± 1.2245.1 ± 0.77

Q4: Why full PoseDiffusion and PixSfM perform better on RealEstate10k?

The lower performance on RealEstate10K is mainly due to domain shift, a trend also observed in other learning-based baselines reported in Table 3. RealEstate10K is a scene-centric dataset containing outdoor scenes, showing a substantial domain shift from the object-centric CO3Dv2 dataset. Unlike CO3Dv2, RealEstate10K does not have much of symmetry-induced ambiguity. This makes, for example, PoseDiffusion w/o GGS significantly drop to 28.7 on AUC@30.

On this dataset, full PoseDiffusion and PixSfM achieve the best performance because they use explicit correspondences and use post-optimisation.

Q6: The performance is non-monotonic with the number of integration steps, why? Is it a mismatch between training and inference discretization?

The integration is conducted following the Euler method, a coarse integration (using 20 steps) may introduce larger numerical integration errors, and would cause overshoot. Whereas, with dense integration updates, the velocity-prediction error may accumulate and and cause trajectory drift. Empirical experiments show 30 integration steps is the optimal setting.

This is not due to a training–inference discretization mismatch, as our continuous-time flow-matching framework samples tt uniformly from [0,1][0,1] during training. While inference uses a user-defined number of Euler integration steps.

Q7: Can the method handle image sets with repeated structures, low overlap, or highly symmetric objects?

Please see Figure 7, in section 4.4, in our uploaded revision version for a demonstration, where we discuss how our method generates plausible hypotheses when the input object is highly symmetric and has repeated structures. We illustrate this with an example of a bowl that appears visually similar from different viewpoints, demonstrating how our method generates a distribution of plausible camera poses.

The repeated texture, low overlap, and highly symmetric objects are presented in the CO3Dv2 dataset, such as bowls and donuts, and so on. We observe numerous cases in which establishing reliable correspondences is particularly difficult, this motivates our probabilistic approach.

Thank you for your timely and detailed response. I appreciate the additional experiments and clarifications, which address several of my earlier concerns and strengthen the paper.

I find your explanation of the optimal transport interpretation reasonable, particularly from a geometric and empirical perspective. The additional ablation results and multi-seed evaluations are also helpful in supporting the effectiveness of the proposed method.

However, I believe a more detailed ablation study isolating the contributions of individual components would further strengthen the paper. Additionally, while selecting 30 integration steps based on empirical performance is reasonable, a more thorough analysis of the non-monotonic behavior across different integration steps would be valuable.

I also appreciate the newly added uncertainty analysis in Figure 7. It provides an intuitive demonstration of how the method captures ambiguity in symmetric objects. A quantitative evaluation of uncertainty or hypothesis coverage would make this analysis more convincing.

Overall, I appreciate the authors' efforts in addressing my questions. The rebuttal has helped clarify the method and strengthen the empirical evidence, and I remain positive about the paper's contribution.

We thank reviewer for appreciating our efforts and maintaining a positive assessment of our work.

Meanwhile, we are conducting other suggested ablation studies and will update the results, to provide further insights into the contributions of individual components.

AuthorsOpenReview ↗

Additional ablation studies for insight discussion

Dear reviewer,

We would like to update you with additional ablation studies, where we individually ablate different components in training and inference. Particularly, we ablate the transport path, Hamilton-product, Euler update, and unit-norm enforcement as in the table below:

 Inference:Hamilton-productEuler integrationUnit-norm enforcementAUC@30AUC@15
Training
slerp ✓\checkmark×\times×\times67.1±1.0067.1 \pm 1.0045.1±0.7745.1 \pm 0.77
slerp ×\times✓\checkmark×\times49.3±1.5449.3 \pm 1.5439.4±1.3239.4 \pm 1.32
slerp ×\times✓\checkmark✓\checkmark48.2±2.1548.2 \pm 2.1538.7±1.8938.7 \pm 1.89
lerp ×\times✓\checkmark✓\checkmark51.5±0.7851.5 \pm 0.7841.6±0.7041.6 \pm 0.70
lerp ×\times✓\checkmark×\times53.3±0.8653.3 \pm 0.8644.4±1.1244.4 \pm 1.12
lerp ✓\checkmark×\times×\times51.1±1.3451.1 \pm 1.3442.2±0.7942.2 \pm 0.79

Note:

  • When using Hamilton-product for integration during inference, the unit-norm enforcement is not needed.
  • When using lerp path, the prediction is q˙\dot{q}, the integration via Hamilton-product is formulated as: qt+Δt=qt⊗(q˙t Δt)q_{t+\Delta t} = q_t \otimes (\dot{q}_t ~ \Delta t)

Discussion: We observe that if the inference integration is consistent with the training transport path, it achieves the highest accuracy with to its category, i.e using lerp with Euler integration without unit-norm enforcement; and using slerp with Hamilton-product.

Using slerp with Hamilton-product for integration yields the best among all of them.

Reviewer_s3kbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes QuacamFM, a flow-matching-based method for sparse multi-view camera pose estimation. Probabilistic methods such as PoseDiffusion represent camera rotations with unit quaternions, but treat them as unconstrained 4D vectors during additive noising and denoising. This breaks the unit-norm constraint, leading to non-smooth generation trajectories and reduced rotation accuracy. To address this, the paper models rotation and translation separately within the flow matching framework. For rotation, the OT path is defined as the spherical linear interpolation (slerp) between the initial quaternion and the ground-truth quaternion, the flow is described by a quaternion differential equation, the network predicts the angular velocity ω, and inference is performed by multiplicative integration via the Hamilton product so that the quaternion always stays on the unit sphere S³. Experiments on CO3Dv2 show improvements over PoseDiffusion and RelPose++ and over a lerp variant of the same model, and generalization is evaluated on held-out categories and RealEstate10K.

Critical strengths
  1. The motivation is clear and the problem is real. Treating quaternions as unconstrained vectors during generation produces invalid rotations, and Fig. 1 illustrates this well by showing that the trajectory of QuacamFM on the manifold is smoother than those of both the lerp variant and PoseDiffusion.
  2. The method is simple and clean. The slerp path, angular velocity prediction, and Hamilton product integration all fit together, so the rotation stays on S³ the whole time and no extra normalization is needed.
  3. Inference takes about 0.46 s without correspondence matching or post optimization, which is much faster than PoseDiffusion with GGS and RelPose++, while the accuracy at the @30 threshold is on par or slightly better.
  4. Compared with PoseDiffusion w/o GGS, which has similar model capacity and runtime, QuacamFM achieves clearly higher accuracy
Critical weaknesses
  1. Missing comparisons with recent state-of-the-art methods. The strongest baselines in Tab. 1 are PoseDiffusion and RelPose++. Several more recent methods evaluated under the same CO3Dv2 protocol, such as Cameras as Rays (ICLR 2024) and DiffusionSfM (CVPR 2025), are cited in the related work but not compared, and both report substantially higher accuracy. The comparison should also include recent feed-forward methods such as VGGT and Depth Anything 3.
  2. The gap between QuacamFM and QuacamFM-lerp is the main evidence for the quaternion constraint, but the paper does not say how the double cover of quaternions is handled. Since q and −q represent the same rotation, if the sign of the target is not aligned with the source, the lerp path often passes close to the origin, where normalization becomes unstable. This alone could explain much of the poor performance of the lerp variant. The authors should clarify whether sign alignment is applied and report the lerp results with it.
  3. The generalization to RealEstate10K is overstated(L422-L424). The paper claims that QuacamFM “outperforms the other baselines” on RealEstate10K. However, Tab. 3 shows that while its rotation accuracy is good, its translation accuracy and overall AUC are clearly worse than PoseDiffusion and also worse than the classical PixSfM and COLMAP+SPSG. The text should reflect this, and it would be useful to discuss why translation generalizes much worse than rotation when moving from object-centric to free-camera motion.
Questions for authors
  1. Is the sign of the target quaternion aligned with the source (so that ⟨q₀, q₁⟩ > 0) during training? Is this done for both the slerp and the lerp variant?
  2. The introduction motivates probabilistic models by their ability to capture multi-modal ambiguity. At inference, is a single sample drawn or multiple? Have the authors examined multiple hypotheses on symmetric objects?
  3. In Tab. 5, the results do not change monotonically with the number of integration steps (30 steps are best, 50 steps are worse, and 80 steps are better again). This variation of about 3 points in AUC@30 is larger than the margin over PoseDiffusion and RelPose++ in Tab. 1. How much run-to-run variance is there? Error bars over several runs would help.
1 comment

Thank you for your thoughtful feedback and recommendations. We provide the answers to your questions below:

Q1: Is the sign of the target quaternion aligned with the source?

Our slerp implementation ensures that q0q_0 and q1q_1 lie in the same hemisphere by flipping the sign of q1q_1 if ⟨q0,q1⟩<0\langle q_0, q_1 \rangle < 0. Therefore, the interpolated qtq_t at all t∈[0,1]t\in [0,1] lie on the shortest great arc.

In QuacamFM-lerp, we do not perform this alignment procedure since we want to treat it as 4D Euclidean vectors, where the vector sign should not matter.

We also would like to provide another variant QuacamFM-lerp w/o norm, also suggested by reviewer 4, to reflect the nature of 4D Euclidean vectors. This variant does not perform normalization at any intermediate step but only the final step to convert to valid rotation for evaluation.

MethodRTA@30RRA@30AUC@30RTA@15RRA@15AUC@15
QuacamFM-lerp77.6 ± 1.1976.5 ± 0.6551.5 ± 0.7861.3 ± 2.1067.8 ± 1.6541.6 ± 0.70
QuacamFM-lerp w/o norm78.1 ± 1.6377.9 ± 1.0453.3 ± 0.8663.2 ± 0.3570.8 ± 1.6244.4 ± 1.12
QuacamFM85.5 ± 1.2591.4 ± 0.9167.1 ± 1.0074.2 ± 1.5183.2 ± 1.2245.1 ± 0.77

Q2: At inference, is a single sample drawn or multiple? Have the authors examined multiple hypotheses on symmetric objects?

In our manuscript, we sampled 100 hypotheses and reported the average results.

We now also discuss how our method generates plausible hypotheses with input objects that are highly symmetric and have repeated structures, please see the Fig. 7 in section 4.4, in our uploaded revision version for the demonstration.

We observe that with distinct appearance in different views, our method will generate reliable, and certain poses for multiple hypotheses.

Q3: Why do the results not change monotonically with the number of integration steps? What is the error bar?

The integration in inference is using the Euler method, a coarse integration (using 20 steps) may introduce larger numerical integration errors, and would cause overshoot. Whereas, with dense integration updates, velocity-prediction error may accumulate and and cause trajectory drift. Empirical experiments demonstrate that 30 integration steps is the optimal setting.

We provide the error bars for Tab. 5 as requested:

Metrics20 steps30 steps50 steps80 steps
AUC@1543.6 ± 0.7045.1 ± 0.7744.1 ± 0.5543.7 ± 0.90
RRA@1577.9 ± 1.3883.2 ± 1.2281.5 ± 0.9579.5 ± 1.18
AUC@3064.4 ± 1.0767.1 ± 1.0065.3 ± 1.6566.4 ± 1.30

More baseline comparison:

We provide the comparison against Ray Diffusion and DiffusionSfM, which share the probabilistic modeling paradigm of our method, in the table below. The experiments are conducted in the same setting as Tab 1. We update the table with error bars, as suggested by reviewer 4. Please see Table 1 in the updated revision version.

RayDiffusion and DiffusionSfM use a dense-ray representation to parameterize the cameras, hence, we observed longer inference costs. RayDiffusion estimates the patch-wise rays, while DiffusionSfM estimates pixel-wise ray endpoints. They achieve higher accuracy but take 20×20 \times and 5×5 \times times longer, respectively.

MethodRTA@30RRA@30AUC@30RTA@15RRA@15AUC@15s
parameterized-as-(R,t)
QuacamFM85.5 ± 1.2591.4 ± 0.9167.1 ± 1.0074.2 ± 1.5183.2 ± 1.2245.1 ± 0.770.46
parameterized-as-rays
RayDiffusion88.393.574.482.191.252.111.8
DiffusionSfM89.694.474.987.493.756.52.2
Reviewer_EaaDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes QuacamFM for sparse multi-view camera pose estimation. It uses SLERP and multiplicative quaternion updates to preserve valid rotations throughout flow matching, while modeling translations with Euclidean flows.

Critical strengths

1.Preserving the unit-quaternion constraint throughout generation is a well-motivated design. 2. The reported improvements over QuacamFM-lerp are encouraging, and the evaluation includes unseen categories and cross-dataset generalization.

Critical weaknesses
  1. Riemannian Flow Matching [1] already addresses manifold-valued data, and ReQFlow [2] uses unit quaternions and SLERP. The paper should clarify its methodological contribution beyond adapting these ingredients to camera pose estimation.

  2. Equation (7) defines a four-dimensional Lerp target, but the accompanying description refers to angular velocity. It is unclear whether QuacamFM-lerp predicts a four-dimensional quaternion derivative or a three-dimensional angular velocity, and how normalization is handled during training and inference. These details are essential for interpreting the reported gains.

  3. Ray Diffusion [3] and DiffusionSfM [4] are discussed but not evaluated. Comparisons with these generative pose estimation methods would better establish competitiveness beyond PoseDiffusion, considering both accuracy and runtime under consistent evaluation settings.

Reference

[1] Chen and Lipman. Flow Matching on General Geometries. ICLR, 2024.

[2] Yue et al. ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation. ICML, 2025.

[3] Zhang et al. Cameras as Rays: Pose Estimation via Ray Diffusion. ICLR, 2024.

[4] Zhao et al. DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion. CVPR, 2025.

Questions for authors
  1. Could the authors provide training and sampling pseudocode for QuacamFM-lerp, specifying its output dimensionality, target velocity, and normalization procedure?
  2. Could the authors compare with Ray Diffusion and DiffusionSfM , while accounting for differences in training data and additional optimization?
2 comments

Thank you for your thoughtful feedback and recommendations. We provide the answers to your questions below:

Q1: Training and sampling pseudocode for QuacamFM-lerp

We confirm that in Quacam-lerp, the network predicts a 4D vector, not the angular velocity, as it treats qq as an unconstrained Euclidean vector. This is consistent with the integration step in Eq. (11) during inference. The intermediate state qtq_t is normalized during both training and in each inference step.

We provide the pseudo-code for the variant Quacam-lerp as below:

Training:

Sample θ0∼N(0,I3)\theta_0 \sim \mathcal{N}(0, \mathbf{I}_3), then convert it to a quaternion: q0= quat⁡(θ0)q_0 = ~\operatorname{quat}(\theta_0)

Given a ground-truth rotation R1R_1: q1=quat⁡(R1)q_1 = \operatorname{quat}(R_1)

Sample random t∈[0,1]t \in [0,1] q_t = \frac{\operatorname{lerp}(q_0,q_1,t)}{\left\\| q_t\right\\|} q˙t=q1−q0 ,q˙t∈R4\dot{q}_t = q_1 - q_0 ~, \dot{q}_t \in \mathbb{R}^4 q˙^t=model⁡(qt,t,I), ,q˙^t∈R4\hat{\dot{q}}_t = \operatorname{model}(q_t,t,I), ~, \hat{\dot{q}}_t \in \mathbb{R}^4 \hat{q}_1 = \frac{q_t + (1-t)\hat{\dot{q}}_t}{\left\\| q_t + (1-t)\hat{\dot{q}}_t \right\\|}

Inference:

Sample q0= quat⁡(θ0)q_0 = ~\operatorname{quat}(\theta_0), where θ0∼N(0,I3)\theta_0 \sim \mathcal{N}(0,\mathbf{I}_3).

For t∈[0,1]t \in [0,1], with Δt=1/30\Delta t = 1/30:

q˙^t=model⁡(qt,t,I)\qquad \hat{\dot{q}}_t = \operatorname{model}(q_t,t,I)

\qquad q_{t+\Delta t} = \frac{q_t + \hat{\dot{q}}_t \Delta t}{\left\\|q_t + \hat{\dot{q}}_t \Delta t\right\\|}

Q2: Comparison against more baselines

We provide a comparison against Ray Diffusion and DiffusionSfM in the table below. The experiments are conducted in the same setting as Tab 1. We have updated the table below with mean and standard deviation, as suggested by reviewer 4.

RayDiffusion and DiffusionSfM use a dense-representation to parameterize the cameras, hence, we observed longer inference costs. RayDiffusion estimates the patch-wise rays, while DiffusionSfM estimates pixel-wise ray endpoints. They achieve higher accuracy but take 20×20 \times and 5×5 \times times longer, respectively.

All currently reported baselines in Tab 1 are directly comparable to our method, since they all parameterize cameras as (R,t).

MethodRTA@30RRA@30AUC@30RTA@15RRA@15AUC@15s
parameterized-as-(R,t)
PoseDiffusion†\mathrm{PoseDiffusion}^{\dagger}79.986.766.579.880.547.58.83
QuacamFM85.5 ± 1.2591.4 ± 0.9167.1 ± 1.0074.2 ± 1.5183.2 ± 1.2245.1 ± 0.770.46
parameterized-as-rays
RayDiffusion88.393.574.482.191.252.111.8
DiffusionSfM89.694.474.987.493.756.52.2

We have added these to the Tab. 1, and categorize them as parameterized-as-rays methods, please see uploaded revision version for updates.

Thank you for the detailed response and additional experiments. The clarification of QuacamFM-lerp and the new baseline comparisons address two of my concerns.

However, my main concern regarding methodological novelty, particularly the distinction from existing manifold flow matching methods such as ReQFlow, remains unresolved.

I appreciate the authors' efforts, but I will maintain my original rating.

Reviewer_1VgEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes QuacamFM, an image-conditioned flow-matching model for sparse multi-view camera pose estimation. It models camera rotations on S3S^3 via SLERP conditional paths and multiplicative Hamilton-product updates, while modeling translations with standard Euclidean flow matching, conditioned on multi-scale DINO-ViT features. Experiments on CO3Dv2 (seen and unseen categories) and zero-shot RealEstate10K compare QuacamFM against COLMAP+SPSG, PixSfM, PoseReg, RelPose++, PoseDiffusion (with and without GGS), and a Euclidean flow baseline (QuacamFM-lerp).

Critical strengths
  1. Geometrically valid rotation trajectory with fast inference. Formulating rotation flow on S3S^3 via SLERP and multiplicative updates avoids off-manifold intermediate states without requiring expensive correspondence-guided post-optimization, running in 0.46 s for 10 views vs. 8.83 s for PoseDiffusion with GGS (Table 1).
  2. Consistent gains over unguided PoseDiffusion and QuacamFM-lerp. Across CO3Dv2 seen categories (Table 1), unseen categories (Table 2), and RealEstate10K (Table 3), QuacamFM consistently outperforms PoseDiffusion w/o GGS and the QuacamFM-lerp ablation at similar runtime.
Critical weaknesses
  1. Inconsistent quaternion equations and ambiguous specification of the QuacamFM-lerp baseline.
    • Ordering mismatch: Section 3.1 defines quaternions as scalar-first q=(w,v)q=(w,\mathbf{v}), but the 4×44\times 4 matrix in Equation (1) places the 3×33\times 3 block −[ω]×-[\boldsymbol{\omega}]_\times in the top-left and 00 in the bottom-right, which is the vector-first convention.
    • Missing factor of 2: Equation (2) integrates angular velocity ωt\boldsymbol{\omega}_t using half-angles θ/2=∥ωt∥Δt/2\theta/2 = \|\boldsymbol{\omega}_t\|\Delta t / 2. Under the standard unit-quaternion logarithm (e.g., Sola, 2017, cited in Section 3.2), Vec⁡(log⁡(q0−1⊗q1))=θ2u\operatorname{Vec}(\log(q_0^{-1}\otimes q_1)) = \frac{\theta}{2}\mathbf{u}, so the body angular velocity is ω∗=2Vec⁡(log⁡(q0−1⊗q1))\boldsymbol{\omega}^* = 2\operatorname{Vec}(\log(q_0^{-1}\otimes q_1)). Equation (5) omits the factor of 2 (which would integrate a 90∘90^\circ rotation to only 45∘45^\circ) and mislabels log⁡(q0−1⊗q1)\log(q_0^{-1}\otimes q_1) as q˙∗\dot{q}^*.
    • Ambiguous QuacamFM-lerp and antipodal sign handling: Equation (7) defines a 4D derivative q˙lerp∗=q1−q0∈R4\dot{q}_{\text{lerp}}^* = q_1 - q_0 \in \mathbb{R}^4, yet the next sentence states that the 3D angular velocity ω∗\boldsymbol{\omega}^* calculation and objective remain unchanged, while Section 3.3 says QuacamFM-lerp integrates in R4\mathbb{R}^4 via Equation (11). Furthermore, the paper does not state whether q0q_0 and q1q_1 are aligned to the same hemisphere (⟨q0,q1⟩≥0\langle q_0, q_1 \rangle \ge 0) to resolve the q≡−qq \equiv -q double-cover ambiguity, nor does it define the auxiliary losses LqL_q and LτL_\tau in Equation (12).
  2. Limited methodological novelty over ReQFlow and missing comparisons with recent camera-pose estimators. The core SLERP quaternion flow formulation follows ReQFlow (Yue et al., 2025, cited in Section 3.2). Because the mathematical flow construction is inherited, the paper's contribution relies on its empirical effectiveness for camera pose estimation. However, all quantitative baselines in Tables 1–3 date from 2024 or earlier (PoseDiffusion, ICCV 2023; RelPose++, 3DV 2024), while more recent probabilistic camera-pose methods already cited in Section 2—such as Cameras as Rays / Ray Diffusion (ICLR 2024) and DiffusionSfM (CVPR 2025)—are not compared. In addition, Tables 4 and 5 in the Appendix report RTA@15 = 45.1 for the default ViT-DINO (30 steps) model, which contradicts RTA@15 = 74.2 in Table 1 (where 45.1 is instead the value of AUC@15).
Questions for authors
  1. How are the scalar/vector ordering in Equation (1), the missing factor of 2 between Equations (2) and (5), the antipodal sign ambiguity (q≡−qq \equiv -q), and the definitions of Lq,LτL_q, L_\tau in Equation (12) handled in your actual implementation?
  2. In QuacamFM-lerp, does the network predict a 3D vector (as stated after Equation (7)) or a 4D vector (as implied by Equations (7) and (11)), and is the intermediate state qtq_t normalized during training as well as at each inference step (Section 1 vs. Section 3.3)?
  3. How does QuacamFM compare quantitatively against Ray Diffusion (Zhang et al., 2024) and DiffusionSfM (Zhao et al., 2025) on CO3Dv2, and is RTA@15 = 45.1 in Appendix Tables 4–5 a mislabeling of AUC@15 from Table 1?
3 comments

Thank you for your thoughtful feedback and recommendations. We provide the answers to your questions below:

Q1: Correction of equations

We have corrected Eq. (1) to the scalar-first convention to be consistent.

\begin{bmatrix} 0 & -\boldsymbol{\omega}^{\top} \\\\ \boldsymbol{\omega} & -[\boldsymbol{\omega}]_{\times} \end{bmatrix} \mathbf{q}$$ * We thank the reviewer for pointing out the missing factor of 2. We have modified Eq. (5) to $\boldsymbol{\omega}^* = 2 \textrm{Vec}(\log(\mathbf{q}_0^{-1} \otimes \mathbf{q}_1))$. We confirm that our implementation is correct and follows this expression. * In our implementation of *slerp* for generating the transport path, we ensure that $q_0$ and $q_1$ lie in the same hemisphere by flipping the sign of $q_1$ when $\langle q_0, q_1 \rangle < 0$. Therefore, the interpolated $q_t$ at all $t\in [0,1]$ lie in the same hemisphere. * The $L_\tau$ is the *l1* loss between two vectors, similar to Eq. 9, and the $L_q$ is defined as: $L_q =1-\left|\hat{\mathbf q}_1^{\top}\mathbf q_1\right|$, where $\hat{q}_1$ is the estimated rotation, and $\mathbf{q}_1$ is the ground-truth. **Q2**: *Clarification of QuacamFM-lerp implementation* We confirm that in *QuacamFM-lerp*, the network predicts a 4D vector as treating $q$ an un-constrained Euclidean vector, as implied in Eq. (7) and Eq. (11). We corrected line 237, after Eq. (7), in the **updated revision** version, and confirm that our implementation predicts a 4D vector. We confirm that the intermediate state $q_t$ is normalized during both training and in each inference step. This is to ensure that $q_t$ is a pure rotation, similar to when using *slerp*. **Q3**: *Comparison against more baselines* We would like to provide the comparison against Ray Diffusion and DiffusionSfM in the table below. The experiments are conducted in the same setting as Tab 1. We update the table below with mean and standard deviation, as suggested by reviewer 4. RayDiffusion and DiffusionSfM use a dense-representation to parameterize the cameras, hence, we observed longer inference costs. RayDiffusion estimates the patch-wise rays, while DiffusionSfM estimates pixel-wise ray endpoints. They achieve higher accuracy but take $20 \times$ and $5 \times$ times longer, respectively. All currently reported baselines in Tab 1 are directly comparable to our method, since they all parameterize cameras as (R,t). | Method | RTA@30 | RRA@30 | AUC@30 | RTA@15 | RRA@15 | AUC@15 | s | | -- | -- | -- | -- | -- | -- | -- | -- | | *parameterized-as-(R,t)* | |$\mathrm{PoseDiffusion}^{\dagger}$ | 79.9 | 86.7 | 66.5 | 79.8 | 80.5 | 47.5 | 8.83 | |QuacamFM | 85.5 ± 1.25 | 91.4 ± 0.91 | 67.1 ± 1.00 | 74.2 ± 1.51 | 83.2 ± 1.22 | 45.1 ± 0.77 | **0.46** | | *parameterized-as-rays* | |RayDiffusion | 88.3 | 93.5 | 74.4 | 82.1 | 91.2 | 52.1| 11.8 | |DiffusionSfM | 89.6 | 94.4 | 74.9 | 87.4 | 93.7 | 56.5 | 2.2 | We add these to the Tab. 1, and categorize them as *parameterized-as-rays* methods. Please visit the newly **uploaded revision** version, we have marked changes in **blue**. **Novelty**: Despite the similarity in flow construction, our method learns to *recover* correct poses from sparse-view observations, while ReQFlow focuses on *generating* protein backbones. **Typo**: We fixed the typo $RTA@15$ to $AUC@15$ in Tabs 4 and 5 in the Appendix. We use $AUC$ and $RRA$ in these experiments as we focus on rotational and camera accuracy.
AuthorsOpenReview ↗

Quaterion alignment clarification

We clarify that our slerp implementation ensures that q0q_0 and q1q_1 lie in the same hemisphere by flipping the sign of q1q_1 if ⟨q0,q1⟩<0\langle q_0, q_1 \rangle < 0. Therefore, the interpolated qtq_t at all t∈[0,1]t\in [0,1] lie on the shortest great arc.

Reviewer_1VgEOpenReview ↗

Follow-up after the author response

Thank you for the response and revised manuscript. The corrections to Equations (1) and (5), the 4D lerp specification, the appendix metric labels, and the added baseline comparisons address important parts of my original concerns. The additional multi-seed ablations provide useful evidence, including the substantial AUC@30 improvement.

I maintain my rating of 2 (Weak rejection). The component study still mixes different quaternion sign-handling conventions, and the reported lerp/Hamilton update requires clarification. The endpoint-loss formulas and sign conventions also remain insufficiently specified in the manuscript. These unresolved issues limit how convincingly the component study isolates the quaternion-specific contribution and keep my overall assessment below the acceptance threshold.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Brief summary of submission 11758 rebuttal.

We thank all reviewers for their thoughtful feedback and recommendations. We summarize our responses as below:

  • Quaternion alignment: We clarify that our slerp implementation aligns q0q_0 and q1q_1 lie in the same hemisphere by flipping the sign of q1q_1 if ⟨q0,q1⟩<0\langle q_0, q_1 \rangle < 0. Therefore, the interpolated qtq_t at all t∈[0,1]t\in [0,1] lie on the shortest great arc.

  • Variant QuacamFM-lerp: We clarify that we do not apply quaternion alignment on this, since we want to treat the quaternions completely as 4D Euclidean vectors. We perform normalization in each intermediate step, both training and inference. We provide one more variant, QuacamFM-lerp w/o norm, where we do not perform normalization in intermediate steps. Please see rebuttal sections.

  • Correction: We fixed the Eq. (1) to scalar-first convention to be consistent. We added the missing factor of 2 to Eq. (5), and fixed the notation for better reading. We fixed the typo RTARTA to AUCAUC in Tab. 4 and Tab. 5 in the Appendix.

  • Additional baselines: We added recent baselines, i.e. Ray Diffusion and DiffusionSfM, to the comparison Table. We observe that they achieves higher accuracy but take 20×20 \times times and 5×5 \times times longer inference than our method, due to their dense-ray camera parameterization.

  • Error bars: We provided the mean and standard deviation as suggested by reviewers.

  • Ambiguous case analysis: We provide section 4.4 and Fig. 7 in the uploaded revision version to discuss how our method handles images with repeated structures, or highly symmetric objects.

  • Method contribution: Despite the similarity in flow construction, our method learns to recover correct poses from sparse-view observations, while ReQFlow focuses on generating new samples.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.