- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a feed-forward framework for joint object pose estimation and reconstruction from RGB sequences. The core idea is to combine dense camera-local geometry with sparse sequence-shared geometry, recovering object poses through differentiable rigid alignment and using the same transformations for multi-view reconstruction. Experiments across different motion settings and datasets demonstrate strong performance.
Critical strengths
- The idea of connecting object pose estimation and reconstruction through a shared representation is well-motivated and intuitive.
- The framework is reasonably designed, avoiding predefined canonical object coordinates and test-time optimization while maintaining a unified geometric relationship between the two tasks.
- The experiment is relatively extensive, covering different motion conditions, external datasets, and component-level analyses.
Critical weaknesses
- The direct-regression baseline uses a frozen Stage-I representation, whereas the full method benefits from additional Stage-II training and geometric supervision. A comparison under matched training and supervision conditions would better isolate the contribution.
- The reported translation-direction metric is insensitive to both translation magnitude and direction reversal. Additional sign-sensitive direction and metric translation evaluations would provide a more complete assessment of pose accuracy.
- The AUC@15 values for ObjAligner differ between the main tables and Appendix Table 8. Similar discrepancies appear in several reconstruction metrics.
- Although the method avoids test-time optimization, it introduces additional shared-geometry prediction and rigid alignment on top of a pretrained multi-view backbone. End-to-end runtime and memory comparisons would better support the claimed efficiency.
Questions for authors
Please refer to the Weaknesses.
2 comments
Thank you again for your helpful suggestions. We have added Direct training, signed translation and NRMSE evaluations, and runtime and GPU memory measurements, and unified the inference settings and corresponding results in the main and appendix tables. Signed evaluation preserves the reported AUC, while ObjAligner reduces runtime by approximately 33% compared with SAM2 + π³ + black-bbox. We would appreciate your reassessment in light of these results and revisions when you have time.
Thank you for your careful review of our paper. In response to your questions about training ablations, translation evaluation, result consistency, and runtime overhead, we provide the following additional experiments and clarifications.
1. Additional training for Direct (W1)
Following your suggestion regarding direct regression training, we used the same Stage-I weights to conduct matched Stage-II training with direct pose regression, retaining the original relative-pose supervision. The results are as follows:
After this training, Direct still exhibits a clear gap relative to the full framework. These results will be added to the ablation table in the main paper.
2. Evaluation of signed translation direction and translation magnitude (W2)
Following your suggestion regarding translation evaluation, we additionally recomputed pose AUC using a direction error that preserves the translation sign, namely the angle between the predicted and ground-truth translation vectors (0°–180°). AUC@15 on COPE119 / UOPE56 / HO3D remains 27.84 / 21.18 / 11.01, indicating that the original AUC advantage does not rely on forgiving scoring of direction reversals.
Without GT calibration, the model can recover inter-frame relative rotation, translation direction, and relative translation at a common sequence scale. Translation magnitude is determined up to a single scale factor shared across the sequence.
Building on this, to evaluate the accuracy of translation magnitude, we further aligned the predicted and ground-truth translations using one nonnegative scale per sequence and computed scale-normalized translation NRMSE. The results are as follows:
3. Consistency of results between the main paper and appendix (W3)
Thank you for pointing out this discrepancy. We reported ablations of the inference settings in the appendix: a fusion threshold of 0.75 achieves the highest F in Table 14, and 32×32 queries achieve the highest AUC in Table 15. We therefore adopted this setting for the corresponding comparisons in the appendix, as explicitly stated at the beginning of Appendix D and in the caption of Table 8. Under the main-table setting of 12×12 queries and a 0.5 fusion threshold, AUC@15 on COPE119 / UOPE56 / HO3D is 27.84 / 21.18 / 11.01, still higher than all comparison methods in their respective main tables.
However, recognizing that this can indeed cause ambiguity, we will consistently use the main-table setting of 12×12 queries and a 0.5 fusion threshold and revise B.1, Appendix D, and the relevant table captions accordingly, so that the numerical results in the main paper and appendix correspond directly.
4. Runtime efficiency and GPU memory usage (W4)
Regarding your concern about runtime overhead, on the same RTX 4090 with a fixed 8-frame input, ObjAligner takes 0.57 seconds from RGB to poses and a fused point cloud, compared with 0.85 seconds for SAM2 + π³ + black-bbox, reducing runtime by approximately 33%.
These results will be added to the paper.