- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces a large-scale real-world dataset containing 9D pose annotations for 22.8M images from 114K object-centric videos. The authors propose a scalable canonicalization pipeline that combines multi-view reconstruction, cross-instance alignment, reference-based pose propagation, and human verification. Experiments demonstrate promising cross-dataset generalization and improved pose estimation performance over existing foundation models.
Critical strengths
- The paper addresses an important limitation in existing 9D pose datasets.
- The proposed annotation pipeline is well motivated and practical.
- The dataset offers impressive scale and category diversity. The cross-dataset experiments demonstrate strong generalization performance.
Critical weaknesses
- The current annotation quality analysis mainly focuses on rotation accuracy using 125 sampled objects. However, translation and object extent are also essential components of 9D pose annotations and may be affected by reconstruction errors, incomplete geometry, and scale ambiguity.
- The proposed pipeline relies on clustering and aligning different object instances to shared references. While reasonable, the main alignment validation is conducted on a relatively small subset.
- While the cross-dataset results are encouraging, it remains unclear how much of the improvement comes from the number of training instances, category diversity, or viewpoint coverage.
Questions for authors
Please refer to the Weaknesses.
3 comments
I appreciate the authors' response. My score remains unchanged.
We sincerely appreciate your time, positive assessment, and constructive suggestions, which have helped us further refine and strengthen the manuscript.
Response to Reviewer BM49 (R3)
We thank the reviewer for their positive assessment of the pipeline, dataset scale, and cross-dataset results. We have added direct annotation-quality measurements and controlled ablations in response to the concerns below; the shared questions are also discussed in the General Response (G2, G3).
W1. Quality of translation and extent
We agree that rotation accuracy alone is not sufficient to validate 9D pose annotations. We therefore manually annotated oriented 3D bounding boxes, including rotation, center, and extent, for 125 training objects from 125 categories, and compared them directly with our pipeline's boxes:
Regarding metric scale ambiguity, please see General Response G3. Our model handles arbitrary depth scales through point-cloud normalization, although the annotations themselves are not metric.
W2. Size of the alignment validation
We agree that the alignment validation is based on a relatively small subset. It was the largest we could identify with ground-truth poses for real-world object-centric videos. Importantly, though, the alignment ablation evaluates an intermediate step, not the quality of the final accepted labels:
The direct box evaluation in W1, together with exhaustive verification and downstream transfer, therefore provides additional evidence about the final annotations beyond the small alignment ablation.
W3. Instances vs. category diversity vs. viewpoint coverage
Please see General Response G2. We conducted controlled single-factor ablations and training-scale experiments (Tab. 6). Reducing viewpoint coverage or category diversity causes drops of up to 7.8 and 9.4 points, respectively, while reducing the instance count to 38% with all categories retained causes at most a 2.0-point drop. These results show that dataset breadth contributes substantially to cross-dataset generalization beyond instance count.