acceptodds
Under review as a conference paper at ICLR 2027

UniPose9D: Universal Category-Agnostic Object Pose Estimation

Abstract

Object pose estimation is a fundamental problem in 3D vision. Although recent state-of-the-art approaches achieve strong performance, generalization to novel categories and unseen scenes remains challenging. We propose UniPose9D, a unified model for category-agnostic 9D object pose estimation: given an instance mask/ROI and either an RGB-D observation or an RGB image with predicted depth, the model estimates rotation, translation, and metric size without category labels, CAD models, mean-shape priors, or reference views. Specifically, UniPose9D samples point pairs from the observed object geometry and uses DINOv2 and PointNet features to predict NOCS coordinates for each pair. To improve accuracy, we introduce a point-pair-based RANSAC N-hop Kabsch–Umeyama algorithm with an adaptive threshold. We further employ flow matching to address symmetric ambiguities and construct a large-scale training set by curating and aligning pose annotations from existing public datasets. Experiments across eight datasets show that a single unified model achieves competitive performance on standard benchmarks while generalizing to unseen objects, unseen categories, and in-the-wild scenarios.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
object pose estimation, category-agnostic generalization, 3D vision, flow matching, normalized object coordinates, RGB-D perception

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hEZBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

UniPose9D estimates rotation, translation, and metric box dimensions from a masked RGB-D observation or RGB with predicted depth, without category labels or object-specific references at inference. It combines DINOv2 and PointNet features, joint flow prediction of canonical point-pair coordinates and box size, and scale-calibrated RANSAC–Kabsch–Umeyama with adaptive refinement. A single model is trained on four datasets and evaluated across eight, including in-domain benchmarks, transfer tests, and component ablations.

Critical strengths
  • Broad evaluation of a shared model: The evaluation across multiple domains, including baselines retrained on the same four-dataset mixture (Table S1), provides meaningful evidence for the practical value of sharing one category-agnostic model. Additional tests on unseen instances and category subsets support the empirical contribution.
Critical weaknesses
  • Insufficiently established novelty and incomplete positioning. CPPF++ already predicts distributions over canonical point-pair coordinates, and its official implementation applies the same observed-to-canonical pair-distance ratio as Eq. (6) for scalar scale calibration. Point-pair prediction, probabilistic canonical coordinates, and this calibration therefore do not constitute clear standalone contributions. The potentially distinctive contribution is the joint NOCS–size flow and its specific integration with alignment; the incremental value of this combination is insufficiently established. The omission of OmniNOCS/NOCSformer (ECCV 2024) and Object Pose Transformer (arXiv, 2026) further weakens the positioning, given their relevance to shared canonical pose prediction and inference without category-specific heads or category labels.
  • Ablations do not adequately substantiate the claimed contributions. The flow ablation uses direct L2 regression (Appendix A.6), leaving its advantage over symmetry-aware regression unresolved. Consequently, the reported gain does not distinguish benefits specific to flow modeling from benefits of handling symmetry-induced ambiguity. The contribution of 3DCorrEnhance beyond standard DINOv2-S/14 is also unquantified. Although Table 8 includes a separate scale-decoupling ablation, the plain-RANSAC comparison changes scale calibration and the alignment formulation, so it cannot isolate iterative refinement. The controlled increase from one to three hops improves REAL275 5∘5cm5^\circ 5\mathrm{cm} mAP only from 59.9 to 60.2 (+0.3 percentage points; Table S5), providing limited support for presenting N-hop refinement as a major contribution.

References

  • You, Yang, et al. "Cppf++: Uncertainty-aware sim2real object pose estimation by vote aggregation." IEEE Transactions on Pattern Analysis and Machine Intelligence 46.12 (2024): 9239-9254.
  • Krishnan, Akshay, et al. "Omninocs: A unified nocs dataset and model for 3d lifting of 2d objects." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
  • Li, Weihang, et al. "Object Pose Transformer: Unifying Unseen Object Pose Estimation." In CVPR. 2026.
Questions for authors
  • Which specific advances over CPPF++ are claimed as novel, and how does the proposed setting differ from OmniNOCS/NOCSformer and Object Pose Transformer?
  • Does the joint flow outperform symmetry-aware regression with identical features, training data, and alignment, particularly on symmetric objects?
  • How much does 3DCorrEnhance improve performance over standard DINOv2-S/14, and is the isolated 0.3-point gain from additional refinement hops reproducible across runs or datasets?
Reviewer_VRwjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

UniPose9D estimates rotation, translation and metric box dimensions from a localized RGB-D observation, or RGB with predicted depth. A joint flow model predicts contextual canonical point pairs and size, followed by scale calibration and robust alignment. It learns canonical conventions from aligned annotations and uses no category label, CAD model or reference view at inference. One model is trained on four datasets. The paper evaluates in-domain performance, unseen-instance transfer and unseen-category subsets separately.

Critical strengths
  1. Good performance from one shared RGB-D model. In Table S1, UniPose9D exceeds the All-DS GenPose++ variant in all four reported columns, including 40.4 versus 31.2 on PACE and 45.4 versus 28.5 on REAL275. This is meaningful in-domain evidence for a shared estimator. The paper also reports stronger native specialists and separates unseen-instance from unseen-category tests.
  2. Useful ablations and practical limits. Removing the flow head or point-pair representation reduces REAL275 5°5 cm mAP from 60.2 to 33.2 or 51.6 (Table 8). Tables S3–S6 report pair-budget, solver and input-perturbation studies. A 3,000-pair configuration retains similar accuracy at 0.979 s per object. The paper openly reports poor RGB translation accuracy, slow sampling and the small HANDAL diagnostic.
Critical weaknesses
  1. The central generalization claim is not adequately validated. Table 6 compares published mAP on 17 Omni6D categories with pooled accuracy on 13 categories for UniPose9D; aggregation and IoU-gating equivalence are not established. It cannot support a cross-method ranking. Section 4.3 also omits the canonical-frame transforms and calibration data behind the large rotation gains. Learning semantic conventions is reasonable, but the evaluation must specify how they map to each dataset and whether all methods receive equivalent calibration. These are central requirements for interpreting unseen-category pose accuracy.
  2. The gains over baselines and inherited components are not sufficiently isolated. Table S1 shows CleanPose dropping from 61.6 to 4.0 on REAL275 and GenPose++ from 10.0 to 7.1 on its native Omni6DPose metric. Against native GenPose++, Table 2 instead gives 10.5 versus 10.0 at 5°2 cm and a tie at 10°2 cm. Baseline-specific adaptation and convergence, plus UniPose9D single-dataset controls, are needed to interpret the All-DS advantage. The point-pair/tuple design and flow machinery have close precedents; Table 8 changes context or the loss alongside the component, while extra refinement hops add only 0.3 points. The benefit of joint NOCS–size modeling remains unclear.
Questions for authors
  1. Please provide each baseline's All-DS adaptation and explain the native-domain regressions. How does UniPose9D trained only on SOPE or REAL275 compare under the same evaluation?
  2. Can every Table 6 method use the same 13-category examples, aggregation, IoU gate, masks and symmetry rules? Please report per-category or symmetry-group results.
  3. What fixed canonical-frame mappings and calibration data are used for each method in YCB-Video, HOPE and HANDAL? Are mappings object- or category-specific, and were they fixed before evaluation? A relative pose-consistency diagnostic, as in OV9D, could help distinguish a convention mismatch from inaccurate pose prediction.
  4. With the features and five-point context fixed, what is gained by joint rather than separate NOCS and size prediction? Please clarify the architectural inheritance from CPPF++ and isolate the proposed additions.
Reviewer_9fHWOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces UniPose9D, a category-agnostic framework for 9D object pose and bounding box size estimation from masked RGB-D or RGB inputs. The method extracts point-pair features using DINOv2 and PointNet, regresses NOCS coordinates and 3D dimensions via Gaussian mixture flow matching, and solves the final pose through an iterative RANSAC-Umeyama algorithm. Trained on a combined mixture of four heterogeneous datasets, the unified model achieves zero-shot generalization across multiple benchmarks without requiring CAD models or category templates at inference.

Critical strengths
  • Unifying multiple heterogeneous datasets into a single category-agnostic model establishes a practical, strong baseline for open-world object pose estimation.
  • The evaluation spans an extensive set of benchmarks, convincingly validating the model's cross-dataset generalization capability under RGB-D inputs.
Critical weaknesses
  • Limited methodological novelty While the system is well-engineered and effective, its core algorithmic design relies heavily on established components e.g., the point-pair parameterization builds on CPPF/CPPF++; symmetry handling adapts standard Gaussian Mixture Flow Matching to flattened 9D vectors rather than a formulation tailored to 3D manifolds; and the scale ratio and adaptive RANSAC are straightforward extensions of classic geometric alignment. The primary contribution seems lie in engineering and scaling across diverse datasets.

  • Heavy reliance on ground-truth masks for evaluation: Although the framework targets category-agnostic generalization, zero-shot benchmarks on unseen categories are evaluated using ground-truth masks. In open-world deployment where category-specific detectors are absent, off-the-shelf open-vocabulary segmenters often introduce bg clutter and boundary bleed. The controlled mask erosion/dilation tests in Table S6 may not fully capture model robustness under real-world segmentation noise.

Questions for authors

Given that ground-truth masks are unavailable in realistic open-world scenarios, how does UniPose9D perform end-to-end when integrated with open-vocabulary detection and segmentation pipelines (e.g., Grounding DINO + SAM)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.