acceptodds
Under review as a conference paper at ICLR 2027

NOCSFlow: Learning Dense Canonical Fields for Monocular Category-Level 9D Pose Estimation

Abstract

Category-level monocular 9D object pose estimation remains challenging because accurate pose recovery requires reasoning about object geometry from a single RGB observation without explicit geometric information. Dense canonical representations such as Normalized Object Coordinate Space (NOCS) are promising for this problem, as they preserve rich pixel-wise 2D–3D correspondence structure for downstream pose recovery, yet learning such representations directly from monocular RGB remains underexplored. In this work, we introduce NOCSFlow, a pixel-space flow-matching framework that learns dense NOCS representations by injecting features from a pretrained geometry encoder as implicit geometric conditioning into multiple transformer layers. We further introduce coupled supervision that jointly optimizes the generated canonical fields with direct canonical-space supervision and pose-space supervision backpropagated through a differentiable PnP module. Experiments on CAMERA25 and REAL275 demonstrate substantial improvements over prior methods, including on the rotation metric and on the joint metric on REAL275.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
9D Object Pose Estimation, NOCS Representation, Generative Pose Estimation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PQ2VOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents NOCSFlow, a flow-matching framework that learns dense NOCS representations from a single monocular RGB image for category-level 9D pose estimation. A frozen MoGe-2 geometry encoder extracts implicit geometric features from the RGB image and object mask, which are injected into a pixel-space flow-matching generator through multi-layer patch-to-patch conditioning; the generator iteratively denoises a noise field into a per-pixel NOCS canonical coordinate field. The predicted field is supervised jointly by a per-pixel field loss and a pose-space loss backpropagated through a differentiable EPro-PnP solver, together with a scale regression term. At inference, the terminal field is decoded into the 9D pose through a single pass of weighted PnP. NOCSFlow outperforms prior methods on CAMERA25 and REAL275 and achieves strong zero-shot transfer on Wild6D.

Critical strengths

++ The design is clean and well motivated: a pretrained geometry encoder supplies implicit geometric priors, and the canonical field is trained under coupled supervision from direct field losses and pose-space gradients through a differentiable PnP module.

++ The method outperforms existing approaches on the evaluated benchmarks and generalizes well to Wild6D without fine-tuning, supporting the effectiveness of the proposed method.

Critical weaknesses

-- The paper does not include a failure case analysis. It would be helpful to provide qualitative failure examples and an analysis of typical error modes.

-- Code is not available for review, which limits reproducibility given the magnitude of the reported gains.

-- Figure 2 does not clearly convey the input pipeline: the resolutions of the image and mask, shapes of the features, and whether the network consumes the full image or a resized crop of the object region. Since these details affect both correspondence supervision and multi-object scaling, it would help to annotate them in the figure.

Questions for authors

-- Among the conditioning variants, cross-attention did not converge. Is this an optimization issue or a sign of mechanistic incompatibility with the flow-matching formulation? Some diagnosis would further justify the choice of additive patch-to-patch injection.

-- The geometry encoder is kept frozen throughout pose training. Have the authors compared against a fine-tuned variant? An ablation of this choice would clarify whether freezing is essential.

1 comment

Dear Reviewer PQ2V,

Thank you again for your suggestions on the failure cases and architecture figure. In our rebuttal, we outlined how we plan to illustrate small-object failures and clarify the figure. We also followed up on your questions about cross-attention and the decision to freeze the geometry encoder. We would appreciate hearing whether our response addresses the points you raised, or if there is anything you would like us to explain more clearly in the revision.

Thank you for your time and thoughtful feedback.

Reviewer_N9m9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NOCSFlow estimates category-level 9D object pose from an RGB image, a predicted instance mask, and known camera intrinsics. A frozen MoGe-2 geometry encoder conditions a pixel-space flow model that generates dense canonical coordinates. Learned metric-scale and size-ratio heads recover object dimensions, while confidence-weighted PnP estimates rotation and translation. Training combines direct canonical-field supervision with pose losses applied through the solver to intermediate clean-field predictions. The paper reports substantial improvements on CAMERA25 and REAL275 and transfer to Wild6D without pose-task fine-tuning. The principal contribution is the integration of geometry-conditioned canonical-field generation and downstream pose supervision.

Critical strengths
  1. A coherent connection between canonical prediction and pose estimation. Predicting the clean field at each sampled flow time makes it possible to apply solver-based pose supervision without unrolling the complete sampling trajectory. The paper also explicitly predicts metric scale rather than assuming that normalized correspondences determine absolute size. Table 4 provides supporting evidence: adding differentiable PnP reduces NOCS MAE from 0.060 to 0.051 and increases joint pose mAP from 79.5 to 83.2. Improving both the intermediate representation and the downstream task supports the proposed coupling.

  2. Strong reported empirical performance across several settings. Table 1 reports 85.1 joint mAP on REAL275 and 69.6 on CAMERA25, substantially above the listed monocular baselines. The additional Wild6D evaluation addresses transfer beyond the training-domain scenes and instances. The separate reporting of observed-depth DiffusionNOCS in Table 3 appropriately acknowledges that its input setting differs. Table 4 further shows gains from multi-layer conditioning under the same pretrained-encoder baseline, so the evidence is not limited to comparisons against older backbones.

Critical weaknesses
  1. The metric definition and provenance of the Wild6D baseline comparison need clarification. Table 2 and supplementary Table A8 label GIVEPose's joint result as 40.3 under (10°, 10 cm). Table A8 additionally lists per-category values of 76.7, 96.2, 2.8, 18.1, and 7.5. All five values and the overall result match GIVEPose's supplementary Table D.3, where they are reported under the scale-agnostic metric (10°, 0.5d), with d denoting the object-box diagonal. This correspondence leaves the comparability of the reported joint results unresolved. The authors should identify their source and establish a consistent metric through corrected labels or evaluation under a common protocol. The IoU50, IoU75, and translation columns in Table 2 use different metric definitions and values from GIVEPose's original scale-agnostic results, so the scale-recovery and reevaluation procedures also need explanation. In particular, IoU-based pose pre-matching/filtering and the treatment of unmatched objects should be specified consistently across methods. GIVEPose original paper; official supplement; NOCS evaluator.
Questions for authors
  1. How should the matching GIVEPose per-category results in Table A8 and GIVEPose's Table D.3 be reconciled with their different metric labels? Which results in Table 2 were independently reevaluated? Please identify the checkpoint, metric definition, scale-recovery procedure, and evaluator settings, and provide the comparison under a common metric. How are pose matching/filtering and missed objects handled across methods?
  2. Appendix B.1 aligns the NOCSformer comparison's training data, input resolution, augmentations, and training budget, while retaining its own heads and objectives. Table 4 also supports multi-layer conditioning and pose feedback. Which comparison best isolates flow generation from deterministic canonical-field prediction with the same encoder, metric heads, field/pose supervision, and comparable capacity? If no such comparison is available, could the authors clarify the scope of the evidence for the contribution of flow generation?
3 comments

Dear Reviewer N9m9,

Thank you again for pointing out the discrepancy in our Wild6D metric labels and for suggesting a direct regression baseline. In our rebuttal, we clarified which evaluation protocol applies to each comparison and discussed how the number of inference steps affects NOCSFlow’s performance relative to one-pass regression. We would appreciate your feedback on whether these clarifications address your main concerns. We would be happy to discuss any remaining questions during the discussion period.

Thank you for your time and consideration.

Thanks for the clarification and the new regression baseline. Most of my concerns have been addressed. I maintain my rating of weak acceptance.

Many thanks for your constructive questions and suggestions. We are glad our conversation and extra experiments addressed your concerns.

Reviewer_8Vv3OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes NOCSFlow for category-level monocular 9D pose estimation. Given an RGB image, a predicted mask and the camera intrinsics, a pixel-space flow-matching transformer generates a dense NOCS map. The generator is conditioned on features from a frozen MoGe-2 encoder injected in several blocks, and the same features are used to predict the metric scale and size. The pose is recovered with a differentiable weighted PnP (based on EPro-PnP), which is also used during training to add a pose loss on top of the NOCS loss. Experiments on CAMERA25 and REAL275 show large gains over monocular baselines (e.g. 85.1 on 10°10cm on REAL275), plus zero-shot results on Wild6D.

Critical strengths
  • paper is well written and easy to read
  • From the paper, result look qualitatively and quantitatively good, clearly overperforming state of the art.
Critical weaknesses

W1: The manuscript also cite Lee et al., 2025 but does not compare with them, even though it is highly relevant, given that they perform equivalently (or better) in terms of 3D IoU, they still underperform in most cases for mAP but given that the method does not rely on masks, nor external geometric prior, this us not a surprise. Hence the claim in the current manuscript are not correctly supported.
In the current state, it is not possible to recommend acceptance given that the main baseline is conveniently ignored (even though cited). It preforms similarly in terms of IoU, and drastically over perform the current method on REAL275 in terms of IoU, even though they don’t get external knowledge from the a strong geometry encoder (which explains the lower performances in terms of 10cm metric in there case).

W2: Nothing really exciting, since NOCS + PNP is not really novel (LaPose, EPro-PnP, DMSR), nevertheless results are clearly over performing presented baselines, mostly due to the added knowledge from the geometric encoder.

W3: The paper claims “It first generates a pixel-wise Normalized Object Coordinate Space (NOCS) field from the RGB image and object mask, without using depth, point clouds, or category-level CAD models as explicit conditioning inputs.”, but in reality the MoGe backbone is equivalent as providing a pointmap with metric anchoring, even though only “implicitly", I find the statement over claiming the actual capacities.

Minor:

  • “representations directly from monocular RGB remains underexplored.” seems like a strong statement I would disagree with.
  • Why is the norm term in Eq. (10) needed?
Questions for authors

Currently, without proper discussion of the most related baselines, it is impossible to suggest acceptance. Please address this critical concern to consider leaning towards acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.