acceptodds
Withdrawn from ICLR 2027

DeformR: Single-View 3D Shape Retrieval with Image-Guided Shape Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Single-view 3D shape retrieval is a critical enabler for inverse graphics and detailed 3D scene representation, allowing generic 3D assets to serve as proxies for real-world instances. Recent feature-level analysis-by-synthesis approaches have improved robustness to occlusions and domain gaps through pose-conditioned feature alignment at test time. However, a rigid template may not match local geometry visible in an image. To address this mismatch, we propose DeformR, a framework that extends pose-aware 3D retrieval with local, image-guided shape adaptation. First, we formulate a 2D-to-3D feature-distillation and pose-refinement pipeline for shape retrieval and pose estimation. The pipeline supports multiple categories while retaining point-level representations. Second, with the selected template and pose fixed, DeformR is designed for conservative local adaptation rather than free-form reconstruction. Pose-aligned image features guide corrections to image-supported geometric residuals, while geometric regularizers preserve the retrieved template's global structure. Large geometric changes are neither expected nor desired. Experiments on Pascal3D and ImageNet3D show improved retrieval accuracy over the evaluated baselines and competitive pose-estimation performance. With the retrieved template and pose fixed, deformation improves alignment between the real-world image and the deformed shape.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
deformation, retrieval, 2D-3D correspondence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GXKZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for single-view 3D shape retrieval with image-guided shape adaptation. The core idea is to combine hybrid 2D-3D feature distillation, dense spatial contrastive learning, and test-time pose optimization to improve retrieval and pose estimation. A flow matching-based deformation module adapts the retrieved template using image features while preserving its geometric structure. Experiments on two datasets demonstrate improved retrieval and pose estimation performance.

Critical strengths
  1. The motivation of adapting rigid CAD templates to instance-specific geometric variations is reasonable and clear.
  2. Combining semantic and correspondence features with dense spatial contrastive supervision is intuitive and interesting.
  3. The experimental results are strong, showing consistent improvements in retrieval and pose estimation across different occlusion levels.
Critical weaknesses
  1. The proposed framework builds upon PASR and combines existing techniques, e.g., semantic and correspondence feature fusion, contrastive learning, and flow-matching-based deformation
  2. The ablation results suggest that much of the pose accuracy improvement comes from contrastive supervision, DIY-SC features, and rotation parameterization, while combining DINOv3 and DIY-SC provides relatively limited additional gains.
  3. The deformation module is trained toward annotated CAD proxies, and part of the evaluation also measures similarity to these proxies. It remains unclear how much of the adaptation is driven by instance-specific image information.
Questions for authors

Please refer to the Weaknesses.

Reviewer_hkKSOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeformR for single-view 3D shape retrieval under occlusion. It combines DINOv3 and DIY-SC features, improves retrieval and pose estimation through contrastive learning and test-time pose optimization, and locally adapts the retrieved CAD template using conditional flow matching.

Critical strengths

1.Strong retrieval and pose-estimation results on Pascal3D and ImageNet3D, especially under severe occlusion. 2.Dense spatial contrastive learning helps resolve ambiguities between repetitive parts such as wheels and wings. 3.The framework is clearly structured and includes ablations, runtime analysis, and failure cases.

Critical weaknesses

1.The deformation model is directly supervised by annotated CAD proxies, while CAD-proxy CD is also used for evaluation, creating coupling between the training objective and evaluation metric. 2.Deformation improves RS-IoU by only 0.76–2.93 percentage points, which provides limited evidence of instance-level geometry recovery. 3.Independent ablations of image conditioning, flow matching, Chamfer, ARAP, and Laplacian losses are missing. 4.The data split and generalization protocol are unclear, including whether target CAD models appear in the gallery and how the method performs on unseen CAD instances or categories.

Questions for authors
  1. Is it reasonable that the annotated CAD corresponding to a test image appear in the training set or retrieval gallery?
  2. How much improvement remains without image conditioning or CAD-proxy supervision?
  3. It's better that the authors compared against PASR using the same backbone, parameter count, and pose-optimization setting.
  4. Other issues mentioned above.
Reviewer_8yH8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the shape mismatch between rigid CAD templates and real instances in single-view 3D retrieval and proposes DeformR: it first performs pose-aware rigid retrieval and freezes both the selected template and its pose, and then applies deformation via 2D-conditioned flow matching, thereby avoiding the coupling that arises when retrieval and deformation are trained jointly—in which retrieval is biased toward deformable candidates rather than the best-matching ones. Its pipeline combines hybrid 2D feature distillation, dense contrastive alignment, 6D pose optimization, and flow-matching-based deformation into a sequentially decoupled pipeline, which achieves non-trivial improvement of retrieval performance.

Critical strengths
  1. DeformR attains the best Top-1 on Pascal3D at all occlusion levels (L0 79.90 vs PASR 76.43; L3 70.36 vs 63.05) with much stronger pose accuracy (L0 Accπ/18 66.14 vs 40.30; MedErr 6.37 vs 12.10). On ImageNet3D L3, Top-1 reaches 63.30, outperforming the strongest feed-forward baseline Uni3D (43.56) by 19.74 pp and PASR (42.66) by 20.64 pp, and the margin widens with occlusion, consistent with the claimed local-visibility matching behavior.
  2. Explicitly freeze retrieval ranking and pose before deformation, preventing deformation from contaminating retrieval/pose results.
Critical weaknesses
  1. While the experimental results and system-level design are impressive, the technical contributions need to be further clarified. The pose-aware render-and-compare backbone, discrete-pose initial search plus test-time pose refinement, are taken from PASR. The debias operation is borrowed from INSID3, CVPR2026.
  2. The ablation study in Table 3 adds components monotonically (rows 1→7) with no rollback or factorial design, so interactions (e.g., Debias × contrastive loss, hybrid features × 6D rotation) are unidentifiable.
  3. Writing: Missing citation entry for SDFit, please add the full reference and its reported Pascal3D+ numbers. Figures 1 and 2 are never referenced in the text. Figure 2 in particular depicts exactly the training pipeline described in 3.2 (Eq. 1–3), and its reference should be added, similar for Fig. 1
Questions for authors

How do you attribute the robustness of your algorithm under occlusion? the effects of foreground mask and sensitivity to fine-grained geometry under occlusion are evaluated thoroughly in this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.