DeformR: Single-View 3D Shape Retrieval with Image-Guided Shape Adaptation

Single-view 3D shape retrieval is a critical enabler for inverse graphics and detailed 3D scene representation, allowing generic 3D assets to serve as proxies for real-world instances. Recent feature-level analysis-by-synthesis approaches have improved robustness to occlusions and domain gaps through pose-conditioned feature alignment at test time. However, a rigid template may not match local geometry visible in an image. To address this mismatch, we propose DeformR, a framework that extends pose-aware 3D retrieval with local, image-guided shape adaptation. First, we formulate a 2D-to-3D feature-distillation and pose-refinement pipeline for shape retrieval and pose estimation. The pipeline supports multiple categories while retaining point-level representations. Second, with the selected template and pose fixed, DeformR is designed for conservative local adaptation rather than free-form reconstruction. Pose-aligned image features guide corrections to image-supported geometric residuals, while geometric regularizers preserve the retrieved template's global structure. Large geometric changes are neither expected nor desired. Experiments on Pascal3D and ImageNet3D show improved retrieval accuracy over the evaluated baselines and competitive pose-estimation performance. With the retrieved template and pose fixed, deformation improves alignment between the real-world image and the deformed shape.