- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a strong 3D asset reconstruction method that jointly performs 3D reconstruction and generation from sparse, unposed multi-view images. Specifically, it unify reconstruction and generation in a shared anchor-guided coordinate space, enabling bidirectional feature exchange between a VGGT-based reconstruction branch and a TRELLIS.2-based 3D generator. Experiments show state-of-the-art 3D object reconstruction quality.
Critical strengths
This paper aims at the timely 3D object reconstruction/generation subject and achieves state-of-the-art results in geometry and appearance fidelity. The difference of relative camera space and canonical object space is a long lasting issue when unifying 3D reconstruction and generation, which is addressed here using anchor-guided shared space. The idea is novel and results are good, which might be a good foundation work if the qualitative examples are not extremely selected cases.
Critical weaknesses
The method currently operates on synthetic renderings, it may face significant degradation when it comes to real-world unposed images (with background filtered).
Questions for authors
I would like to know how is the training views prepared, especially the sample distribution. Does each object randomly sample from the same distribution, or from a larger range.
2 comments
Thanks for the additional information. I have no major concerns thus I will retain the positive score.
We thank the reviewer for recognizing the shared-space formulation and the strong geometry and appearance results.
Generalization to real-world images.
Our method generalizes to unposed real-world images, with results presented in Fig. 5 and the supplementary video.
Training view preparation.
For each object, we randomly sample two azimuth angles and rotate the asset about its vertical axis to each corresponding orientation. For each rotated asset, we render six candidate anchor views and 48 images from randomly sampled viewpoints. Each training sample uses one randomly selected anchor view and three additional views.
Training augmentations.
To simulate variations in real-world image capture and background removal, we apply the following augmentations:
Rotation, scaling, and translation. To simulate camera roll and changes in framing, we apply random in-plane rotation, scaling, and translation about the foreground center.
Lighting and exposure perturbations. To simulate different lighting and exposure conditions, we randomly adjust white balance, brightness, contrast, saturation, gamma, and black and white clipping levels.
Lens distortion. To simulate deformation introduced by real camera lenses, we apply radial and tangential distortion using randomly sampled coefficients. Views within the same sample share these coefficients.
Image-quality degradation. To simulate reduced sharpness, compression artifacts, and camera noise, we apply Gaussian blur, downsampling followed by upsampling, JPEG compression, and additive Gaussian noise. Under regular augmentation settings, the blur standard deviation is 0.4–1.4 pixels, downsampling factors are 0.55–0.90, JPEG quality is 55–95, and noise standard deviation is 0.003–0.015.
Foreground boundary perturbations. To simulate object boundaries being mistakenly removed or background regions being incompletely removed, we randomly erode or dilate foreground masks. We then soften mask boundaries using Gaussian filtering and extend foreground colors outward, creating color fringes around the boundaries to simulate imperfect foreground extraction.
These perturbations are randomly combined at different strengths. After augmentation, images are cropped around the foreground and resized to the network input resolutions.