- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a systematic approach for simultaneous scene layout and object estimation. The contributions lie in three folds: (1) the pipeline that parses the scene into object poses and instances and then feeds the features into a latent diffusion network for object reconstruction. (2) an HC-VAE that compacts the latents without loss of quality. (3) a comprehensive dataset for training such a network.
Experiments show the approach's effectiveness in both versatility and accuracy.
Critical strengths
-
A systematic contribution that leads to solid progress toward automatic composite scene reconstruction.
-
A compact latent that reduces computation, with potentially broader applications in generative 3D methods.
-
A dataset for following works to build on, suppose it will be released.
Critical weaknesses
I do not see a "critical weakness" but do have concerns about the design choice.
-
The model adopts a two-stage approach where object proposals with lower confidence will be filtered out before 3D reconstruction. Will some critical but difficult-to-localize objects be omitted? What if you adopt an end-to-end approach where the object features are directly sent as the condition?
-
The technical pipeline is a composition of a DETR and a TRELLIS-like model, which itself is relatively incremental.
Questions for authors
-
It will be nice to see how performance varies with different compression rates in Table A6.
-
As HC-VAE is trained from scratch on top of SC-VAE, how much data is minimally required to match the performance of SC-VAE as indicated in Table A6?
-
There is a lack of a real-world dataset for 3D reconstruction quantitative comparison. The comparison under ScanNet++ is suggested.
-
Some related research works on amodal scene reconstruction should be discussed:
World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible
LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning
NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction
VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching
Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning