- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper places generated 3D objects back into a scene. It takes a generated mesh and a partial point cloud of the same object. Instead of regressing the pose, it predicts which part of the mesh is visible in the observation. It then aligns this visible part to the observed points to get the pose. The model builds on a pretrained shape VAE and is trained only on object meshes with rendered depth. On 3D-FUTURE, it beats SAM3D, Any6D and 3DFixer under both ground truth and estimated depth.
Critical strengths
The method leaves the object generator untouched. Shape quality is therefore not sacrificed for pose accuracy, and the module can be paired with any generator or depth model. Predicting the visible part of the mesh is a simple and natural way to frame the problem. It replaces full-to-partial alignment with partial-to-partial alignment, and training needs only object-level data rather than scene-level pose labels. Reusing a pretrained shape VAE is a sensible choice. Freezing the mesh encoder and letting the observation modulate it are both supported by the ablations, and the gains over the baselines are large and consistent.
Critical weaknesses
-
Predicting visibility from an unaligned point cloud already requires the network to infer the relative pose. The output is a mask rather than point matches, so the final pose still depends on a Chamfer fit that needs a good initialization. In this sense, the hard part is not removed. The idea is also close to overlap prediction in point cloud registration. Furthermore, the paper does not explain how the pose solver is initialized or how scale is recovered.
-
The training input is always a clean subset of the same mesh, with no shape gap, depth noise or occlusion. The model is thus never trained on the very difficulty that motivates it. In addition, each object is placed independently, so the scene-level claim involves no reasoning about support or collision.
Questions for authors
-
How is Eq. 4 solved? How do you initialize it and recover scale? How much better is the result if the solver gets the ground truth visible region?
-
In real scenes the viewing direction is known, and it largely decides what is visible. Why is it removed during training?
-
How does accuracy change as the generated mesh drifts away from the real shape?