- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RM3D is a retrieval-and-morphing pipeline for single-image 3D generation using frozen OpenShape, DINOv2, and TRELLIS models. It retrieves an asset from a 23,835-object database and uses its cached SLAT as a source reference, while the target endpoint is generated from the input image. The method combines condition-token alignment, SLAT-guided structural matching, correspondence-matched attention, and target-only attention in late blocks. It requires neither task-specific training nor per-instance optimization. Experiments on Toys4K report improvements over TRELLIS-S and other generators, cumulative component ablations, and a database-size scaling study.
Critical strengths
-
Using retrieved 3D assets to supplement a frozen generator is a useful direction. The method incorporates external references without task-specific training or per-instance optimization.
-
The distinction between borrowing source structure and preserving target appearance is well motivated. The correspondence and target-focused attention designs offer a practical way to address the mismatch between a retrieved asset and the input image.
Critical weaknesses
-
Configuration A omits the semantic, structural, and texture alignment already available in full Interp3D. Since configuration C introduces related alignment mechanisms, its improvement over A may partly reflect recovering capabilities removed from the baseline. A comparison with full Interp3D under the same retrieval setup is needed to establish the benefit of RM3D's modifications.
-
The comparisons omit more recent methods such as Direct3D-S2 and TRELLIS.2.
-
The objects selected for the qualitative comparisons are quite simple, and recent 3D generators (TRELLIS.2, Pixel3D, etc.) can already produce reasonable results for such objects. The real challenge lies in complex or visually ambiguous objects. The paper should show cases where existing methods produce implausible details and RM3D makes them more plausible, but such examples are not presented.
-
The three-pass sampler and target generation add computation, but the paper gives no runtime comparison with TRELLIS. Figure 6 reports average gains from a larger database, without showing how many of the 200 test objects improve or get worse.
-
Some descriptions are inconsistent across the paper: (1) Figure 2 suggests multi-view source conditioning, but Section 3.2 specifies cached view-059 tokens. (2) Section 4.2 describes native 3DGS rendering, although Hunyuan3D-2.1 outputs meshes; how these meshes are converted or rendered is not explained.
-
Eq. (6) jointly reorders target K/V before concatenation, which leaves standard unmasked attention unchanged. I cannot tell how the correspondence in Eq. (4) affects structure generation. Any positional reassignment or masking that makes this operation effective needs to be stated explicitly.
Questions for authors
The following questions correspond, in order, to the critical weaknesses above.
-
Using the same retrieved assets, does RM3D still outperform Interp3D when its original semantic, structural, and texture alignment are all retained?
-
Could you compare RM3D with Direct3D-S2 and TRELLIS.2 using the same input images? How do their geometry and texture quality compare?
-
Could you show complex or visually ambiguous objects where TRELLIS produces missing parts or incorrect connections, and RM3D corrects them? Please show the retrieved asset and close-ups of the same details in both outputs.
-
How long does RM3D take compared with TRELLIS on the same GPU? In Figure 6, does a larger database help most test objects, or do some results get worse?
-
Does source conditioning use multiple views or only view-059? Please clarify any view aggregation and how mesh-based outputs are rendered or converted for comparison with native 3DGS.
-
What positional or masking operation makes Eq. (6) sensitive to the K/V permutation? Please specify it and compare outputs with and without reordering while holding other inputs fixed.