- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents Cog2Gen3D, a latent diffusion model for 3D Gaussian generation conditioned on a learned “3D cognition graph”. The 3D cognition graph is formed by pre-trained ResNet50 (semantic tokens), VGGT (geometric tokens), and CLIP (logical tokens). The semantic and geometric tokens form two separate latent graphs which are then fused via cross-attention with the CLIP tokens acting as queries. The denoiser is conditioned on this fused representation and during training the diffusion, node classification and multi-view rendering losses are all combined. CogSG-3D is constructed from publicly available 3D datasets as well as Marble generated worlds.
Critical strengths
- Table 6 compares ResNet50, CLIP ViT-L, and VGGT as geometric encoders, with VGGT giving the best 3D-Front results. This provides a useful empirical result for conditioning 3D generation on a geometry foundation model.
- The evaluation covers text-to-3D generation and image-to-3D generation of objects and scenes. The ablations cover graph construction, encoder choice, fusion, positional encoding, node count, inference steps, and loss weights.
- Table 11 reports inference time and memory use for the method and baselines, providing useful information about practical cost.
Critical weaknesses
- I found it difficult to reconcile the latent-graph conditioning in the main text (Eqs. 5–8) with the warped frames and masks described in Appendix A.2. The connection between these representations is not explained, and the appendix’s temporal-modeling claims contradict Sec. 5.4 and Appendix B.6. The text-only path is also unclear: the semantic and geometric branches require images, but their inputs for T3Bench are unspecified. There are also no experiments varying the graph or comparing multiple outputs for a fixed input to demonstrate graph-level control and sampling diversity.
- VGGT is trained at a normalized canonical scene scale, but the paper does not explain how its features or the learned positional embeddings establish calibrated physical scale. Tables 6 and 8 demonstrate benefits from geometric features and positional encoding, respectively, but do not directly validate scale recovery or consistency. Describing EchoScene and Layout2Scene as limited to “2D relative” geometry is also inaccurate, since both explicitly use 3D geometry.
- Table 2’s OmniObject3D FID/MMD values match DiffTF’s class-conditional results, but it is unclear how these are comparable to image-conditioned generation. Table 1’s combined T3Bench quality/alignment scores are called quality scores, and the DreamFusion row repeats SJC’s surroundings/multi-object scores and overall average. The procedure used to obtain the proposed method's scores is unspecified. Table 3 does not explain image-to-graph/layout conversion for EchoScene and Layout2Scene or alignment/cropping for LucidDreamer. Using LucidDreamer as an object baseline in Fig. 15 also needs justification. Missing comparisons include TRELLIS[1], Hunyuan3D 2.0[2] and TripoSG[3] for objects; GALA3D[4], DecompDreamer[5] and LayoutDreamer[6] for composition; and DreamMesh[7] and VP3D[8] on T3Bench.
- The qualitative comparisons show one rendering per output, and the supplementary video pans around the frontal view without revealing the objects’ sides or backs. This makes it difficult to assess the claimed structural quality and multi-view consistency, particularly on surfaces unseen in single-image inputs. The views used for Table 2’s image metrics are also unspecified.
References:
[1] Xiang, Jianfeng, et al. “Structured 3D Latents for Scalable and Versatile 3D Generation.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 21469–21480.
[2] Zhao, Zibo, et al. “Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation.” arXiv, 2025, arXiv:2501.12202.
[3] Li, Yangguang, et al. “TripoSG: High-Fidelity 3D Shape Synthesis Using Large-Scale Rectified Flow Models.” arXiv, 2025, arXiv:2502.06608.
[4] Zhou, Xiaoyu, et al. “GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-Guided Generative Gaussian Splatting.” Proceedings of the 41st International Conference on Machine Learning, PMLR, vol. 235, 2024, pp. 62108–62118.
[5] Nath, Utkarsh, et al. “DecompDreamer: A Composition-Aware Curriculum for Structured 3D Asset Generation.” Transactions on Machine Learning Research, 2026.
[6] Zhou, Yang, et al. “LayoutDreamer: Physics-Guided Layout for Text-to-3D Compositional Scene Generation.” Pattern Recognition, vol. 178, 2026, article 113427.
[7] Yang, Haibo, et al. “DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation.” Computer Vision – ECCV 2024, Lecture Notes in Computer Science, vol. 15117, Springer, 2025, pp. 162–178.
[8] Chen, Yang, et al. “VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generation.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4896–4905.
Questions for authors
Refer to Weaknesses