- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses the problem of subjective and objective quality assessment for Text-to-3D generated content (3DGCs), where generative models are referred to as "Dreamers". The authors curate 484 prompts across 10 semantic categories and generate 4,691 valid 3D meshes using 11 Text-to-3D models, including both open-source and closed-source models. To enable and accelerate automated 3D generation assessment, the authors propose Hyper-VG, a quality prediction model that leverages prior information from the Huanyuan world model and persistent homology to estimate different quality measurements.
Critical strengths
- The authors construct a large-scale dataset of generated 3D content from both open-source and closed-source models, which is labeled by humans rather than non-3D knowledge injection Large Language Model, compared to previous research.
- The quality assessment is detailed with 9 quality issues, which provides a comprehensive analysis of the factors that influence 3D generation content quality.
- The writing is easy to follow, even though there are some issues with table references.
Critical weaknesses
Technical Approach:
- I think the current dataset, collected and labeled in this research, has little impact on future development of 3D generation since the assets are generated by current and some outdated methods. As a result, to evaluate the quality of newer methods, we have to sample more generated content and rerun the whole protocol.
- The use of persistent homology is not a strong feature since all the open-source tools used in the research leverage Marching Cubes [1] to extract an iso-level surface from the volume, which is not topology-aware, while closed-source methods like Tripo might have their own retopology module, which is a precondition for UV-texture unwarping; therefore, there might be a mismatch in quality assessment.
Experiments:
- The ablation study in Table 3 is confusing. The authors should list feature combinations as checkboxes.
- The authors need to include newer Text-to-3D methods [2][3][4][5] as well.
Writing:
- The term "homotopy" used in the paper should be "homology". The term "homotopy" is used vastly in the paper, causing critical confusion for the readers.
- There is a minor table link reference on line 130.
[1] Marching cubes: A high-resolution 3D surface construction algorithm
[2] ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation
[3] JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillation
[4] Structured 3D Latents for Scalable and Versatile 3D Generation
[5] Consistent Flow Distillation for Text-to-3D Generation
[6] Text-to-3D Generation using Jensen-Shannon Score Distillation
Questions for authors
- Can the author provide the protocol for how the participants evaluate or give a score for topology, triangle quality assessment? Does the author show the wireframe of the mesh or directly use some triangle quality metrics?
- Can the author explain why persistent homology can benefit the assessment of the quality of 3D generated assets?