- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Struct2Real, a system capable of generating object-centric images conditioned on the 2D rendering of a sketchy 3D mesh created with simple primitives, and further supports viewpoint and structural editing. The authors claim 2 main contributions: 1) a structure modeling system that enables users to create a 3D structural representation named StructMap, 2) a Structure-Aware Generation Agent (SAGA), an agent that turns the StructMap into images with skills to enhance details and realism. The authors compared the proposed system to other condition variants such as text, line art, and sketches, showing better results.
Critical strengths
- The idea of using the rendered image of 3D geometric shapes (the so-called StructMap) as conditions makes sense, and the generation quality indeed looks better than the other condition candidates presented in this paper. While the advantage over lineart is not that clear from the numbers and visualizations, I agree that creating the StructMap is a simpler task for ordinary people.
- The results in the editing experiments shown in fig 5 are pretty promising, achieving better consistency and ease of control compared to baselines. This system is more suitable for multi-round interaction between a human artist and the image generation system, compared to representations such as linearts.
- The paper is well written and easy to follow. The authors also provide sufficient implementation details (e.g., the prompts) for reproducibility.
Critical weaknesses
I think the system presented in the paper is interesting, but the missing details specified below make it hard to make a fair judgment on the soundness of the claims made in this paper, including both the generation quality and the claimed ease of use.
- The authors list the structure modeling system as a core contribution, but it is unclear that the system has advantages over existing software such as Blender. From my understanding, the main selling point of the proposed system is some pre-built primitives (e.g., a set of 4 rectangles that can be used as chair legs). The problem with this is that the pre-defined degrees of freedom are likely not enough when the user starts to create complex objects, where they have to fall back to basic primitives. To make the point stand, the authors should make a systematic study of asset creation time using the proposed systems, compared to software such as Blender and CAD, across objects of different complexity. A good example of a complex object is the desk in cell (e) in fig 5.
- The main evaluation dataset used in this paper is constructed by the authors, and many details are unclear. For example, basic statistics such as the total number of scenes and category distributions; is there a reference image given when the human constructs the structure priors? If so, the authors should show the reference image so we can examine the faithfulness of the reconstruction. If not, how are the SSIM and LPIPS computed?
- I'm not an expert on image generation, so my confidence is low on this. But from what I see in the related works section, the methods discussed there are mostly from 2023 and 2024, which are pretty old for such a fast-evolving research area. The same holds for the baselines compared; I'm not sure if ControlNet is still a relevant baseline as of 2026. The authors should clarify the reason for not comparing with more recent baselines.
- More details are needed for the part involving humans. How many human artists are hired, and what is their level of expertise in creating the different forms of art? This is important to ensure the creation time and difficulty in tab1 are meaningful. For the human study on MOS-R and MOS-A, how many human subjects are involved?
Questions for authors
N/A