- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a multimodal inverse design framework for 3D microstructure synthesis, capable of generating designs based on textual descriptions and hand-drawn sketches. The workflow integrates a VQ-VAE quantizes voxel microstructures into 512 structural tokens for structural discretization, GPT-2 for generating structure tokens, and a Property-Guided Adaptation module for performance optimization.
Critical strengths
1.This paper combines textual descriptions with hand-drawn three-view sketches, offering non-expert users an intuitive and flexible approach to microstructure design, treating sketches as optional controls rather than mandatory inputs is reasonable from an interaction design perspective.
2.This paper constructs the large-scale GenText-Microstruct dataset comprising 14,000 samples and establishes a multidimensional evaluation framework incorporating FID, CLIP score, Chamfer Distance and , providing a reference benchmark for research in this field. The authors clearly state that established benchmarks are lacking in this direction, and therefore this contribution has its own value.
3.Some reports are transparent. The author fairly discussed the inference cost and the "not strictly fair" compared to 3D-CGAN, as well as the potential losses caused by attribute optimization, thereby enhancing the credibility of the article.
Critical weaknesses
1.The significant performance drop in the "w/o Sketch" variant compared to the baseline suggests that the MLLM’s structural guidance is weaker than the baseline’s latent diffusion. Under the same evaluation criteria, pure-text LLMs without sketches underperform earlier baselines on both datasets: on Geometries 2000, FID is 97.68 vs. 77.80 for Txt2Microstruct-Net; on GenText, FID 114.17 / CD 0.1355 vs. diffusion baseline FID 84.46 / CD 0.0666. Only when sketches are included does the model surpass them (65.55, 55.47). An explanation for this performance gap is required.
2.In Table 1, several sub-optimal metrics are bolded, which undermines the clarity of the performance hierarchy. Why 3D-CGANtrained on a single class of Voronoi-based open-cell foams is the reason of lower FID? T2M-LLM's CD of 0.1352 is worse than Txt2Microstruct-Net’s 0.1162, and its Young’s modulus of 0.713 is also lower than 0.757,both values are still bolded.
3.The manuscript relies exclusively on quantitative metrics. Lack qualitative comparison with competing methods. Figures 6, 7, and 10 show only side-by-side generation and optimization results from T2M-LLM, with no visual comparisons against Txt2Microstruct-Net, 3D-CGAN, or diffusion baselines.
4.The core technical approach of the paper follows relatively mature methods in the cross-modal generation domain (e.g., Text-to-Image/3D), lacking significant innovation. The combination of VQ-VAE, expanded vocabulary, LoRA, and latent optimization at test time is a well-established approach in cross-modal generation. Although the authors applied the aforementioned approach to materials science, they did not propose fundamental modeling innovations specific to structures in this field, leaning more towards an engineering integration of existing techniques.
Questions for authors
-
In the plain text setting, this method is on par or slightly superior in terms of Accuracy (0.8600 vs 0.8200) and CLIP (0.5269 vs 0.5238), but significantly lags behind in FID and CD. Do you agree that a more accurate statement would be "LLM mainly contributes semantic and category alignment rather than geometric fidelity"? If so, do the wording "Significant improvement in generation quality" in the main text and abstract need to be narrowed?
-
If the bottleneck of generation quality is "insufficient semantic understanding", then giving the model more semantic capabilities (switching to LLM) should lead to an improvement in text-only mode, the results shows the opposite. The assumption seems not to hold.
-
Please explain the bolding rule for the table. Is it the optimal performance value that is bolded?
4 comments
Weakness 4: The methodological novelty mainly lies in combining existing modules, with limited material-specific modeling innovations.
Response: We appreciate the reviewer's comments and would like to further clarify the scope and contributions of our work. This study explores a multimodal conditional generation paradigm for 3D metamaterial microstructure design, aiming to provide a natural-language-driven approach to 3D microstructure generation. Rather than introducing an entirely new generative architecture, our contributions lie in the integration and adaptation of language modeling, multimodal conditioning, and property-guided optimization for this task. Specifically, they comprise three aspects:
-
Discrete Microstructure Representation and Autoregressive Generation: We employ VQ-VAE to encode 3D voxel-based microstructures into discrete tokens, enabling a pretrained language model to learn the correspondence between textual descriptions and microstructure representations through autoregressive sequence modeling.
-
Multimodal Conditional Control: We integrate textual descriptions and multi-view sketches into a unified generation framework, allowing the model to leverage both semantic information and spatial-structural priors. This helps alleviate structural ambiguity inherent in text-only generation.
-
Property-Guided Latent Space Optimization: During inference, we leverage a learned property predictor to optimize the latent representation of generated structures, enabling controllable generation toward target physical properties.
We acknowledge that material-specific constraints, such as periodicity, connectivity, and manufacturability, are important for practical metamaterial design. However, these constraints are not explicitly implemented as mechanisms in the current framework, and we therefore do not claim them as established technical contributions. We have also strengthened the related-work discussion to better contextualize our approach with respect to existing LLM-based generation methods.
Thanks for the authors' response. I appreciate the effort made to help me better understand the paper.
That said, I still find the novelty of the proposed method limited, and the baselines somewhat outdated. However, I consider the additional ablation experiments provided by the authors to be meaningful, and I recommend including them in the paper or the supplementary material. I also suggest that the authors explain what the bold font denotes in the paper, since its current usage may not be easy to interpret.
I have also seen the qualitative experiments added in the revision. In the first row ("Microstructure with finely spaced perforations. It has pentagon pores."), the proposed method does show a clear improvement over Propdiff-TMG.
For these reasons, I am raising my score from 2 to 3. However, I would like to note that, in my view, this paper remains borderline. If the AC or the other reviewers lean toward rejection, I will have no objection.
Thank you for your careful consideration of our response and for raising your rating. We appreciate your recognition of our ablation experiments experiments and are glad that they have addressed most of your concerns. Thank you again for your time and constructive feedback.
Weakness 1 & Question1 & Question 2: The main performance improvements appear to stem primarily from the sketch modality, while evidence supporting the necessity of the LLM remains insufficient.
Response: First, to better characterize the performance of text-only generation, we compare T2M-LLM with the text-based baseline on Geometries-2000. As shown in Table 1, T2M-LLM achieves higher Accuracy (0.8600 vs. 0.8200) and CLIP (0.5269 vs. 0.5238) than Txt2Microstruct-Net. However, its FID is higher (97.68 vs. 77.80). These results suggest that the text-only model provides certain advantages in semantic category alignment, while its generative distribution quality remains limited.
Table 1. Comparison between T2M-LLM and the text-based baseline under text-only conditioning on Geometries-2000.
Second, to investigate the contribution of sketch conditioning, we compare the model's performance before and after incorporating sketches. As shown in Table 2, adding sketch conditions substantially improves generation performance on Geometries-2000, particularly in terms of distribution quality and semantic alignment. This indicates that sketches provide complementary spatial and geometric priors that are difficult to capture through textual descriptions alone.
Table 2. Ablation study of sketch and property-guided adaptation on Geometries-2000.
Furthermore, as shown in Table 3, removing sketch conditioning from T2M-LLM on GenText-Microstruct leads to a substantial deterioration in all three metrics: FID increases from 55.47 to 114.17, CLIP decreases from 0.4200 to 0.3856, and CD increases from 0.0589 to 0.1355. This further demonstrates the important role of sketch conditions in improving the quality and geometric fidelity of generated structures.
Table 3. Comparison between the full model and its sketch ablation on GenText-Microstruct.
Taken together, these results suggest that the different components play complementary roles: the pretrained language model establishes semantic mappings between textual descriptions and structural tokens; sketch conditioning provides spatial and geometric priors; and property-guided adaptation further refines the generated structures toward the target properties. Importantly, we do not attribute the overall performance gains solely to the LLM backbone. Rather, our contribution lies in integrating linguistic and visual information within a unified token-based generation framework.
Finally, we acknowledge the reviewer's suggestion to include a diffusion-based baseline under the same Text+Sketch conditioning setting. Accordingly, we provide an additional comparison with PropDiff-TMG (2026). As shown in Table 4, T2M-LLM achieves better Accuracy, FID, and CLIP scores, while PropDiff-TMG performs better in terms of CD. These results further support the advantages of T2M-LLM in semantic alignment and distribution-level generation quality, while indicating that geometric fidelity remains an area for further improvement.
Table 4. Comparison between T2M-LLM and PropDiff-TMG under Text+Sketch conditioning.
Weakness 2 & Question 3: The meaning of bolding
Response: We have bolded the optimal results and the experimental results of the methods presented in this paper. Specifically, if there are two bolded entries in the same column, it indicates that our results are not the optimal ones.
Weakness 3: The qualitative results lack visual comparisons with other methods, particularly in cases where geometric fidelity metrics are poor.
Response: We thank the reviewer for pointing out this deficiency in our qualitative experiments. We agree that presenting only the results generated by our own method is insufficient to highlight the differences between various generative approaches regarding the geometric morphology and structural details of microstructures. We have included additional qualitative comparative experiments in the revised paper.