- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces StructVista, a benchmark for reconstructing molecular structures from scientific figures. It contains 419 expert-verified figures and 3,412 compound–structure pairs, divided into three levels: local structure recognition, global reconstruction with explicit assembly relations, and context-dependent reconstruction requiring implicit assembly reasoning. The authors evaluate multimodal language models and tool-augmented agents, showing that current systems struggle with complete figure-level reconstruction, particularly when structural information must be integrated across different regions.
Critical strengths
Useful benchmark construction. The dataset is carefully curated and expert-verified, with a meaningful distinction between local recognition, explicit structural instantiation, and context-dependent assembly.
Systematic evaluation. The evaluation of direct multimodal models and tool-augmented agents provides useful evidence about current limitations in extracting complete molecular structures from scientific literature. The authors also investigate the contributions of recognition errors and target multiplicity.
Critical weaknesses
Limited demonstrated scientific significance. While extracting molecular structures from scientific figures is practically useful, the paper does not convincingly establish why this particular benchmark represents a substantial advance for machine learning research. The contribution is primarily dataset construction and evaluation of existing models, with limited methodological or conceptual insights beyond identifying another challenging extraction setting. The paper would benefit from a stronger argument for the broader significance of this task beyond literature digitization.
Insufficient formalization of molecular assembly reasoning. The central distinction between L2 and L3 relies on whether assembly relations are explicitly specified or inferred from context, but these relations are described largely in natural language. A more rigorous formulation in graph or grammar notation could characterize the structural operations involved, such as fragment substitution, attachment-site matching, and graph fusion. Relevant precedents include Representing Molecules as Random Walks Over Interpretable Grammars (ICML 2024) and Foundation Molecular Grammar (ICML 2025), which demonstrate how molecular composition can be represented through interpretable graph grammars and how foundation models can interact with such representations. The paper should clarify why it evaluates only final SMILES reconstruction rather than explicitly representing intermediate assembly operations. Without this formalization, it is difficult to isolate failures of molecular assembly reasoning from bottlenecks in representation/generation.
Questions for authors
-
What broader ML research questions does StructVista enable beyond evaluating molecular structure extraction from scientific figures? How does the benchmark's difficulty translate into scientific significance or methodological insights?
-
Can the authors formalize L3 assembly reasoning using graph operations or a molecular grammar, explicitly defining the relations and transformations that must be inferred? Why is the task formulated solely around final SMILES outputs rather than an intermediate graph-based assembly representation? Would evaluating foundation models with explicit molecular grammar representations, as explored in the ICML 2024 and 2025 works mentioned above, provide a more direct assessment of their assembly reasoning capabilities?
5 comments
Response to Reviewer dETp (2/2)
W2 and Q2: Graph formalization and intermediate assembly representations
We have added a graph-based formulation in Appendix H, Equations (2)–(5). For figure and target , reconstruction is expressed as
where contains component graphs and templates grounded in the figure, and specifies their roles, replacement domains, attachment correspondences, and shared interfaces. Role/reference selection determines which components are used and what they represent. The graph transformations are:
1. Subgraph replacement: remove a region from reference graph , insert replacement graph , and reconnect the retained structure through boundary bonds :
= \bigl(G[V(G)\setminus U]\sqcup H\bigr)+B_\beta.$$ **2. Attachment:** add specified labeled bonds $B$ between selected atoms of component union $K$, including ring closure: $$\operatorname{Attach}(K;B)=(V(K),E(K)\cup B).$$ **3. Shared-interface fusion:** identify atoms or edges that depict the same molecular entities through a compatible correspondence $\phi$: $$\operatorname{Fuse}(G,H;\phi)=(G\sqcup H)/{\sim_\phi}.$$ Here $\sqcup$ denotes disjoint union; the operations preserve the required chemical labels and stereochemical constraints. This makes the L2/L3 distinction precise: both can use the same operations, but L2 supplies their arguments explicitly, whereas L3 requires at least one argument to be inferred from figure context. **Final structures and intermediate evaluation.** SMILES provides a common output interface. Scoring compares parsed, standardized molecular structures and their identifiers, allowing different valid SMILES, decompositions, and construction orders. This supports comparisons among direct prediction, programmatic assembly, and grammar-based approaches. Intermediate representations would provide complementary evidence about which structural decisions a model gets right. We have incorporated the suggested [ICML 2024 grammar work](https://proceedings.mlr.press/v235/sun24c.html) and [Foundation Molecular Grammar](https://proceedings.mlr.press/v267/sun25aa.html). Their compositional representations motivate an extension in which a model receives reference component graphs, predicts $z_t$, and a deterministic assembler produces the final graph. Evaluating both the relations and the assembled graph would reduce recognition and SMILES-generation burdens and more directly assess assembly decisions. Scoring would accommodate atom renumbering, symmetry, and alternative valid programs. The additional challenge in StructVista is grounding the components and operation arguments in visual evidence. Appendix H connects this task to molecular grammars and specifies the proposed intermediate diagnostic; the completed matched-presentation experiment provides complementary end-to-end evidence about the effect of implicit relationships. Overall, we thank the reviewer for the constructive feedback and helpful suggestions. We would be happy to address any remaining questions or concerns.Reply to Authors
Thanks for the additional experiments and formulation. Your proposed extension to have the model predict the assembly program is the missing L3 result. All baselines clearly struggle with L3 using SMILES, but without the graph formalization we cannot conclude if their failures are due to the implicit information, or the ill representation. Now that you have the formalization, please repeat the L3 experiments to see whether your conclusions still hold.
Response to Reviewer dETp (1/2)
We thank the reviewer for the constructive suggestions on significance and formalization. We have clarified how StructVista supports research on native molecular understanding, using controlled experiments to connect its difficulty to model capabilities. We also formalize assembly as graph operations and explain how intermediate representations can complement final-graph evaluation. We welcome any further questions or suggestions.
The broader objective of StructVista is to study native molecular understanding: whether a multimodal model can interpret molecular entities and their relationships from visual evidence and recover the structure they jointly specify. This connects local perception to compositional reasoning. A model must determine what each component represents, what is inherited or replaced, and which interfaces correspond. Molecular graphs make the outcome of these decisions precisely testable.
To establish why this capability merits separate study, we added an experiment that fixes molecular identity, molecular size, and target count while varying how the same structures are presented. For 1,379 targets from 105 figures, we constructed complete L1 depictions, explicit L2 decompositions, and L3 depictions retaining context-dependent relationships. Direct GPT-6 Sol, evaluated with the same prompt and configuration and one output per condition, obtains 55.53, 54.32, and 34.35 F1, respectively. L3 is 20.67 points below L2 (95% paired figure-bootstrap interval [9.20, 32.05]). Thus, the underlying molecules alone cannot explain the difficulty: performance also depends on how structural information and relationships are expressed.
A second experiment examines how this capability interacts with external tools. On all 419 original figures, removing the chemistry toolkit while retaining the Codex backbone, code execution, and iteration reduces overall F1 from 83.77 to 78.98. Tools provide clear gains on L1 and L2, while the L3 comparison does not resolve a benefit. This suggests that access to recognition and validation tools does not automatically yield reliable reconstruction from implicit relationships.
Together, these results motivate models that develop native molecular interpretation alongside tool use, so that they can determine the structural decisions a tool must execute and assess whether the resulting graph matches the figure. StructVista provides an evaluation setting for studying this interaction between perception, compositional interpretation, and tool-assisted construction. We have strengthened this motivation in Sections 1 and 5.4 and report the experiments in Appendix L.1–L.2, Tables 18–19.