- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes a relatively novel idea: learning reaction mechanisms from images rather than SMILES or molecular graphs. The authors develop MechaVLM to implement this idea and demonstrate the advantages of visual inputs for learning reaction mechanisms and generalizing the acquired knowledge to OOD data. Through detailed experiments, the paper demonstrates the effectiveness of the proposed method and its individual components.
Critical strengths
- The idea of learning reaction mechanisms from images is relatively novel. The training tasks for the vision encoder and the designs for precedent retrieval and utilization offer valuable insights.
- Extensive experiments demonstrate MechaVLM’s superior generalization capability in reaction mechanism prediction relative to the baselines. The paper also investigates and analyzes the contributions of its individual components.
- The experiments further demonstrate that the proposed training approach of the vision encoder benefits other related tasks.
Critical weaknesses
I have concerns regarding the setup and implementation of some experiments, which leave me uncertain about the soundness of the corresponding conclusions. Please see the questions in "Questions For Authors" for details.
Questions for authors
- Atom types in molecules exhibit a highly imbalanced, long-tailed distribution, with neutral carbon atoms accounting for the vast majority. How do you address this imbalance during your training of the vision encoder? Similarly, the appendix states that bonded and non-bonded atom pairs are sampled "at approximately equal frequency". However, among bonded pairs, single bonds account for the vast majority. Do you take any further measures to address this imbalance?
- What is the source of the molecular data used to train the vision encoder, and how exactly are different visual forms of molecular images under different rendering conditions generated? These details are not clearly explained in the method section.
- Considering that MechaVLM is a task-specific model and that this work places particular emphasis on OOD generalization, a fair comparison requires the baseline models to undergo corresponding task-specific fine-tuning as well. In the experiments reported in Tables 1 and 2, were all baseline models evaluated after additional task-specific fine-tuning on the corresponding source dataset (FlowER or ReactMech)? Why are results for some models, such as ChemVLR and DeepMech, reported for only one of the two source datasets?
- Similarly, did the VLMs undergo task-specific fine-tuning for the experiments in Tables 3 and 4?
- For the experiments in Table 5, how exactly were the SMILES-input and graph-input variants implemented, and did they undergo task-specific fine-tuning? These details are critical to proving whether learning from images provides stronger generalization. Furthermore, if comparable precedent retrieval were incorporated into the SMILES-input and graph-input variants, could they achieve similar improvements in generalization?
- Purely out of curiosity: what do you see as the value of reaction mechanism prediction for AI for chemistry? Given that current OOD performance remains far from practically usable, how could this capability be applied in concrete chemical scenarios? Alternatively, what benefits or insights could it offer for models addressing specific practical chemical tasks or problems?
2 comments
Thank you for your positive follow-up and kind words. We sincerely appreciate your thoughtful questions and constructive feedback. Your suggestions have helped us improve the clarity and rigor of our work, and we are grateful for your careful consideration.
Thanks for the detailed response. Your explanations have addressed my questions about the experimental setup as well as those raised out of curiosity. I have therefore decided to raise my soundness score to 3. I appreciate your rigorous work and inspiring insights.