- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents MAELLE, a discrete flow-matching framework for reaction outcome prediction that represents chemical reactions as directional electron-pair edits. An optimal transport (OT) procedure constructs an unordered set of edits from reactant–product pairs, and a continuous-time Markov chain is trained to generate product structures without elementary-step annotations. The paper evaluates the method on USPTO-480K and AbSynth and analyzes agreement between predicted electron flows and expert-annotated mechanisms.
Critical strengths
- The paper proposes an interesting framework that combines directional electron-pair edits with discrete flow matching, enabling reaction prediction without elementary-step annotations.
Critical weaknesses
-
Insufficient baseline comparisons. The paper omits stronger existing methods on USPTO-480K, including NERF* (91.5% Top-1, 95.1% Top-5), ReactionSink (91.3%, 94.5%), Chemformer-Large (92.8% Top-1 under the separated-input setting) [1], and RXNGraphormer[2]. These results exceed MAELLE's 87.2% Top-1 accuracy, raising concerns about its competitiveness.
-
Mismatch between OT and chemical motivation. The OT objective minimizes graph-distance costs rather than chemical plausibility or mechanistic feasibility. The inferred edits are therefore optimal under a heuristic cost, but are not necessarily chemically meaningful. Since these edits define the training targets, this weakens the justification for interpreting generated trajectories as mechanistic electron flows.
[1]: Joonyoung F. Joung, Nicholas Casetti, Priyanka Raghavan, Connor W. Coley; An overview of reaction outcome prediction with physics-based and data-driven methods. Chem. Soc. Rev. 2026; 55 (12): 6768–6813. https://doi.org/10.1039/d6cs00079g [2]: Xu, LC., Tang, MJ., An, J. et al. A unified pre-trained deep learning framework for cross-task reaction performance prediction and synthesis planning. Nat Mach Intell 7, 1561–1571 (2025). https://doi.org/10.1038/s42256-025-01098-4
Questions for authors
-
Why were stronger and more recent baselines, particularly NERF* and ReactionSink, excluded? Can the authors provide comparisons under consistent evaluation protocols?
-
How do the authors justify the chemical validity of OT-inferred electron transfers when the transport cost only reflects graph distances? What shows that these edits correspond to chemically meaningful pathways rather than arbitrary endpoint alignments?
4 comments
Thank you for the rebuttal and for correcting the NERF evaluation. I have two remaining concerns.
- Comparison with stronger baselines. My request was for a targeted comparison with the methods cited in my review that report higher accuracy on USPTO-480K. The corrected NERF results address one part of this concern. A comparison with a stronger baseline such as RXNGraphormer under the same input and evaluation protocol would help establish MAELLE’s competitiveness. Where a matched rerun is infeasible, please include the published results and identify any protocol differences.
- Evidence that graph-distance OT improves OOD prediction. On AbSynth, replacing graph-distance OT with random source–sink assignment changes Top-1 accuracy from 48.2% to 47.5% and Top-5 from 60.7% to 60.0%. The directional-agreement analysis supports graph distance as a locality prior. However, for OOD product prediction, the small accuracy gaps leave its contribution unclear. This weakens the empirical motivation for the graph-distance OT component as a way to improve generalization. For these reasons, I am keeping my original rating of 1.
About RXNGraphormer and graph-distance OT
Inclusion of RXNGraphormer into the baselines:
It is fortunate that RXNGraphormer comes with open sourced code and model weights, so it was quick to test the model on AbSynth. We would hereby include RXNGraphormer into our main table:
Top-k accuracy on USPTO-480K and AbSynth, mixed setting, best in bold, second best in italic
| Model | Prediction modality | USPTO Top-1 | USPTO Top-3 | USPTO Top-5 | AbSynth Top-1 | AbSynth Top-3 | AbSynth Top-5 |
|---|---|---|---|---|---|---|---|
| Molecular Transformer | SMILES | 0.886* | 0.935* | 0.942* | 0.354 | 0.419 | 0.441 |
| Graph2SMILES | SMILES | 0.903* | 0.940* | 0.948* | 0.395 | 0.452 | 0.471 |
| RXNGraphormer | SMILES | 0.906* | 0.940* | 0.949* | 0.410 | 0.471 | 0.491 |
| MEGAN | Graph edits | 0.863* | 0.924* | 0.940* | 0.457 | 0.561 | 0.595 |
| NERF | Electron diff | 0.898 | 0.913 | 0.916 | 0.395 | 0.429 | 0.437 |
| MAELLE | Electron flow | 0.883 | 0.934 | 0.942 | 0.482 | 0.581 | 0.607 |
* Number reported in the original papers.
Similar to other SMILES-based model, RXNGraphormer performs well on USPTO-480K, but falls short on AbSynth.
The contribution of the graph-distance OT component
As our results show, removing the graph-distance OT component, by performing random coupling instead, only slightly decreases the model performance on end-to-end prediction. This is understandable because both coupling methods ensure end-to-end consistency.
What the graph-distance OT component constributes however, is the chemical interpretability of the CTMC trajectories, evidence by the correct direction of electron moves, using FlowER elementary steps as references.
Additionally, our abalation also shows that when either the OT component or the FLOW operation get removed, MAELLE still performs better than MEGAN (graph-edit-based) and NERF (oneshot electron redistribution) at most sample sizes, suggesting that it was the electron representation + discrete flow matching formulation that helps MAELLE. We would like to emphasize that, and we will do so in the revised version of the paper, these two components are also our novelty, as to the best of our knowledge, MAELLE is also the first discrete flow matching model on electrons for reaction prediction.
Thank you for the additional experiments. The RXNGraphormer comparison addresses my baseline concern. However, my concern about the motivation for graph-distance OT remains. I am keeping my original rating of 1.
Response to Reviewer vUW8
Thank you for your constructive feedback. We address your concerns below. Before we go into that, as reviewer L2ch points out, the performance of NERF is indeed lower than usual, and we identified and corrected bugs in our evaluation script. First, we would like to provide an updated main result:
Updated main results
The corrected NERF performance is substantially closer to its originally reported results. Our main conclusion remains unchanged: MAELLE is competitive on USPTO-480K and achieves the highest AbSynth accuracy among the evaluated baselines.
Why were stronger and more recent baselines excluded?
We selected representative template-free methods spanning three prediction modalities: SMILES generation (Molecular Transformer, Graph2SMILES), graph editing (MEGAN), and electron redistribution (NERF). Our goal was to compare these modeling representations under a consistent input setting rather than exhaustively benchmark architectural variants.
We agree that NERF* and ReactionSink are relevant. However, NERF* was reported in the initial version of the preprint. However, the authors subsequently replaced the NERF-based approach with ReactionTeam, a substantially different Graph2SMILES-based architecture, in the latest version. The NERF* method and its originally reported results are no longer presented in the revised paper. For ReactionSink, we could not identify publicly available source code for reproduction and benchmarking on AbSynth.
Moreover, our primary contribution is not a new state-of-the-art USPTO-480K score, but an end-to-end discrete flow matching reaction prediction model with chemically interpretable CTMC trajectories. We evaluate both its generalization to AbSynth and the agreement of its inferred electron-flow directions with expert annotations.
Chemical motivation for graph-distance OT?
We use graph distance to encourage local electron transfers, rather than to approximate reaction energetics. This provides an endpoint-consistent alignment motivated by the locality of bond rearrangements.
Consider the reaction:
CH3OH + CH3Br → CH3OCH3 + HBrIn our electron-occupation representation, the two source sites are O–H and C–Br, while the two sink sites are C–O and Br–H. There are exactly two endpoint-valid one-to-one couplings:
Here,
Ddenotes a large distance assigned to disconnected sites. Plan A couples electron sites sharing an atom, while Plan B includes a transfer between disconnected sites. Thus, graph-distance OT selects the chemically meaningful assignment.We further evaluated the inferred directions against expert annotations over 17,534 bond-forming events. Graph-distance OT achieves 90.0% directional agreement, compared with 27.7 ± 0.4% for random source–sink coupling (see table). MAELLE's predicted electron-flow directions also exhibit high agreement with expert annotations:
These results support graph distance as a chemically informative locality prior and show that MAELLE's inferred electron-flow directions frequently agree with expert annotations. We note that we only claim directional agreement, not the recovery of the complete elementary reaction mechanism. We will clarify this motivation and limitation in Section 3.3, and add the two additional rows for graph-distance-based OT and random coupling into Table 2 in our paper.