- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper builds a Petri net formulation around chemical constraints on valence, hydrogen counts, and charge. It adapts this unified formulation to atom mapping, forward reaction prediction, reaction classification, and reaction mechanism prediction, yielding Neural Petri Flow (NPF). The Petri net constraints improve data efficiency in low-resource settings and the chemical structural validity of predictions.
Critical strengths
- Defined by invariant valence budgets and a predefined set of operations, the NPF provides a unified, interpretable formulation that can be incorporated into different downstream chemistry tasks, with competitive performance in training-free and low-resource settings.
- The paper systematically and fairly compares NPF and its variants with other baselines across multiple well-defined downstream tasks. It also evaluates the chemical validity of the generated outputs, demonstrating that NPF's constraints improve prediction validity without significantly affecting downstream performance.
Critical weaknesses
- As a network that enforces chemical priors, NPF could be used as a standalone component independent of the trainable architecture. The paper also evaluates combinations of NPF with different readout networks. This raises the question of whether NPF can be combined directly with current state-of-the-art models in the field to improve both predictive performance and validity. NPF's task-specific designs also complicate the comparison: the trainable backbones can themselves affect performance and output validity through their own inductive biases (e.g., molecular editing rather than AR for reaction prediction may itself constrain the structural search space). The ablations do not fully isolate the benefits of NPF under these conditions.
- Related to this concern, a substantial portion of the benchmarks in the main text are already close to saturation, including atom mapping, forward reaction prediction, and reaction classification on USPTO-50K. There is consequently little separation between NPF and the baselines in accuracy and output validity, which further calls into question the benefits attributable to NPF itself. Earlier baselines have, in fact, already achieved performance close to saturation on these tasks.
Questions for authors
Following the concerns above, it is possible to view NPF as a standalone network encoding prior rules that could restrict the allowed molecular edits at each step or be incorporated into a backbone as an embedding. Have the authors explored integrating it directly with existing backbones, rather than designing their own backbone / readout strategy from-scratch?
2 comments
Comment by U23i
Thanks authors for their response. The new controlled backbone experiments clarify where NPF provides benefits, and the additional comparisons with models of the same budget demonstrate the value of its constraints, or priors. In my view, NPF's self-consistent and interpretable prior design could be explored further as a source of priors for existing backbones, or as a first principle for designing domain-specific backbones.
Based on these considerations, I have increased my rating to 3. My confidence score primarily reflects my familiarity with this field and background (which appears to be how confidence is defined in the ICLR system), rather than my personal confidence in the rating.
Thank you for your feedback. Even though you declare low confidence, we would like to answer your questions.
W1. Combination with existing models
We absolutely agree that the backbone matters. The firing form, the mask, and the state equation do not have trainable parameters, which means that a stronger network can replace our rate law and keep the valence guarantee, whose proof does not use the rate law (see Proposition 4). We have not tried this with our forward model. Our controls change one component and keep the backbone and its size (see Tables 14 and 22). In forward prediction, the backbone without the mask is as accurate on Schneider50k, but breaks valence budget in 0.02 % of the predictions, while a one-shot model does so in 2.79 % (see Table 18). Editing alone already keeps 99.98 % of the predictions within the budgets and the mask makes this a guarantee. So what the net gives to the forward model is a guarantee and not accuracy (see our answer to reviewer TgvJ, W1e).
W2. Saturated benchmarks
We agree for USPTO-480K, where NPF is not ahead at top-1 (see Table 4). Also for organic atom mapping and Schneider50k with all labels. What we believe separates us there is efficiency and methodology. For example, the mapper needs no training and NPF reaches 67.4 % top-1 with 1 % of the USPTO-480K training data (see Table 16). However, harder benchmarks separate the models. NPF reaches 90.2 vs 84.6 % at EC level 3 on ECREACT (see Table 3), of which 1.6 points come from the form of the readout (see Table 14), and 70.2 vs 59.3 % at the full EC number on CARE against the best method that only uses the reaction (see Table 3). Furthermore, it maps 88.7 vs 77.9 % of the EnzymeMap reactions (see Table 11)., recovers 94.5 vs 92.5 % of FlowER pathways (see Table 5), and reaches 81.2 vs 65.0% for DRFP with 250 labels on Schneider50k (Table 13). We credit these leads to NPF as a whole (beyond the controls mentioned in W1). On FlowER, every single top-1 prediction of our net is a valid molecule, and every prediction conserves atoms, protons, and electrons. The sequence models only do this for 17 to 33% of the predictions (see Table 21).
Q1. Integration with existing backbones
That’s only in classification, where an existing backbone (Chemprop CGR D-MPNN) reads the maps of our mapper. With 1k labels on Schneider50k, its best setting reaches 89.2 vs 95.1 % for our classifier on the firing vector (see Table 13). For forward prediction, the mask and the order of the edits must be chosen jointly. MEGAN orders its targets with all new bonds first, and the valence rule blocks every order like this in 59.7 % of our correct USPTO-480K predictions (with two or more changes, see Appendix D.5). When trained without the mask on a fixed order with all new bonds first first, we lose 32 points once the mask is turned on (see Table 23). This means that a mask in MEGAN would require a different order.