- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks what to do when the barriers in a reaction network are uncertain. Instead of propagating one set of estimated barriers, it averages the predicted populations over perturbed barriers. The perturbations are generated from shared minimum and transition-state energies, so detailed balance is preserved for each sample. The paper then uses the assumed uncertainty to decide when a Krylov solve has done enough work. Results are reported on synthetic networks and on two literature-derived ozonolysis networks.
Critical strengths
-
The main idea is interesting. The paper uses the same uncertainty model twice: first to average the predicted populations, and then to decide how much numerical accuracy is worth paying for. That is a sensible connection between uncertainty in the input and the cost of solving the dynamics. The shared-energy construction also avoids breaking detailed balance by adding unrelated noise to each directed edge.
-
The controlled experiments are consistent with the proposed mechanism. On the synthetic networks, the uncertainty-aware predictor generally has lower population RMSE and better top-K accuracy than the deterministic baseline. The time-temperature sweep and the runtime study make the evaluation more useful than a single toy example. The CRN-E failure is also informative: the method can reweight paths that are present, but it cannot recover a reaction that is missing from the network.
Critical weaknesses
-
The evidence is still mostly tied to the assumed synthetic error model. The quantitative experiments generate the barrier errors with the same Gaussian covariance structure used by the method, and most of them set the assumed uncertainty equal to the true uncertainty. Figure 2b changes the assumed scale, which is useful, but it does not test biased errors or a different correlation structure. The two real-network examples are qualitative. CRN-T moves the prediction in the right direction but still misses the reported yield, while CRN-E cannot be solved because a required pathway is absent. The real calculations also omit free-energy corrections, and the choice of 4 kcal/mol is not calibrated against an independent reference. I would therefore describe the results as a promising first demonstration, not yet as evidence that the method is reliable for realistic reaction networks.
-
The truncation rule needs a clearer error story. Appendix B derives the eigenvalue-variance relation for independent edge perturbations, but the method in Section 4.1 uses correlated perturbations induced by shared minima and transition states. That derivation does not directly cover the model used in the experiments. Also, a small incremental SNR in Eq. (10) does not by itself tell us how large the remaining error in the ensemble mean can be. Figure 3 is a useful sanity check, but the paper would be stronger if it reported the discrepancy between the truncated solver and a converged ensemble calculation across the main experiments. A comparison with an ordinary tolerance-based stopping rule at matched accuracy would also help.
Questions for authors
-
How much do the gains change if the barrier errors have a systematic bias or a different covariance structure? For CRN-T, how was the 4 kcal/mol uncertainty chosen, and does the qualitative improvement persist over a reasonable range of values?
-
Can you report the error between the truncated and converged ensemble predictions using the same barrier samples? It would be useful to see representative worst-case or upper-quantile errors, as well as a comparison with tolerance-based stopping at matched prediction accuracy and hardware.
-
Is Appendix B intended to apply to the correlated perturbations used in Section 4.1? More generally, should Eq. (10) be read as an empirical stopping heuristic, or is it meant to provide a numerical error guarantee? These points are important for interpreting the scope of the claims. I do not think the paper needs a large new benchmark for the current submission, but the authors should be careful not to present the synthetic results as a validation of realistic chemistry errors.
Response to Reviewer HxEi
We thank the reviewer for the constructive feedback and address each point below.
As the reviewer notes, Fig. 2b varies only the assumed scale σ'. We have extended this into a sweep over σ and σ′ (Appendix E.1): p̄ improves over p for all 0 < σ′ ≤ 2σ, for σ ∈ {3, 6, 12} kcal/mol. p̄ therefore needs only a rough estimate of the barrier uncertainty, not a calibrated one. To test errors outside p̄'s noise model, we added an experiment in which the true barrier errors carry a systematic bias that p̄'s zero-mean model does not include. q̄ still reduces the total error across the whole bias range [−12, 12] kcal/mol for multiple σ ∈ {3, 6, 12} kcal/mol (Appendix E.2)
We now report the yield for CRN-T. The experimental rate constant (Dowideit & von Sonntag, 1998) implies at least 57% conversion after 1 ms; q̄ predicts 23%, against 0.5% for p. We also swept σ' ∈ {2, 4, 6, 8} kcal/mol, and the improvement of q̄ over p persists (Appendix E.3). As this is a single example, we still regard it as qualitative. The quantitative evaluation is on synthetic networks (Section 5.1–5.2).
Appendix B is not intended to cover the correlated perturbations of Section 4.1, and the SNR rule (Eq. 10) does not depend on it: Eq. 10 is an empirical stopping rule computed directly from the sampled populations, not a numerical error guarantee. Appendix B supports only the intuition in Section 4.2 that fast modes are the most affected by noise. We now state its role explicitly.
We thank the reviewer for this point. q̄ and p̄ reach the same population RMSE in all experiments in Sections 5.1-5.3. This was stated only briefly at the start of Section 5. We have now quantified this in Section 5.1: with the same networks and barrier samples, their population RMSEs differ by at most 1e-4 at every M and σ. The SNR cutoff therefore adds no measurable error, and the speed-up comes at matched accuracy. We now state this clearly in experiment 5.1