- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies reaction-to-procedure generation, arguing that operation ordering, material bindings, and intermediate-state dependencies should be explicitly represented within the generation process, reducing the need to infer these relations implicitly from textual context. To this end, it introduces ReactPG, a conditional discrete graph diffusion method that represents experimental procedures as typed process graphs. Joint node-and-edge denoising coordinates interdependent structural predictions, while a constrained support excludes certain invalid configurations. The generated graph is subsequently completed with numerical parameters and compiled into a textual procedure.
Critical strengths
- The formulation is well motivated by the task and offers methodological novelty in procedure generation.
- Procedural correctness depends on the relationships between operations and material states; locally plausible operations alone do not ensure consistency across steps. ReactPG explicitly encodes these relationships in a process graph and imposes corresponding constraints during generation, making relations that would otherwise need to be recovered from text directly available for prediction and verification.
- Conditional discrete graph diffusion also allows operations and their dependencies to be revised jointly, providing a suitable mechanism for coupled procedural decisions. This design addresses procedural consistency through both the generative representation and explicit structural checks.
Critical weaknesses
- The reporting of baseline inputs and adaptation settings is insufficient for reproducibility and for assessing the fairness of the comparisons. The main text states that all methods use the same data splits and supervised records while retaining their respective input or retrieval mechanisms, but it does not specify the material-role information, reaction features, and reference procedures available to each baseline.
- The appendix describes ReactPG’s training-set retrieval and exclusion rules without establishing whether the other retrieval-based baselines use comparable settings. Although general fine-tuning and decoding strategies are provided, baseline-specific training hyperparameters are missing.
- The adaptation of QFANG’s reinforcement learning and reward mechanisms, as well as Thoth’s staged training, is also insufficiently documented.
Questions for authors
- Please provide the input information, retrieval settings, and main training configurations for each baseline, and clarify how they compare with those of ReactPG.
- For QFANG, please explain how reinforcement learning and reward mechanisms are handled in the adaptation. For Thoth, please clarify the training of the planning and completion stages.
- Please elaborate on the implementation of the autoregressive graph baseline and its comparability with the diffusion model in terms of model capacity, training budget, and constraint handling. Could you report variability across random initializations for both generators or provide a statistical analysis of their performance difference?
- Please compare the computational cost of ReactPG with existing methods, particularly the autoregressive graph baseline. Reporting training cost, end-to-end inference latency, throughput, and peak GPU memory usage under comparable hardware and execution settings would be useful. Where possible, please separate the costs of graph generation and numerical completion to characterize the trade-off between performance and efficiency associated with iterative denoising.
4 comments
Part 2 of our response, addressing Q3 and Q4.
Comment: Q3. Please elaborate on the implementation of the autoregressive graph baseline and its comparability with the diffusion model in terms of model capacity, training budget, and constraint handling. Could you report variability across random initializations for both generators or provide a statistical analysis of their performance difference?
Reply: Thank you for your careful review and insightful comments. Graph diffusion jointly generates and refines the process graph through a fixed number of denoising steps, so the number of denoiser evaluations is independent of graph size. In contrast, graph AR predicts one operation together with all its associated discrete graph attributes in a single autoregressive decoding step, conditioned on previously generated operations. The number of autoregressive decoding steps therefore scales with the number of operations in the generated process graph.
The two models use exactly the same constraints, molecular encoder, and numerical completion component. Their graph-generation modules have comparable parameter counts: 57.51M for graph AR and 58.55M for graph diffusion. These counts exclude the molecular encoder and numerical completion component in both cases. The training-budget comparison is provided in Tables R1 and R3. We additionally report performance variability for both generators across three independent training seeds in Table R2.
Table R2. Performance variability across three independent training seeds.
| Generator | LEV75 (%) | Action-sequence exact match (%) | Whole-graph exact match (%) | SCVR (%) | Edit similarity |
|---|---|---|---|---|---|
| Graph diffusion | |||||
| Graph AR |
Comment: Q4. Please compare the computational cost of ReactPG with existing methods, particularly the autoregressive graph baseline. Reporting training cost, end-to-end inference latency, throughput, and peak GPU memory usage under comparable hardware and execution settings would be useful. Where possible, please separate the costs of graph generation and numerical completion to characterize the trade-off between performance and efficiency associated with iterative denoising.
Reply: Thank you for your careful review and insightful comments. Table R3 compares the computational cost of ReactPG with six baselines, reporting training time, end-to-end inference latency, throughput, and peak GPU memory usage. We report three ReactPG configurations: retrieval-based numerical completion, prediction-based numerical completion, and graph generation only. All methods are benchmarked on the same NVIDIA A100 80GB GPU hardware using BF16 precision, with an inference batch size of 1 for these efficiency measurements.
In our implementation, graph AR predicts one complete operation together with its associated discrete graph attributes at each decoding step, whereas graph diffusion jointly refines the graph over 32 denoising steps. For shorter procedures, graph AR therefore requires fewer sequential generation steps, which contributes to its lower graph-generation latency.
Table R3. Computational cost comparison.
| Method | Training time (h) | E2E latency (ms/sample) | Throughput (samples/s) | Training peak GPU memory (GiB) | Inference peak GPU memory (GiB) | Graph generation (ms/sample) | Numerical completion (ms/sample) |
|---|---|---|---|---|---|---|---|
| ReactXT | 87.2 | 1898.2 | 0.53 | 28.63 | 7.47 | — | — |
| ReactGPT | 42.1 | 3413.5 | 0.29 | 58.89 | 18.04 | — | — |
| Thoth | 37.3 | 4698.7 | 0.21 | 56.44 | 18.06 | — | — |
| QFANG | 39.9 | 8254.0 | 0.12 | 60.42 | 18.07 | — | — |
| MolT5 | 60.2 | 2212.3 | 0.45 | 25.07 | 6.90 | — | — |
| ReactPG-R | 12.2 | 718.4 | 1.39 | 35.70 | 2.94 | 574.7 | 140.2 |
| ReactPG-P | 12.2 | 2616.8 | 0.38 | 35.70 | 7.72 | 567.3 | 2045.6 |
| ReactPG (graph generation only) | 12.2 | 577.8 | 1.73 | 35.70 | 2.85 | 574.3 | — |
| ReactPG-AR | 11.9 | 2544.7 | 0.39 | 35.16 | 7.68 | 490.1 | 2051.4 |
Thank you for the response. The details on the shared inputs, retrieval setup, and training configurations address my main concerns about the baseline comparisons. The descriptions of QFANG’s reward design and Thoth’s sketch-and-completion training also clarify how the two methods were adapted.
The matched training schedules and similar parameter counts make the comparison with graph autoregression more informative. The timing results also clarify the trade-off: diffusion is slower at the graph generation stage, but ReactPG-P has similar end-to-end latency to the autoregressive baseline under the reported settings.
These clarifications give me more confidence in the experimental comparisons. I am raising my recommendation from weak rejection (2) to weak acceptance (3).
Thank you for carefully considering our responses and for your constructive feedback throughout the discussion. We are glad that these clarifications have addressed your main concerns about the experimental comparisons. We have incorporated the relevant details into the revised manuscript. Thank you again for raising your score.
Part 1 of our response, addressing Q1 and Q2.
Comment: Q1. Please provide the input information, retrieval settings, and main training configurations for each baseline, and clarify how they compare with those of ReactPG.
Reply: Thank you for your careful review and insightful comments. For a fair comparison, all methods receive structured input in JSON format containing reaction information and reference experimental procedures. The reaction information includes materials (reactants, products, solvents, and catalysts), reaction SMILES, and molecular encodings. Reference procedures are retrieved from the training set using cosine similarity between reaction fingerprints, retaining the top two results. One of the baselines additionally receive method-specific inputs: ReactXT receives the real temperature and duration values associated with reaction operations.
All methods use the same data splits and corresponding supervised examples. ReactPG is trained on process graphs, whereas the language-model baselines use the corresponding procedure texts. Detailed training configurations are provided in Table R1. Baselines marked with an asterisk use supervised fine-tuning followed by DPO or GRPO, as specified in Table R1. A slash separates the epoch counts of the two stages.
Table R1. Main training configurations.
Comment: Q2. For QFANG, please explain how reinforcement learning and reward mechanisms are handled in the adaptation. For Thoth, please clarify the training of the planning and completion stages.
Reply: Thank you for your careful review and insightful comments. We adapt QFANG and Thoth to our experimental setting with fair comparison in mind.
For QFANG, we use reaction information (materials, reaction SMILES, and molecular encodings) and reference reactions retrieved from the training set to construct the reaction-fact scaffold required for supervised training. During the GRPO stage, the model generates five complete procedure candidates per sample. We compute reward scores based on the matching of action formats, action types, and parameters, calculate relative advantages across the candidates, and use these advantages to update the generation policy.
For Thoth, we construct step sketches on OpenExp-PG using action–object–parameter triplets. During SFT, the model is jointly supervised to generate a sketch followed by the complete procedure conditioned on that sketch. During the GRPO stage, the model also generates five “sketch–complete procedure” candidates per sample. We compute a structural-quality reward for each sketch, together with sketch-consistency and format-compliance rewards for the complete procedure. We then calculate relative advantages from the combined rewards across candidates and use them to update the policy for the two-step response.