acceptodds
Under review as a conference paper at ICLR 2027

ReactPG: Rethinking Reaction-to-Procedure Generation via Process Graph Diffusion

Abstract

Experimental procedure generation is a key task in automated chemical synthesis, requiring plausible operation sequences while maintaining global consistency across operation ordering, material-state dependencies, and parameter assignments. Conventional sequence-generation paradigms linearize these interdependent decisions into text, requiring the underlying relational structure to be recovered implicitly from token-level context. To address this limitation, we propose ReactPG, the first conditional discrete graph diffusion framework for chemical procedure generation. ReactPG reformulates procedure generation as conditional generation over typed process graphs and employs a categorical Markov diffusion process over a constrained state space, parameterized by a joint process graph Transformer. We further construct OpenExp-PG, an experimental process graph dataset derived from OpenExp, and evaluate ReactPG using both graph-level and text-level metrics. Experimental results show that ReactPG achieves strong performance across multiple metrics while attaining near-zero violation rates on the evaluated procedural consistency constraints, outperforming existing large language model-based approaches.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
chemical procedure generation, graph diffusion, graph representation learning, AI for chemistry

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sidROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies reaction-to-procedure generation, arguing that operation ordering, material bindings, and intermediate-state dependencies should be explicitly represented within the generation process, reducing the need to infer these relations implicitly from textual context. To this end, it introduces ReactPG, a conditional discrete graph diffusion method that represents experimental procedures as typed process graphs. Joint node-and-edge denoising coordinates interdependent structural predictions, while a constrained support excludes certain invalid configurations. The generated graph is subsequently completed with numerical parameters and compiled into a textual procedure.

Critical strengths
  1. The formulation is well motivated by the task and offers methodological novelty in procedure generation.
  2. Procedural correctness depends on the relationships between operations and material states; locally plausible operations alone do not ensure consistency across steps. ReactPG explicitly encodes these relationships in a process graph and imposes corresponding constraints during generation, making relations that would otherwise need to be recovered from text directly available for prediction and verification.
  3. Conditional discrete graph diffusion also allows operations and their dependencies to be revised jointly, providing a suitable mechanism for coupled procedural decisions. This design addresses procedural consistency through both the generative representation and explicit structural checks.
Critical weaknesses
  1. The reporting of baseline inputs and adaptation settings is insufficient for reproducibility and for assessing the fairness of the comparisons. The main text states that all methods use the same data splits and supervised records while retaining their respective input or retrieval mechanisms, but it does not specify the material-role information, reaction features, and reference procedures available to each baseline.
  2. The appendix describes ReactPG’s training-set retrieval and exclusion rules without establishing whether the other retrieval-based baselines use comparable settings. Although general fine-tuning and decoding strategies are provided, baseline-specific training hyperparameters are missing.
  3. The adaptation of QFANG’s reinforcement learning and reward mechanisms, as well as Thoth’s staged training, is also insufficiently documented.
Questions for authors
  1. Please provide the input information, retrieval settings, and main training configurations for each baseline, and clarify how they compare with those of ReactPG.
  2. For QFANG, please explain how reinforcement learning and reward mechanisms are handled in the adaptation. For Thoth, please clarify the training of the planning and completion stages.
  3. Please elaborate on the implementation of the autoregressive graph baseline and its comparability with the diffusion model in terms of model capacity, training budget, and constraint handling. Could you report variability across random initializations for both generators or provide a statistical analysis of their performance difference?
  4. Please compare the computational cost of ReactPG with existing methods, particularly the autoregressive graph baseline. Reporting training cost, end-to-end inference latency, throughput, and peak GPU memory usage under comparable hardware and execution settings would be useful. Where possible, please separate the costs of graph generation and numerical completion to characterize the trade-off between performance and efficiency associated with iterative denoising.
4 comments
AuthorsOpenReview ↗

Part 1 of our response, addressing Q1 and Q2.

Comment: Q1. Please provide the input information, retrieval settings, and main training configurations for each baseline, and clarify how they compare with those of ReactPG.

Reply: Thank you for your careful review and insightful comments. For a fair comparison, all methods receive structured input in JSON format containing reaction information and reference experimental procedures. The reaction information includes materials (reactants, products, solvents, and catalysts), reaction SMILES, and molecular encodings. Reference procedures are retrieved from the training set using cosine similarity between reaction fingerprints, retaining the top two results. One of the baselines additionally receive method-specific inputs: ReactXT receives the real temperature and duration values associated with reaction operations.

All methods use the same data splits and corresponding supervised examples. ReactPG is trained on process graphs, whereas the language-model baselines use the corresponding procedure texts. Detailed training configurations are provided in Table R1. Baselines marked with an asterisk use supervised fine-tuning followed by DPO or GRPO, as specified in Table R1. A slash separates the epoch counts of the two stages.

Table R1. Main training configurations.

MethodBase modelTraining approachEpochsBatch sizeOptimizerLearning rate
ReactXTGalactica-1.3BFull-parameter SFT1016AdamW1×10−4→5×10−51 \times 10^{-4} \rightarrow 5 \times 10^{-5}
MolT5MolT5-largeFull-parameter SFT1016AdamW1×10−4→5×10−51 \times 10^{-4} \rightarrow 5 \times 10^{-5}
ReactGPT*Qwen3-8BLoRA SFT / DPO2/116AdamW5×10−55 \times 10^{-5} / 1×10−61 \times 10^{-6}
Thoth*Qwen3-8BLoRA SFT / GRPO2/116AdamW5×10−55 \times 10^{-5} / 1×10−61 \times 10^{-6}
QFANG*Qwen3-8BLoRA SFT / GRPO2/116AdamW5×10−55 \times 10^{-5} / 1×10−61 \times 10^{-6}
ReactPG-R / -PGraph diffusion—5064AdamW2×10−4→5×10−52 \times 10^{-4} \rightarrow 5 \times 10^{-5}
ReactPG-ARGraph AR—5064AdamW2×10−4→5×10−52 \times 10^{-4} \rightarrow 5 \times 10^{-5}

Comment: Q2. For QFANG, please explain how reinforcement learning and reward mechanisms are handled in the adaptation. For Thoth, please clarify the training of the planning and completion stages.

Reply: Thank you for your careful review and insightful comments. We adapt QFANG and Thoth to our experimental setting with fair comparison in mind.

For QFANG, we use reaction information (materials, reaction SMILES, and molecular encodings) and reference reactions retrieved from the training set to construct the reaction-fact scaffold required for supervised training. During the GRPO stage, the model generates five complete procedure candidates per sample. We compute reward scores based on the matching of action formats, action types, and parameters, calculate relative advantages across the candidates, and use these advantages to update the generation policy.

For Thoth, we construct step sketches on OpenExp-PG using action–object–parameter triplets. During SFT, the model is jointly supervised to generate a sketch followed by the complete procedure conditioned on that sketch. During the GRPO stage, the model also generates five “sketch–complete procedure” candidates per sample. We compute a structural-quality reward for each sketch, together with sketch-consistency and format-compliance rewards for the complete procedure. We then calculate relative advantages from the combined rewards across candidates and use them to update the policy for the two-step response.

AuthorsOpenReview ↗

Part 2 of our response, addressing Q3 and Q4.

Comment: Q3. Please elaborate on the implementation of the autoregressive graph baseline and its comparability with the diffusion model in terms of model capacity, training budget, and constraint handling. Could you report variability across random initializations for both generators or provide a statistical analysis of their performance difference?

Reply: Thank you for your careful review and insightful comments. Graph diffusion jointly generates and refines the process graph through a fixed number of denoising steps, so the number of denoiser evaluations is independent of graph size. In contrast, graph AR predicts one operation together with all its associated discrete graph attributes in a single autoregressive decoding step, conditioned on previously generated operations. The number of autoregressive decoding steps therefore scales with the number of operations in the generated process graph.

The two models use exactly the same constraints, molecular encoder, and numerical completion component. Their graph-generation modules have comparable parameter counts: 57.51M for graph AR and 58.55M for graph diffusion. These counts exclude the molecular encoder and numerical completion component in both cases. The training-budget comparison is provided in Tables R1 and R3. We additionally report performance variability for both generators across three independent training seeds in Table R2.

Table R2. Performance variability across three independent training seeds.

GeneratorLEV75 (%) ↑\uparrowAction-sequence exact match (%) ↑\uparrowWhole-graph exact match (%) ↑\uparrowSCVR (%) ↓\downarrowEdit similarity ↑\uparrow
Graph diffusion39.64±0.9739.64 \pm 0.9716.65±0.0816.65 \pm 0.086.77±0.286.77 \pm 0.280.03±0.010.03 \pm 0.0174.78±0.0274.78 \pm 0.02
Graph AR32.19±0.2632.19 \pm 0.2615.34±0.1015.34 \pm 0.102.86±0.332.86 \pm 0.333.76±0.323.76 \pm 0.3270.57±0.3670.57 \pm 0.36

Comment: Q4. Please compare the computational cost of ReactPG with existing methods, particularly the autoregressive graph baseline. Reporting training cost, end-to-end inference latency, throughput, and peak GPU memory usage under comparable hardware and execution settings would be useful. Where possible, please separate the costs of graph generation and numerical completion to characterize the trade-off between performance and efficiency associated with iterative denoising.

Reply: Thank you for your careful review and insightful comments. Table R3 compares the computational cost of ReactPG with six baselines, reporting training time, end-to-end inference latency, throughput, and peak GPU memory usage. We report three ReactPG configurations: retrieval-based numerical completion, prediction-based numerical completion, and graph generation only. All methods are benchmarked on the same NVIDIA A100 80GB GPU hardware using BF16 precision, with an inference batch size of 1 for these efficiency measurements.

In our implementation, graph AR predicts one complete operation together with its associated discrete graph attributes at each decoding step, whereas graph diffusion jointly refines the graph over 32 denoising steps. For shorter procedures, graph AR therefore requires fewer sequential generation steps, which contributes to its lower graph-generation latency.

Table R3. Computational cost comparison.

MethodTraining time (h)E2E latency (ms/sample)Throughput (samples/s)Training peak GPU memory (GiB)Inference peak GPU memory (GiB)Graph generation (ms/sample)Numerical completion (ms/sample)
ReactXT87.21898.20.5328.637.47——
ReactGPT42.13413.50.2958.8918.04——
Thoth37.34698.70.2156.4418.06——
QFANG39.98254.00.1260.4218.07——
MolT560.22212.30.4525.076.90——
ReactPG-R12.2718.41.3935.702.94574.7140.2
ReactPG-P12.22616.80.3835.707.72567.32045.6
ReactPG (graph generation only)12.2577.81.7335.702.85574.3—
ReactPG-AR11.92544.70.3935.167.68490.12051.4

Thank you for the response. The details on the shared inputs, retrieval setup, and training configurations address my main concerns about the baseline comparisons. The descriptions of QFANG’s reward design and Thoth’s sketch-and-completion training also clarify how the two methods were adapted.

The matched training schedules and similar parameter counts make the comparison with graph autoregression more informative. The timing results also clarify the trade-off: diffusion is slower at the graph generation stage, but ReactPG-P has similar end-to-end latency to the autoregressive baseline under the reported settings.

These clarifications give me more confidence in the experimental comparisons. I am raising my recommendation from weak rejection (2) to weak acceptance (3).

Thank you for carefully considering our responses and for your constructive feedback throughout the discussion. We are glad that these clarifications have addressed your main concerns about the experimental comparisons. We have incorporated the relevant details into the revised manuscript. Thank you again for raising your score.

Reviewer_LBAmOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to automatically generate experimental procedures from a given chemical reaction and material information. The authors propose ReactPG, which represents a procedure as a graph that explicitly records what each step does, which materials it uses, and how it connects to the outputs of earlier steps. Using reference procedures retrieved from the training set, the model first predicts the number of operations, then generates and adjusts the procedure graph through multiple denoising rounds using discrete graph diffusion. Numerical values, such as temperatures, durations, and quantities, are subsequently filled through retrieval or a separate predictive model, before the graph is converted into textual instructions. The authors also construct OpenExp-PG, a dataset of 115,000 records derived from OpenExp. They evaluate agreement with reference procedures and check whether material references, parameter assignments, and step dependencies follow predefined rules. They report improvements over the evaluated baselines on multiple metrics and near-zero violation rates for the assessed structural rules.

Critical strengths
  1. The paper explicitly represents each experimental operation, its associated materials, and its dependencies on earlier steps, making this information directly available for model prediction and rule checking. Ablation results show that removing message passing along graph connections reduces performance in predicting complete procedure graphs, material bindings, and dependencies between steps. These results provide experimental support for using graph structure to model experimental procedures.

  2. On the evaluated dataset, the ReactPG variant with model-based numerical filling outperforms the tested baselines on text similarity and structured-step matching metrics, while reducing violations of the assessed structural rules. The retrieval-based filling variant also maintains a low structural violation rate. Results across three independent training runs are similar, suggesting stability in the current experimental setting.

Critical weaknesses
  1. The respective contributions of design constraints and model predictions to the near-zero structural violation rate remain unclear. The graph representation and sampling constraints directly rule out some structural errors, so the aggregate violation rate reflects both their effect and the model's predictions. The autoregressive comparison using the same graph representation and constraint support provides evidence that diffusion offers additional improvements. However, the paper does not detail which assessed errors these constraints rule out or separately report the performance of the autoregressive and diffusion approaches on the remaining errors. The ablation removing joint constraints also retains other representational and sampling restrictions. The available evidence therefore does not fully explain which improvements in modeling material relationships and cross-step dependencies account for the near-zero violation rate.

  2. The extent to which improved structural consistency reduces substantive errors in complete experimental procedures remains insufficiently validated. The paper checks structural rules concerning references, compatibility, parameter validity, and conflicting values, but a procedure that passes these checks may still select incorrect operations, use the wrong materials, or specify inappropriate temperatures, durations, and quantities. Improvements in graph matching and text similarity provide additional evidence, and the authors acknowledge that accurate graph structure does not ensure accurate numerical values. Nevertheless, the paper lacks a systematic assessment of these substantive errors, particularly their types and frequency among complete procedures that pass structural checks, and how much they decrease relative to the strongest baseline. The available evidence therefore mainly establishes metric advantages under a particular graph representation and set of constraints on a filtered data subset, while reductions in substantive procedure errors remain unclear. This limits the scientific and practical significance of the work. Even if the claims are restricted to the current evaluation scope, I still consider the established contribution insufficient for acceptance.

  3. The effect of data filtering on coverage of key material relationships and cross-step dependencies remains insufficiently analyzed. More than half of the original records are excluded because they cannot be unambiguously annotated as process graphs. Many exclusions concern material identity, material roles within operations, and which material is retained after separation or filtration—issues directly related to the relationships the paper aims to model. However, the paper does not sufficiently distinguish missing information in the source descriptions from limitations of the graph representation or annotation rules. Similar operation and material counts do not establish comparable coverage of these dependencies. The results therefore support effectiveness on the selected subset, while the scope of applicability remains insufficiently established.

Questions for authors
  1. Which assessed violations are ruled out directly by the graph representation or sampling constraints? Among the violations that remain possible, which error types does diffusion primarily reduce relative to the autoregressive comparison using the same representation and constraint support?

  2. On the same set of reaction samples, does ReactPG reduce substantive errors in operation selection, material use, and experimental conditions relative to the strongest baseline? Please report the main error types and their frequencies through expert review of sampled outputs or targeted content evaluation, and indicate how many such errors remain after structural checks are passed.

  3. Please audit samples from the main exclusion categories to distinguish missing information in the source descriptions from limitations of the graph representation or annotation rules, and clarify whether filtering systematically excludes particular material relationships or cross-step dependencies.

4 comments
AuthorsOpenReview ↗

Part 1 of our response, addressing Q1 and Q2.

Comment: Q1. Which assessed violations are ruled out directly by the graph representation or sampling constraints? Among the violations that remain possible, which error types does diffusion primarily reduce relative to the autoregressive comparison using the same representation and constraint support?

Reply: Thank you for your careful review and insightful comments. The graph representation and constraints primarily rule out validity violations, including invalid action names or output formats, premature YIELD operations, and mismatches between numerical values and units. The model must learn to avoid consistency violations, including incorrect bindings between product quantities and identities, material selections inconsistent with input identities, incorrect material-role assignments, conflicting values within a parameter slot, and contradictions among conditions. Together, these validity and consistency violations constitute the categories assessed by SCVR.

Table R1 shows that both generators exhibit isolated product-binding failures, but graph diffusion substantially reduces condition conflicts and incorrect role assignments. Graph AR produces 360 outputs with incompatible roles or conflicting acidity or atmosphere conditions. On these same samples, graph diffusion produces only two acidity conflicts.

Table R1. Violation comparison between graph diffusion and graph AR.

Violation typeGraph diffusionGraph AR
Value conflict within parameter slot07
YIELD without a target product19
Incompatible role098
Conflicting acidity conditions2158
Conflicting atmosphere conditions0104
Overall SCVR (%)0.033.76

Comment: Q2. On the same set of reaction samples, does ReactPG reduce substantive errors in operation selection, material use, and experimental conditions relative to the strongest baseline? Please report the main error types and their frequencies through expert review of sampled outputs or targeted content evaluation, and indicate how many such errors remain after structural checks are passed.

Reply: Thank you for your careful review and insightful comments. We compared ReactPG with QFANG, the strongest baseline, using their outputs for the same 100 sampled test reactions. We used GPT6 Astra to assess these 200 outputs in four categories: reasonable experimental procedures; minor, easily fixable issues; issues that the LLM could not determine conclusively, mainly ambiguities relative to the reference for which the available evidence did not establish a clear serious error; and clear serious errors. Table R2 reports the results.

ReactPG performs relatively better in this assessment. Its two minor issues are as follows. In sample ID 223295, a malononitrile anion replaces a chlorine substituent on a triazine to form a carbon-carbon bond. The necessary reactants and transformation are represented, but a later empty ADD operation can simply be removed. In sample ID 199501, the reaction of an amidoxime with an acid anhydride to form an oxadiazole is represented, but an empty ADD operation before water addition can likewise be removed.

QFANG's only clear serious error occurs in sample ID 29465. The Staudinger reaction first forms an iminophosphorane, which is then hydrolyzed to the amine. The output includes only the reaction of the azide with PPh3 in DCM, followed by filtration and concentration, omitting the hydrolysis stage explicitly described in the source. The iminophosphorane cannot be treated directly as the target amine. This involves errors in material use and operation selection.

ReactPG's only clear serious error occurs in sample ID 9459. In this Mitsunobu reaction, phthalimide replaces a hydroxyl group, but phthalimide (material $2$) is never added, leaving no source for the new imide moiety in the target product. This is a material-use error.

Table R2. LLM-based evaluation of 100 outputs per method.

MethodReasonable proceduresMinor, easily fixable issuesIndeterminate issuesClear serious errors
ReactPG772201
QFANG710281

We also analyzed the main issue types for both methods (Table R3). An output may involve multiple issue types, so their summed counts can exceed the number of outputs with issues. ReactPG's issues are concentrated in material use, while it has substantially fewer issues in operation selection and experimental conditions than QFANG. Both methods have low SCVR: only one QFANG output with conflicting experimental conditions was detected by SCVR. The remaining flagged issues were not detected by these structural checks.

Table R3. Counts of issue types; an output may contribute to multiple categories.

MethodOperation selectionMaterial useExperimental conditionsOutputs with issues
ReactPG915723
QFANG17141929
AuthorsOpenReview ↗

Part 2 of our response, addressing Q3.

Comment: Q3. Please audit samples from the main exclusion categories to distinguish missing information in the source descriptions from limitations of the graph representation or annotation rules, and clarify whether filtering systematically excludes particular material relationships or cross-step dependencies.

Reply: Thank you for your careful review and insightful comments. We agree that exclusion reasons cannot be directly equated with missing information in the source descriptions. To distinguish these causes, we examined the source texts, action sequences, and graph-construction rules for the main exclusion categories, and measured the coverage of material relationships and cross-step dependencies in the retained records.

First, we reviewed 23 diagnostic cases used for exclusion and the corresponding samples. Table R4 summarizes the exclusion causes, counts, proportions, and their classification. Exclusions due to insufficient or erroneous source information account for 91.91%, including unspecified information, unclear phase selection or retained fractions, and inconsistencies among source text, reactions, and actions. The remaining 8.09% arise from graph-parsing failures and limitations of graph-annotation rules, including unresolvable information and exclusions imposed by graph constraints.

Table R4. Exclusion causes and their classification.

CategoryExclusion causeRecordsShare (%)
Source informationUnspecified identities, roles, objects, or numerical values69,57548.18
Source informationUnclear phase selection or retained fraction after filtration59,76741.39
Source informationInconsistency among source text, reaction, and actions3,3832.34
Graph parsing / rulesUnresolvable identities, roles, objects, or numerical values5,3663.72
Graph parsing / rulesExclusions due to graph constraints6,3144.37

For example, some source descriptions mention catalysis without specifying the catalyst; others explicitly label an action object as "unknown." These records are excluded because of unresolved information. In filtration and extraction, some descriptions do not specify which fraction or phase is retained, leaving the state passed to the next step unknown and leading to exclusion.

Second, filtering does not eliminate material reuse or cross-step role changes as a whole. Among the retained records, 56.41% reuse the same material across steps, 20.74% assign different roles to the same material across steps, and 70.57% contain at least one operation with two output states (Table R5).

Table R5. Coverage of material relationships and cross-step reuse.

RelationRecordsShare (%)
Material reuse across steps64,86656.41
Different roles for the same material across steps23,85120.74
At least one two-output-state operation81,16070.57

We additionally compared the original OpenExp score: the mean is 0.5452 for retained records and 0.5530 for excluded records. The original work describes this field as the TextChemT5 perplexity score, for which lower values are better. We therefore limit our scope to process-graph generation under the current graph schema within the chemical-reaction domain covered by OpenExp, with substantial coverage of material reuse and role changes.

Reviewer_LBAmOpenReview ↗

Response to Rebuttal

Compared with the autoregressive graph baseline, graph diffusion reduces material-role assignment errors and conflicts between experimental conditions. To assess whether these structural gains translate into fewer substantive errors in complete experimental procedures, the authors used a large language model (LLM) to evaluate procedures generated by ReactPG and QFANG for the same 100 reactions. However, the evaluation consisted of a single round. The LLM could not reach a clear judgment on 20 ReactPG procedures and 28 QFANG procedures. Among the remaining procedures, it identified one clear serious error for each method. These results are therefore insufficient to establish whether complete procedures generated by ReactPG contain fewer substantive errors. The authors also report the coverage of cross-step dependencies in the retained data, but it remains unclear whether filtering disproportionately excludes examples with particular types of cross-step dependencies.

For these reasons, I maintain my rating of 2 (weak rejection).

Thank you for your response. We agree that a single round of LLM evaluation on the same 100 reactions, which identified one clear serious error for each method, is insufficient to conclude that ReactPG produces fewer substantive chemical errors overall. However, the breakdown by issue type shows a more specific difference. The evaluation flagged operation-selection issues in 9 ReactPG procedures versus 17 for QFANG, and experimental-condition issues in 7 versus 19 (Table R3). Our earlier wording may have left the meaning of indeterminate unclear. We used this category when a generated procedure differed substantially from the reference and the available evidence was insufficient to establish either chemical correctness or a clear serious error. Fewer ReactPG procedures fell into this lower-confidence category (20 versus 28, Table R2). Taken together, these findings suggest that ReactPG was more reliable in selecting operations and experimental conditions on the reactions assessed.

We agree that coverage in the retained set alone cannot establish whether filtering disproportionately excludes particular cross-step dependencies. To examine how the two methods perform beyond the retained set, we evaluated both without retraining on 10,000 records randomly sampled from the excluded records. As shown in Table R1 below, both methods had lower matching rates on these records than on the original test set. Their structural consistency violation rates (SCVR) also rose. Ambiguous or incomplete source descriptions may make it harder both to generate a procedure and to assess it against the reference, which may partly explain the decline in performance. ReactPG nevertheless retained its consistency advantage on these additional records.

Table R1. Performance on 10,000 evaluable excluded records and the original test set of 10,000 retained records. Results are averaged over three random seeds.

Metric (%)ReactPG (excluded)ReactPG (original)QFANG (excluded)QFANG (original)
LEV50 ↑\uparrow97.1097.7795.0295.36
LEV75 ↑\uparrow35.1039.6433.7136.54
LEV90 ↑\uparrow12.3013.0512.1512.82
SCVR ↓\downarrow0.080.030.920.88
Reviewer_kqTqOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tackles reaction-to-procedure generation: given a reaction and an inventory of materials, produce an executable experimental procedure. The authors argue that existing language-model approaches linearize interdependent decisions into text, so relational structure such as material bindings, intermediate state dependencies, and parameter ownership must be recovered implicitly from token context. ReactPG instead treats a typed process graph as the generation object and learns a conditional categorical diffusion model over it, with a constrained state space that excludes invalid configurations, a grouped transition kernel for material edges, a hierarchical reconstruction distribution, and a joint node-edge graph Transformer denoiser. Continuous numerical values are filled in afterwards by a separate module. The authors also build a graph-annotated dataset derived from OpenExp and evaluate with text-level metrics, structured-text metrics, and a structural consistency violation rate.

Critical strengths

The failure mode being targeted is real and specific: a plausible action that binds the wrong material or references an unavailable intermediate. Decomposing this into an auditable set of violation types and reporting it as a dedicated metric gives considerably more information than BLEU alone.

The diffusion design goes beyond a straightforward port of existing discrete graph diffusion. The grouped transition kernel that separates role changes from quantity-slot changes while keeping closed-form cumulative transitions, and the hierarchical reconstruction that normalizes each factor over parent-compatible candidates, are genuine adaptations to the structure of this task. A full ELBO and an unbiased estimator are provided.

Dataset construction is documented transparently, including a per-criterion breakdown of excluded records, an explicit statement that no model predictions were used for selection, and details of group isolation, deduplication, and deterministic sampling.

The ablations are well designed. Comparing graph diffusion against a graph autoregressive baseline while holding the representation, constraint mechanism, filling module, and compiler fixed is a clean contrast, and the component ablations identify edge messages as the most important ingredient. Results are averaged over multiple seeds and include significance testing against one baseline, which is more than many comparable submissions.

Critical weaknesses

My main reservation is that much of the consistency advantage may come from construction rather than from learning. Validity and near-zero violations are jointly guaranteed by the constrained support, the structural checks, and a deterministic graph-to-text compiler, whereas the baselines generate free text with no comparable guarantees. Q1: If the language-model baselines were given equivalent constrained decoding or a post-hoc validation and repair step, how much of the gap would remain? Relatedly, the graph autoregressive baseline shares the same constraint mechanism and compiler yet still shows a substantially higher violation rate, which is not explained.

Absolute performance makes the word "competitive" worth qualifying: exact match over all graph fields is in the single digits, meaning the large majority of outputs are not fully correct. Q2: Could the authors grade error severity, so readers can tell which errors are chemically tolerable and which are not?

The parameter counts and components of the two variants are somewhat entangled, since the stronger variant appears to incorporate a fine-tuned version of one of the baselines as its numerical filling module. Q3: How much of the improvement over the retrieval-based variant comes from graph generation versus from the filling model, and would an appropriate control isolate this?

Roughly half of the source records were excluded because they could not be parsed into the declared schema, and such records plausibly skew toward more complex or less standardized procedures. The distributional comparison provided is fairly coarse. Q4: Could the comparison be extended to reaction types, material diversity, and condition coverage, and could the baselines be evaluated on the unfiltered corpus to show whether the new dataset is an easier subset?

Evaluation does not touch chemical correctness, which the authors acknowledge for the filtering stage but which also applies to the results. Q5: Could an output with no schema violations still be chemically infeasible, and would a small expert-rated subset be feasible?

I would also soften the novelty framing slightly: the diffusion framework itself largely follows prior discrete graph diffusion work, with the task-specific state space and transition design being the real contribution. Q6: Is there an ablation isolating the grouped material-edge kernel against a standard categorical kernel over the joint category space?

Two smaller points: several baselines are very recent, and one was transferred from a different protocol-generation task, so it is unclear how much adaptation and tuning they received; and no inference latency or memory comparison is reported, which matters for a system motivated by experimental automation. Q7: Could the baseline adaptation protocol and the inference cost comparison be described?

Questions for authors

see weakness

6 comments
AuthorsOpenReview ↗

Part 1 of our response, addressing Q1.

Comment: Q1. If the language-model baselines were given equivalent constrained decoding or a post-hoc validation and repair step, how much of the gap would remain? Relatedly, the graph autoregressive baseline shares the same constraint mechanism and compiler yet still shows a substantially higher violation rate, which is not explained.

Reply: Thank you for your careful review and thoughtful analysis of the constraint mechanism. Among the violations assessed by SCVR, the graph representation and constraints primarily rule out validity violations, such as invalid action names or output formats, premature YIELD operations, and mismatches between numerical values and units. Violations that the model must learn to avoid primarily concern consistency, including incorrect bindings between product quantities and identities, material selections inconsistent with input identities, incorrect material-role assignments, conflicting values within a parameter slot, and contradictions among conditions. We applied post-hoc validation and repair to the outputs of five language-model baselines. Table R1 shows that repairing invalid text does reduce SCVR to some extent, but the main remaining gap concerns consistency violations.

Table R1. SCVR (%) before and after post-hoc validation and repair.

MethodOriginal SCVRSCVR after repair
MolT54.163.28
ReactXT8.206.88
ReactGPT4.784.00
Thoth2.402.16
QFANG0.880.80

Table R2 compares graph diffusion and graph AR. Although both use the same graph representation, constraints, and compiler, these constraints mainly ensure locally valid combinations of actions and material roles; they do not fully prevent global semantic conflicts across steps. For example, a material-role assignment may satisfy local constraints while conflicting with material use or experimental conditions elsewhere in the procedure. Graph AR generates operations and their attributes sequentially without revising earlier decisions, allowing errors to accumulate in subsequent steps. In contrast, graph diffusion uses bidirectional attention and joint node-edge modeling to coordinate operations and their relations during iterative denoising and revise inconsistent predictions.

Graph diffusion reduces acidity conflicts from 158 to 2 and atmosphere conflicts from 104 to 0. These results suggest that its lower violation rate is primarily associated with global relational modeling and iterative refinement, rather than additional constraint rules.

Table R2. Violation counts for graph diffusion and graph AR; overall SCVR is reported in percent.

Violation typeGraph diffusionGraph AR
Value conflict within parameter slot07
YIELD without a target product19
Incompatible role098
Conflicting acidity conditions2158
Conflicting atmosphere conditions0104
Overall SCVR (%)0.033.76
AuthorsOpenReview ↗

Part 2 of our response, addressing Q2 and Q3.

Comment: Q2. Could the authors grade error severity, so readers can tell which errors are chemically tolerable and which are not?

Reply: Thank you for your careful review and suggestion to clarify error severity. Prior work notes that deviations from reference procedures do not necessarily imply chemical errors, and practical executability requires further assessment [1]. ChemCrow also evaluates chemical correctness, reasoning quality, and task completion beyond formal validity [2]. Drawing on these evaluation principles, we defined four categories for our supplementary sample assessment, considering reaction-pathway plausibility, material use, operation selection, experimental conditions, repairability, and evidential sufficiency.

(1) Chemically reasonable: The key reactants, operations, and conditions follow chemical reaction logic and reasonably support formation of the target product. Minor wording issues are acceptable if they do not affect the procedure or require correction.

(2) Easily fixable issues: Clear, local defects do not affect the main reaction pathway or key chemical decisions and can be repaired using deterministic expert rules. For example, an empty ADD instruction without an object can be ignored or deleted once it is confirmed not to correspond to an actual chemical operation, without changing the procedure.

(3) Indeterminate issues: The generated procedure differs substantially from the reference and contains insufficiently supported information. Available evidence establishes neither chemical correctness nor a clear serious error. Such outputs should not be used directly in experiments because their reliability remains insufficiently verified.

(4) Clear serious errors: Chemical mechanisms or available evidence establish critical errors, such as omission of an essential reactant, a missing key transformation, or an operation inconsistent with the target reaction, making the intended transformation or product formation implausible. These cases require replanning the complete experimental procedure.

References

[1] Vaucher, A. C., et al. (2021). Inferring experimental procedures from text-based representations of chemical reactions. Nature Communications, 12, 2573. https://doi.org/10.1038/s41467-021-22951-1 [2] Bran, A. M., et al. (2024). Augmenting large language models with chemistry tools. Nature Machine Intelligence, 6, 525-535. https://doi.org/10.1038/s42256-024-00832-8

Comment: Q3. How much of the improvement over the retrieval-based variant comes from graph generation versus from the filling model, and would an appropriate control isolate this?

Reply: Thank you for your careful review and precise examination of the contributions of different modules. ReactPG-R and ReactPG-P use exactly the same graph generator and generated graphs. Their only difference is numerical completion: ReactPG-R retrieves values from similar procedures, whereas ReactPG-P predicts them with a fine-tuned model. Their graph-generation performance is therefore identical, and all improvements of ReactPG-P over ReactPG-R reported in the paper arise from more accurate numerical completion, not differences in graph generation.

To further separate these contributions, we added a control that excludes numerical information. We recomputed LEV between generated and reference procedures without numerical-parameter differences to assess operation sequences and procedural structure. Because both ReactPG variants share the same generated graphs, their results are identical under this evaluation and are reported together as ReactPG in Table R3.

Table R3. Performance (%) after excluding numerical information.

MetricThothQFANGReactPG
LEV$\geq$5095.0894.0699.66
LEV$\geq$7570.6269.4875.64
LEV$\geq$9024.3824.7028.24
Mean LEV78.2177.8682.90

ReactPG remains better on all metrics. Relative to the strongest baseline for each metric, it improves LEV$\geq$50, LEV$\geq$75, and LEV$\geq$90 by 4.58, 5.02, and 3.54 percentage points, respectively, and mean LEV by 4.69 points. Thus, its advantage in operation-sequence and procedural-structure prediction remains after excluding numerical differences. Thoth also surpasses QFANG on LEV$\geq$50, LEV$\geq$75, and mean LEV, indicating stronger performance on most measures of procedure structure.

AuthorsOpenReview ↗

Part 3 of our response, addressing Q4 (Distributions and Coverage).

Comment: Q4. Could the comparison be extended to reaction types, material diversity, and condition coverage, and could the baselines be evaluated on the unfiltered corpus to show whether the new dataset is an easier subset?

Reply: Thank you for your careful review and detailed comments on filtering bias and representativeness. We agree that filtering may change the data distribution and that operation and material counts alone do not establish representativeness. We therefore added analyses of reaction types, material relationships, and condition coverage, and evaluated the existing ReactPG model and the strongest baseline, QFANG, on excluded records that remain parseable and evaluable.

(1) Reaction-type distribution. Using the ReactionClassifier taxonomy, OpenExp-PG covers all nine major reaction classes in Table R4, including heteroatom alkylation and arylation, acylation, C-C bond formation, heterocycle formation, and redox reactions. All nine classes remain represented, with proportions similar to those in the original corpus.

Table R4. Reaction-type distributions in the original and included sets.

Reaction typeOriginal countOriginal (%)Included countIncluded (%)
Heteroatom alkylation and arylation83,25630.3436,70831.92
Acylation and related reactions55,90220.3725,50122.17
C-C bond formation32,16711.7211,1949.73
Heterocycle formation19,1596.987,4836.51
Deprotection and related cleavage34,32212.5114,06112.23
Reduction15,8205.766,2215.41
Oxidation7,1992.623,1372.73
Functional group interconversion19,8837.247,9116.88
Functional group introduction6,4722.362,7212.37
Other reactions2590.09630.05

(2) Material reuse and cross-step relationships. In the included set, 56.41% of records reuse the same material across steps, 20.74% assign different roles to the same material across steps, and 70.57% contain at least one operation with two output states (Table R5). Thus, filtering does not eliminate material reuse, role changes, or cross-step state dependencies as a whole; the retained data still contain substantial procedural relationships.

Table R5. Material relationships and cross-step reuse in the included set.

RelationRecordsShare (%)
Material reuse across steps64,86656.41
Different roles for the same material across steps23,85120.74
At least one two-output-state operation81,16070.57

(3) Condition coverage. We compared condition coverage without distinguishing specific numerical values, counting only whether each subtype occurs. Under the current identification rules, the included set retains 39 of the 40 identified subtypes (97.50%; Table R6). All temperature, duration, acidity/basicity, and other-condition subtypes are retained, as are 20 of 21 atmosphere-related subtypes.

Table R6. Condition subtype coverage.

Condition categoryOriginal subtypesIncluded subtypesCoverage (%)
Temperature44100.00
Duration22100.00
Atmosphere and related operations212095.24
Acidity/basicity44100.00
Other conditions99100.00
Total403997.50

Temperature subtypes are numerical temperature, room/ambient temperature, reflux, and ice bath; duration subtypes are numerical duration and overnight; acidity/basicity subtypes are numerical pH, acidic, basic, and neutral.

Atmosphere-related subtypes comprise argon, carbon monoxide, carbon dioxide, chlorine, hydrogen, hydrogen sulfide, hydrogen bromide, hydrogen chloride, helium, nitrogen, ammonia, oxygen, ozone, sulfur dioxide, acetylene, air, ethylene, methane, degassing, hydrogenation, and inert atmosphere without a specified gas. The only missing subtype is chlorine (Cl2): all seven original records were excluded because of insufficient source information or parsing problems.

Other conditions comprise numerical pressure, vacuum, reduced pressure, atmospheric pressure, high pressure, stirring speed, flow rate, dropwise addition, and portionwise addition. Conditions may occur during the reaction or workup. These results show high retention of the identified condition subtypes.

Our response to Q4 continues in the next part with the exclusion audit and additional performance evaluation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Reviewers and Area Chair,

Thank you very much for your time and constructive feedback. We have uploaded a revised manuscript incorporating the clarifications and additional analyses discussed in our responses. The main text further clarifies the model design, baseline comparison settings, and evaluation scope. Appendices A and D expand the dataset coverage analysis and provide additional details on baseline adaptation, computational costs, controlled comparisons, training stability, and content assessment with illustrative cases. We welcome any further questions or comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.