acceptodds
Under review as a conference paper at ICLR 2027

Graph Forward Distribution Matching for Molecular Inverse Design

Abstract

Achieving precise control over multiple properties without sacrificing chemical validity remains a central challenge in molecular inverse design. Existing reinforcement learning (RL) methods fine-tune graph diffusion models by treating **reverse** sampling as a sequential policy, using a single terminal reward to optimize hundreds of coupled decisions. They often suffer from instability, validity collapse, and limited property gains. We introduce GraphFDM (Graph Forward Distribution Matching), a new online RL paradigm for graph diffusion that performs optimization through the **forward** process. GraphFDM uses valid generations to define a reward-tilted target distribution jointly optimized over graph size and molecular structure for each property condition, incorporating reinforcement signals into supervised learning without storing reverse trajectories. We derive the unique optimal target, prove a condition-wise improvement guarantee, and show that the fixed graph-size prior of standard graph diffusion leaves an irreducible matching gap. In multi-conditional polymer and small-molecule generation, GraphFDM achieves the lowest MAE on every target property, with reductions of up to 53.0% relative to the strongest baselines and chemical validity above 0.99. It further generalizes to out-of-distribution property combinations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular inverse design, graph diffusion, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1aWqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies molecular generation conditioned on multiple target properties, aiming to produce chemically valid molecular graphs that closely match the specified targets.

To address the risk that reverse-trajectory policy updates may impair structural generation capabilities, GraphFDM builds on the CoMole backbone and uses terminal molecular rewards to weight denoising training on re-noised samples.

Since the inherited backbone (CoMole) samples node counts from a fixed distribution, GraphFDM also introduces SDC to adapt the proportions of molecules with different node counts. SDC places node counts and conditional structures under a common reward objective, constrains their changes separately, and uses the same weights to train the node-count network and denoiser.

Experiments on polymers and QM9 include comparisons from a shared initialization and ablations of SDC and KL budget allocation. The full method achieves the lowest MAE for each target property among the compared methods while maintaining high raw validity.

Critical strengths

S1. Tables 1 and 2 compare CoMole and both GraphFDM variants from the same SFT initialization. GraphFDM without SDC already improves property matching and raw validity, and the full method provides further overall gains. Its raw validity reaches 98.6% on polymers and 96.0% on QM9.

S2. SDC connects node-count adaptation and conditional structure learning through a common reward objective, with both theoretical derivations and ablation results. Table 4 compares joint and separate constraints under the same total KL budget and training configuration, providing evidence for the benefit of separate budget allocation.

S3. The experiments examine several forms of generalization and robustness. Table 3 tests property combinations that are rare in the QM9 conditional training set. Table 9 reports multiple-seed results on QM9, and Table 11 checks method rankings by rescoring the same generated polymers with alternative property predictors.

Critical weaknesses

W1. Two aspects of the explanation for the performance gains remain unclear. Tables 1 and 2 show that the post-training approach works, but do not fully explain how the change in supervision mitigates structural degradation. For SDC, Proposition 3.2 and the ablations support the design, but the paper does not show what node-count mismatch occurs for the tested targets or how SDC corrects it. (Q1 and Q2 addresses this concern.)

W2. The property predictors’ accuracy on generated molecules remains uncertain. Table 10 reports training-set fitting errors. Table 11 checks ranking robustness across predictors trained on the same labeled data, rather than assessing prediction accuracy on new structures. The evidence for improving the surrogate objectives is therefore stronger than the evidence for improving actual molecular properties. (Q3 addresses this concern.)

Questions for authors

Please prioritize Questions 1–3. They can serve as the response to the corresponding weaknesses; there is no need to answer both sections (Weaknesses & this sec) separately.

Q1. What evidence supports reward-weighted denoising as a way to mitigate the risk of structural degradation from reverse-trajectory policy updates? For example, could existing checkpoints or generation logs show how property error, raw validity, and structural errors evolve during post-training? Please explain what these observations reveal about the change in supervision and which components’ contributions remain difficult to distinguish.

Q2. For representative target conditions, could you compare the node-count distribution before an update, the SDC target distribution, and the learned distribution after that update? Please relate these distributions to the property performance of candidates with different node counts, to explain what mismatch arises in practice, how SDC addresses it, and why the adjustment is beneficial.

Q3. How reliable are the property predictions used to evaluate generated molecules? Beyond the training errors in Table 10 and ranking comparison in Table 11, are held-out or cross-validation errors available? How structurally similar are the generated molecules to the predictors’ training molecules, and what does this suggest about the predictors’ applicability to the generated structures?


Questions 4–5 are just secondary implementation clarifications. Brief explanations of the settings used in the reported experiments are sufficient.

Q4. The supplementary code appears to assess validity and properties using repaired molecules while retaining the original sampled graphs as training targets. Was this procedure used in the reported experiments? If so, how does it align with the definition of valid molecular graphs and the guarantees in Eq. (4) and Corollary 3.4?

Code references: graphfdm/common.py, decode_samples, lines 240–256; graphfdm/policy.py, save_round_dataset, lines 185–209.

Q5. Which standard deviations were used for reward normalization? Eqs. (42)–(43) specify training-set standard deviations after log transformation for gas properties, whereas the default preparation script computes them on the original scale, and the reward code subsequently reads these statistics. Please clarify the settings used in the reported experiments, since this changes the relative weighting of properties in the reward.

Code references: scripts/prepare_polymer.py, lines 60–69; graphfdm/common.py, evaluate_final_surface, lines 294–298.
Reviewer_vFfYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a reward-based post-training method for conditional molecular graph diffusion. The method uses reward-weighted denoising of generated molecules and introduces Structural Distribution Control (SDC) to jointly optimize molecular structures and graph-size distributions. The authors provide theoretical results characterizing the reward-tilted target distribution and evaluate the approach on multi-property molecular and polymer generation benchmarks.

Critical strengths
  • The paper addresses a relevant problem in molecular inverse design, particularly improving property control while preserving chemical validity.
  • The proposed method is intuitive and reasonably motivated. Learning the conditional graph-size distribution is a useful extension of conventional graph diffusion.
  • The experiments demonstrate consistent improvements over a broad range of baselines, including several recent diffusion post-training methods.
Critical weaknesses
  • I am not convinced by the property evaluation. The authors use the same random forest predictor to calculate training rewards and evaluate the generated molecules. Furthermore, this predictor is trained on all labeled molecules, including the conditional test set, and its reported accuracy is measured on the training data. How can we know whether the reported improvements reflect actual improvements in molecular properties, rather than simply optimizing the predictions of an imperfect oracle? This is particularly concerning given the known issue of reward hacking in molecular optimization. The additional evaluation using SVR and GP provides some reassurance, but these predictors are also trained on the same labeled data, and their accuracy on generated molecules remains unclear.

  • The novelty of the proposed method is limited, and its relationship to closely related work is insufficiently discussed. The main idea of sampling from a diffusion model, assigning reward-dependent weights to generated samples, and fine-tuning through the original denoising objective resembles reward-weighted regression (RWR), discussed in Appendix C.2 of Black et al. (ICLR 2024). Similar approaches have also been explored in AWM (ICML 2026) and DiffusionNFT (ICLR 2026). Although these works are cited, the authors do not sufficiently explain what is fundamentally new about their forward-denoising approach. In particular, a direct comparison against RWR using the same graph diffusion backbone is missing.

  • The contribution of SDC is not not clear. The method without SDC achieves substantial improvements on its own, the joint-KL variant performs almost identically to the version without SDC on polymers (0.590 vs. 0.589 average MAE). The additional improvement appears to come primarily from imposing separate KL constraints with a particular budget allocation. Why does this specific allocation work better? Under what conditions should we expect it to help? The authors provide little analysis of the learned graph-size distributions or how they change for different property targets. Given that SDC appears to be the main novel component, I think this deserves more attention.

  • The theoretical results have limited implications for the actual optimization procedure. The condition-wise reward improvement and validity guarantees apply to the exact reward-tilted teacher distribution, while the practical method uses finite-sample approximations and neural network optimization. The authors acknowledge this, but do not establish whether the guarantees approximately hold for the learned generator, or investigate how the approximation affects them. Furthermore, several theoretical results follow from relatively standard change-of-measure arguments, KL decomposition, and reward-tilted optimization. I am not convinced that the theoretical contribution is particularly substantial.

  • The experimental evaluation is limited. QM9 contains only small molecules, the polymer benchmark has just 553 labeled examples. The out-of-distribution evaluation considers rare combinations of properties that remain within the training property ranges, with target molecules drawn from the pretraining corpus. This provides limited evidence of generalization to different chemical structures or more challenging molecular design problems. The authors also do not report molecular novelty or overlap with the training data, making it difficult to assess whether the approach discovers genuinely new molecular structures.

Questions for authors
  1. I would like the authors to address my concerns regarding the reliability of the property evaluation. In particular, can they provide independent property evaluations, or at least held-out oracle accuracy, to support the reported improvements? Given the use of the same learned predictor for optimization and evaluation, I consider this important for assessing the paper's soundness.
  2. I would also like the authors to clarify the novelty of their approach relative to existing reward-weighted diffusion post-training methods, particularly RWR, and explain more convincingly the specific contribution of SDC. A controlled comparison against RWR and additional analysis of the structural distribution would be useful.
Reviewer_s1QLOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces GRAPHFDM, an online RL framework for graph diffusion that performs optimization through the forward process. The main experiments are conducted on QM9 and a polymer gas-permeability benchmark.

Critical strengths

The paper is generally easy to follow and understand. The method and experimental setup are presented clearly, and the appendix provides some useful details.

Critical weaknesses
  1. The reward-improvement and validity guarantees are derived for the exact reward-tilted teacher distribution, while the actual GRAPHFDM model only approximates this teacher through finite-sample weighted denoising and a learned structural controller. The analysis only shows that the model can recover the teacher-induced posterior marginals under realizability. It does not show that the trained generator matches the full teacher distribution or has the same improvement guarantee.

  2. GRAPHFDM uses a random-forest property predictor as the training reward oracle and also uses the same predictor as the primary evaluator for the reported property errors. The additional SVR and GP evaluators are trained on the same labeled data, and there is no independent ground-truth or experimental evaluation of the generated molecules. The reported improvement may just come from optimizing the surrogate model, rather than actually improving the molecular properties.

  3. The 10,000-molecule conditional benchmark is a subset of the molecules already used for unconditional pretraining, and the 200 OOD targets are also selected from the remaining pretraining molecules. Moreover, the selected rare property signatures already occur several times in the conditional training split. The evaluation is not really on fully unseen structures. It mainly tests rare target combinations that the model has already been exposed to before.

  4. In the shared-temperature case, the proposed teacher reduces to a standard reward-tilted distribution, while the main additional component of SDC is to make the graph-size distribution depend on the target condition rather than using the original fixed prior. However, the experiments do not include a simple baseline that directly learns a condition-dependent graph-size distribution without the proposed SDC formulation. The current ablation cannot determine whether the improvement comes from the proposed structured optimization or simply from replacing an inadequate fixed size prior with a condition-dependent one.

  5. The polymer benchmark contains only 553 conditional examples, but multi-seed robustness is reported only for the toy dataset QM9, and many competing methods use different backbones, optimization procedures, and computational budgets. The performance differences cannot be cleanly attributed to the proposed optimization.

Questions for authors

See weaknesses

Comments on the paper

2 comments

Thanks, the additional experiments substantially strengthen the paper and address several of my original concerns. I still have some reservations regarding the methodological novelty, particularly the overlap with RWR, and the limited scope of the experiments. However, I think the additional evidence provides a stronger justification for the proposed method.

A few remaining issues:

  • Proposition A.1 in Appendix A.2.8 appears to contain a sign error.
  • Several experiments presented in the rebuttal appear to be missing from the revised PDF. In particular, I could not find the supervised conditional-size baseline, fixed-teacher fitting experiment, or the detailed joint/separate KL analyses reported in the response. The held-out oracle errors and molecular-novelty results also seem to be absent. I'd appreciate seeing at least some of them in the paper.
  • The reward definition in Eq. (45) has been corrected, but the subsequent statement that all properties contribute equally after property-wise normalization, and the similar statement in Appendix A.4, do not accurately describe the polymer reward. These should be corrected as well.

Overall, I think the rebuttal addresses enough of my original concerns to warrant increasing my score from 2 to 3. I would appreciate clarification and correction of the remaining issues above.

AuthorsOpenReview ↗

Summary of Reviews and Author Responses

We really appreciate the Area Chair and reviewers for their time during this process. Below, we summarize the main concerns raised across the reviews and our responses. Changes in the revised manuscript are highlighted in blue.

1. Reliability of Property Evaluation

s1QL-W2, vFfY-W1/Q1, and 1aWq-W2/Q3.

  • Ground-truth evaluation (Table 14) On QM9, we verified improved target matching directly against ground truth. Even for those generated structures absent from both oracle fitting and pretraining, GraphFDM achieves 9.6%–16.7% lower ground-truth error than SFT-init, GraphGRPO, CoMole, and GDPO.
  • Separate reward and evaluation predictors (Tables 15–16). An RF fitted only on the conditional training split supplied rewards. Averaged over three seeds, GraphFDM reduced Avg. MAE relative to best baseline by 26.6%–34.0% across 3 frozen evaluators fitted on the full conditional dataset (RF$_B$, SVR, and GP).

2. Theoretical Guarantees and Approximate Teacher Fitting

s1QL-W1 and vFfY-W4.

  • Error transfer (Appendix A.2.8).
    • Corollary 3.4 guarantees reward improvement and valid support for the exact teacher. Proposition 3.1 connects that teacher to the training objectives for arbitrary model parameters.
    • New Proposition A.1 gives sufficient conditions for valid-conditioned reward improvement and a lower bound on the learned generator’s acceptance probability. Both depend on approximation errors.
  • Finite training. Fitting a fixed empirical teacher reduces NMAE by 4.5% and raises raw validity from 86.4% to 91.5%.
  • Finite sampling (Table 17). With the same number of rounds and epochs, increasing K from 8 to 16 to 32 lowers test NMAE from 0.445 to 0.419 to 0.394 .

3. Additional Analysis of SDC

s1QL-W4, vFfY-W3/Q2, and 1aWq-W1/Q2.

  • Supervised conditional-size baseline (Table 15 and 19). Full SDC achieves 22.2% lower polymer Avg. MAE than a trained conditional-size predictor under the same separated- oracle protocol, showing the additional gain from reward-guided size and structure optimization.
  • Budget allocation (Table 20). On the same candidate bank, separate budgets allocate more KL to refinement within each node count. After standardizing both generators to the same node-count distribution within each test condition, Separate KL retains 12.1% lower gas MAE than Joint KL.
  • Visualization of SDC effect (Appendix A.3.7, Figures 5–6). For representative targets, we compare the node-count distributions before the update, in the SDC teacher, and after fitting alongside candidate costs at each count. The teacher increased the probability of node counts whose candidates better matched the requested properties, and the fitted controller moved in the same direction. The final controller favored different node counts for different test targets.

In summary, GraphFDM introduces a new forward RL pipeline for graph diffusion that improves multi-property control while preserving chemical validity, addressing a key challenge faced by previous post-training methods for molecular graph generation. GraphFDM outperforms all evaluated baselines for target matching on both small-molecule and polymer tasks, with at least 96.0% raw validity.

We sincerely thank the Area Chair and reviewers for their thoughtful feedback and welcome any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.