acceptodds
Under review as a conference paper at ICLR 2027

SCORED: Selective OSDA Generation with COntrastive REwards from Denoising

54 views

Abstract

Scientific design often requires candidates that favor a target condition over plausible alternatives. Conditional generative models do not directly optimize this preference, while evaluating every candidate across multiple conditions is often too costly for online optimization. We introduce SCORED, a model-intrinsic selectivity reward that uses shared noise to contrast a conditional diffusion generator's denoising errors for the same candidate under target and competitor conditions. Scoring needs only forward passes: a six-condition panel takes 7.86 ms, against 8.3 s for a docking-based evaluator. We apply SCORED to generate organic structure-directing agents (OSDAs), molecules that guide zeolite framework formation. Mean-centered reward-weighted regression incorporates offline binding information, after which multi-objective reinforcement learning (RL) combines selectivity and molecular-quality rewards without physical evaluation in the online reward loop. Across three runs, SCORED reaches a mean competition score of kJ/mol Si over six held-out frameworks under frozen-pose force-field evaluation, where a recent LLM-based generator scores 9.72 (lower is better). Matched ablations that drop the competitor contrast (6.34) or the denoising reward entirely (6.50) are both worse, isolating the contribution of the target–competitor comparison. Extensions to structure-based drug design and metal–organic framework generation give uneven gains across models and domains, which we examine through pre-RL score–property alignment. These results support conditional denoising comparisons as inexpensive online feedback for selectivity optimization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
OSDA, diffusion, zeolite

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_4TWBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces an inexpensive selectivity reward for conditional diffusion generators. For the same candidate and shared corruption, SCORED subtracts the target-condition denoising error from a SoftMin over competitor-condition errors. This reward is used in an online RL pipeline initialized by reward-weighted regression, whose offline binding labels are mean-centered separately for each molecule. The main application is generation of organic structure-directing agents for six zeolite frameworks excluded from SFT, RWR, and RL training. Under the stated frozen-pose force-field evaluator, the complete pipeline obtains a mean competition score of 5.026 ± 0.334 kJ/mol Si over three runs, compared with 6.340 ± 0.031 without competitor contrast and 6.503 ± 0.110 without the denoising reward. Additional experiments study drug-design backbones and MOF generation, with mixed transfer results.

Critical strengths
  1. The central target-versus-competitor contribution is tested directly. Table 2 and Table A.2 keep the architecture, RWR initialization, molecular-quality channels, and generation settings fixed while replacing the complete margin with target-only denoising feedback or removing denoising feedback. The resulting competition scores, 5.026 versus 6.340 and 6.503, support a contribution from the contrast itself. Full SCORED is better on all three run-level means according to Appendix C.3. Removing RWR or freezing the scorer increases the score to 8.157 and 7.149, respectively. These controls make the main evidence more convincing than the comparison with OSDA Agent alone. I also appreciate that improved selectivity is evaluated after generation by a force-field protocol rather than simply by reporting the optimized denoising reward.

  2. The reward is simple, usable, and accompanied by unusually careful qualifications. Shared corruption makes the target/competitor comparison paired, and the log-odds identity in Appendix D explains precisely what the SoftMin margin measures. The paper explicitly avoids treating denoising error as physical energy or a calibrated likelihood. Table A.8 demonstrates a substantial reward-path saving, while Appendix F.5 distinguishes the timed six-condition, four-timestep configuration from the main four-condition, ten-timestep configuration. The authors also disclose the modest alignment correlations, the IPDiff non-improvement, and the absence of experimental crystallization evidence. These qualifications make the useful engineering contribution easier to assess.

Critical weaknesses
  1. The primary physical endpoint remains conditional on substantial, method-dependent filtering. Equation (A.3) excludes unavailable competitor energies, and Appendix C.5 requires successful evaluation of the target and at least one competitor. Table A.3 reports joint evaluation for 38.3% of generated outputs for Full SCORED, 37.5% without the denoising reward, and 15.6% without competitor contrast. Its caption states that these counts come from runs different from those used in the main result tables. Figure A.1 is a useful partial response: Full SCORED actually has slightly greater mean competitor coverage than OSDA Agent, so reduced coverage is not an adequate explanation of that comparison. Nevertheless, aggregate coverage does not establish the robustness of score differences within comparable coverage strata or across the three scored runs. A complete-panel analysis and per-run yields would clarify how much of the gain represents better generated candidates versus a difference in which candidates receive a score. This matters because the conditional mean competition score is not an end-to-end success rate, and no crystallization outcome is measured.

  2. Transfer beyond zeolites does not yet isolate the same selectivity mechanism with comparable rigor. Table A.11 provides a useful pocket-level paired bootstrap for TargetDiff, but the gain is small, +0.21 kcal/mol with a 95% interval of [0.01, 0.40], and the comparison removes the denoising reward rather than replacing it with target-only scoring. Target affinity also improves by 0.24 kcal/mol, leaving open whether competitor contrast supplies an additional benefit in this domain. Table A.13 reports an IPDiff change of −0.02 and an AliDiff change of +0.16 without comparable repeated-training uncertainty. The MOF experiment uses 80 outputs per arm and a surrogate-predicted absolute capacity endpoint. Appendix H.4 does not specify the target condition and competitor conditions used to construct its SCORED margin. These limitations are consistent with the authors' cautious descriptive framing, but they prevent the secondary experiments from strongly establishing general transfer of the target-relative mechanism.

Questions for authors
  1. Clarification of evaluation population. Can you report generation-to-evaluation counts for the actual three runs in Table A.2, and the main score comparisons restricted to candidates with all five competitors evaluated or stratified by competitor count? Does the improvement persist when conditional scores and physical-evaluation yield are reported together?
  2. Clarification of cross-domain specificity. For TargetDiff, does SCORED improve the selectivity gap over a matched target-only denoising reward, and how variable is the effect across independent RL runs? This would distinguish a contrast-specific effect from improved target affinity.
  3. Clarification of MOF implementation. What are the target and competitor conditions, scoring timesteps, temperature, and condition-sampling rule in Table A.14? How are failed CIF constructions handled in the reported capacity means?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 4TWB (revisions in orange)

We sincerely thank you for the careful reading of our controls, derivations, and scope qualifications, and for three diagnostic questions on funnel filtering, competitor-count coverage, and single-seed variability. We detail our response below point by point. Please kindly let us know if our response addresses the issues you raised in this paper.

W1 & Q1: Clarification of Evaluation Population: Full SCORED Leads in Physical-Validation Yield and Competition Score Stratified by Competitor Count

Follow your valuable suggestion, we replaced the earlier separate-run counts in Table A.3 with the actual three runs behind Table 2/A.2 (3000 generations each) and provide stratified comparison, and confirm that the improvement persists when conditional scores and physical-evaluation yield are reported together:

  • Per-run funnel for Full SCORED: each run generates 500500 candidates per framework across 66 frameworks, so N=3,000N = 3{,}000 raw outputs, and we provide the comparison of the percentage of molecules with valid competition score, Joint/Gen., in Table R2, where Full SCORED still leads in physical-evaluation yields.
  • Stratification by competitor count (Table A.4): Follow your valuable suggestion, we stratify the three-seed scores of Table A.2 by number of competitors found (ncomp∈{1,…,5}n_{\text{comp}} \in \{1, \ldots, 5\}) in Table R2, where ncomp=5n_{\text{comp}} = 5 is the complete panel. We observe that Full SCORED gets the best (lowest) competition score across all five strata. Gains are therefore not an artifact of differential competitor coverage.

Table R1 -- Per-seed Evaluation Funnel of the Actual Runs of SCORED (as reported in Table 2)

SeedValid ↑\uparrowUnique ↑\uparrowConf. ↑\uparrowTarget ↑\uparrowJoint JJ ↑\uparrowJoint / Gen. ↑\uparrow
02,8172{,}8172,1652{,}1652,0812{,}0811,2151{,}2151,1641{,}16438.838.8\\%
12,7132{,}7132,1182{,}1182,0392{,}0391,1901{,}1901,1321{,}13237.737.7\\%
22,7812{,}7812,2092{,}2092,1292{,}1291,1651{,}1651,1001{,}10036.736.7\\%
Mean / pooled2,7702{,}7702,1642{,}1642,0832{,}0831,1901{,}1903,3963{,}396 (sum)37.737.7\\%

Table R2 -- Comparison of Competition Scores per Stratum (lower is better) and Post-Validation Yields (higher is better)

Full SCORED− competitor contrast− denoising reward− RWR initialization
ncomp=1n_{\text{comp}}=1 ↓\downarrow3.466±0.1543.466 \pm 0.1544.114±0.9244.114 \pm 0.9243.641±0.4653.641 \pm 0.4653.915±1.6123.915 \pm 1.612
ncomp=2n_{\text{comp}}=2 ↓\downarrow4.372±0.5624.372 \pm 0.5624.410±0.5864.410 \pm 0.5864.780±0.0314.780 \pm 0.0314.387±0.1174.387 \pm 0.117
ncomp=3n_{\text{comp}}=3 ↓\downarrow4.908±0.4524.908 \pm 0.4526.187±1.1346.187 \pm 1.1345.561±0.3475.561 \pm 0.3475.655±1.2235.655 \pm 1.223
ncomp=4n_{\text{comp}}=4 ↓\downarrow5.213±0.2065.213 \pm 0.2066.313±0.4396.313 \pm 0.4395.711±0.4315.711 \pm 0.4317.489±0.3767.489 \pm 0.376
ncomp=5n_{\text{comp}}=5 ↓\downarrow (complete panel)5.846±0.3945.846 \pm 0.3946.684±0.1256.684 \pm 0.1256.106±0.0616.106 \pm 0.0618.792±0.2678.792 \pm 0.267
Joint/Gen. ↑\uparrow37.7%15.6%36.9%28.0%

W2 & Q2: Clarification of Cross-Domain Specificity with Multi-Seed SBDD Experiments Supporting Our Generalization Hypothesis and Comparison with Target-only

Follow your valuable suggestion to provide a Target-only ablation for TargetDiff, we repeated over three seeds with three matched experiments: Full SCORED (contrast + QED/SA), Target-Only (negative target denoising error −dθ(x,ctgt;t,ϵ)-d_\theta(x, c_{\text{tgt}}; t, \epsilon) + QED/SA, dropping the off-target term), and Without SCORED (QED/SA only) (§5.4, Appendix H.3, Tables A.17 to A.18) and observed the following:

  • TargetDiff:
    • Comparing with without SCORED or Target-only, the difference is likewise indistinguishable with the same gain of 0.060.06 kcal/mol. Because TargetDiff lacks pre-RL alignment (τb=−0.063\tau_b=-0.063), neither the target-only error nor the target-competitor contrast carries a valid signal, which directly supports our conditional generalization claim.
  • AliDiff vs. IPDiff (§5.4, Appendix H.4, Table A.19):
    • IPDiff (τb=−0.021\tau_b = -0.021) observed mixed results with mean gain of −0.021-0.021.
    • AliDiff, which finetunes on top of IPDiff for affinity alignment (τb=+0.126\tau_b=+0.126), improved in all three seeds with mean gain of +0.198+0.198.
  • Together, the three-seed replication of SBDD generalization experiments supports our generalization hypothesis where SCORED works on backbones with positive pre-RL alignment τb\tau_b.

Table R3 -- Three-seed TargetDiff Results (kcal/mol)

Seed / SummaryPocketsWithoutTarget-OnlyFullΔ\Delta (Full −- Target-Only, 95% CI) ↑\uparrowΔ\Delta ↑\uparrow(Full −- Without, 95% CI) ↑\uparrow
2026 (Table 3 run)302.212.212.252.252.422.42+0.17+0.17 [+0.01,+0.33][+0.01, +0.33] (20/30 wins)+0.21+0.21 [+0.01,+0.40][+0.01, +0.40] (23/30 wins)
2027292.432.432.292.292.382.38+0.10+0.10 [−0.04,+0.24][-0.04, +0.24] (20/29)−0.04-0.04 [−0.18,+0.09][-0.18, +0.09] (15/29)
2028292.352.352.462.462.362.36−0.09-0.09 [−0.22,+0.04][-0.22, +0.04] (10/29)+0.01+0.01 [−0.09,+0.11][-0.09, +0.11] (13/29)
3-seed pooled292.362.362.362.362.412.41+0.06+0.06 [−0.03,+0.15][-0.03, +0.15] (19/29)+0.06+0.06 [−0.03,+0.14][-0.03, +0.14] (16/29)

Table R4 -- Selectivity ΔVina\Delta \text{Vina} Gain of AliDiff and IPDiff (kcal/mol, mean$\pm$std over three seeds)

BackboneWithout SCOREDWith SCOREDΔVina\Delta \text{Vina} ↑\uparrow
AliDiff (τb=+0.126\tau_b = +0.126)2.49±0.082.49 \pm 0.082.69±0.112.69 \pm 0.11+0.198±0.034+0.198 \pm 0.034
IPDiff (τb=−0.021\tau_b = -0.021)2.57±0.082.57 \pm 0.082.54±0.042.54 \pm 0.04−0.021±0.048-0.021 \pm 0.048
Reviewer_5jWHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper makes three claims. First, a diffusion model can judge its own candidates for selectivity without any external simulation: the score is how much better the model denoises a molecule when conditioned on the target framework than when conditioned on competing frameworks, using the same noise draw for each. The evidence here is thin, resting mostly on the construction itself plus a correlation check against the true selectivity metric that is positive but weak, and measured on only a small subset of frameworks. Second, fine-tuning on that score produces more selective molecules. Third, the score is far cheaper than a physics-based reward.

Critical strengths

The paper focuses on a rather important problem, finding computationally feasible and accurarte rewards for RL post-training for molecular design. The cost reduction is the one claim that is clearly established by the papers benchmarks, and designing for relative preference rather than affinity is a good objective for problems where competing phases or off-targets determine the outcome.

Critical weaknesses

The selectivity reward carries no information beyond what the pretrained model already encodes, so optimizing it can only sharpen the model’s existing implicit classifier. Given the weak correlation with the "ground truth" metric, that signal is likely too poor to allow comparison with the actually competitive methods for diffusion guidance. The empirical case for the quality claim is also quite thin:

Questions for authors

Does the reward need to be in the weights at all? Suppose you were to apply the same target-versus-competitor contrast as a sampling-time guidance gradient, with no RWR and no RL. If that matches the fine-tuned model, the pipeline is unnecessary, this is a good ablation.

What happens to the reward’s agreement with ground truth as you optimize against it?

How is the run shown in Tables 1 and A.5 selected?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 5jWH (revisions in teal)

We sincerely thank you for recognizing the value of computationally feasible rewards and of optimizing relative preference. We detail our response below point-by-point. Please kindly let us know if our response addresses the issues you raised in this paper.

W1/Q1: Does the reward need to be in the weights at all (vs. sampling-time guidance)? Is SCORED just sharpening the implicit classifier?

We respectfully clarify that our pipeline does not optimize for (1) the selectivity reward alone, but also combining information beyond what the pretrained model already encodes, including (2) offline physical grounding via RWR and (3) online quality rewards (molecule quality rewards such as validity, novelty, SCSCore, which are not differentiable), as detailed in Line 260-262 of the submission (Line 292-295 in the revised manuscript). The combination of the three allows online RL to jointly optimize for quality and selectivity, not just sharpen the initial implicit classifier:

  • The online quality rewards (3) are the key to driving strong chemical quality (Tables 2 & A.3):
    • The four online external quality rewards (validity, SCScore, novelty, diversity) provide a strong learning signal on their own: even with zero denoising selectivity reward (− denoising reward, Table 2 and Table R1), RWR + those online quality rewards moves the generator onto a valid, feasible manifold, generating OSDAs with similar molecular quality compared to Full SCORED, while adding the selectivity reward (1) is responsible for substantially improving the competition score.

Table R1 -- Effect of molecular quality rewards

Comp. ↓\downarrowValidity ↑\uparrowNovelty ↑\uparrowDiversity ↑\uparrowSCScore ↓\downarrow
Full SCORED5.02692.340.72010.88921.908
Molecular Quality Rewards Only6.50391.260.71180.88381.894
  • Sampling-time guidance gradient cannot optimize for molecular quality and hurts it.

    • Follow your valuable suggestion to test the necessity of training instead of sampling time guidance, we conduct additional experiment in Table R2 using the same target-versus-competitor contrast as a sampling-time guidance gradient, it shows that sampling guidance hurts molecular quality, since the number of molecules that can actually dock to the target framework decreases. Though the competition score seems to be decreasing for SFT, it is at the cost of much less dockable molecules, making it merely a survivorship bias not genuine performance gain.
    • Even on top of RWR, sampling-time guidance (RWR+Guidance) yields only 2.2% dockable molecules and fails to produce valid competition scores.
    • Instead, SCORED combines offline physical grounding (2), online selectivity rewards (1), and online quality rewards (3), re-calibrating the implicit preference onto the valid, synthesizable, dockable manifold while optimizing for selectivity at the same time.
  • Independent quantum-chemical proof of physical gain (Table A.7, Figure A.2):

    • As shown in Table R3, on three external evaluators (MACE-MP-0 + D3, GFN2-xTB, and DFT (PBE + D3)), Full SCORED outperforms the no-denoising-reward ablation on all frameworks, confirming genuine physical selectivity improvement rather than sharpening its own bias.

Table R2 -- Effect of Gradient Guidance Rather Than Optimization

CheckpointComp. ↓\downarrowValid. (%) ↑\uparrowSCScore ↓\downarrowDockable ↑\uparrow
SFT7.197.1991.791.72.5472.54719.4419.44\\%
SFT +Guidance5.735.7390.090.02.6492.64911.6711.67\\%
RWRN.A.84.484.43.0913.0913.893.89\\%
RWR + GuidanceN.A.82.882.83.1323.1322.222.22\\%
Full SCORED5.035.0392.392.31.9081.90837.737.7\\%

N.A. for Comp. of RWR experiments since over half of frameworks yield generations containing neither docked target nor competitor.

Table R3 -- Independent Evaluator Results

EvaluatorFull SCORED ↓\downarrow− denoising reward ↓\downarrowMolecule-level gap (95% bootstrap CI) ↑\uparrowFrameworks won ↑\uparrow
DREIDING5.265.266.046.04+1.00+1.00 [+0.16,+1.84][+0.16, +1.84]6 / 6
MACE-MP-0 + D35.675.676.046.04+0.53+0.53 [−0.09,+1.16][-0.09, +1.16]6 / 6
GFN2-xTB5.375.375.785.78+0.48+0.48 [+0.12,+0.86][+0.12, +0.86]6 / 6
DFT (PBE + D3)5.375.375.825.82+0.55+0.55 [+0.11,+0.98][+0.11, +0.98]6 / 6

Q2. What happens to the reward’s agreement with ground truth as you optimize against it?

We appreciate this insightful question; following your valuable suggestion, we evaluated the scorer every 50 steps from RWR initialization (step 0) to full SCORED (step 500), with DREIDING as ground truth. As shown in Appendix E.3 and Figure A.3, SCORED helps preserve the scorer’s agreement with ground truth under optimization.

  • On a fixed cohort, which removes any distribution-shift confound, quartile AUROC with SCORED stays stable within 0.79–0.88 across all 500 steps, while the same measurement without SCORED falls from ~0.90 to ~0.75 (Figure A.3d).
  • Because that alignment remains, improving the reward genuinely improves ground truth.
    • The ground-truth objective itself indeed improves over training: GULP competition score moves from 6.67 to 4.70 kJ/mol Si (Figure A.3b). It rises during roughly the first 100 steps, while the policy explores, then falls well below its starting value.
    • This preserved alignment, τb=0.209\tau_b = 0.209 against the competition score and +0.306+0.306 against the competition energy gap (corresponds to ∼60\sim 60\\%\text{--}65\\% pairwise ranking accuracy that is more important in RL training), has been demonstrated to be sufficient for RLHF reward models to improve constantly in the prior literature:
      • InstructGPT[2] (NeurIPS, 2022) finetunes a 1.3B model to outperform 175B GPT with a reward model at 70% accuracy;
      • In Figure 2 of [3] (NeurIPS, 2025), a reward model with 61.6% ranking accuracy yields the best gain comparable with training using oracle reward.
Reviewer_fXqiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SCORED for OSDAs for zeolites. It addresses the problem that conditional generative models do not ensure that a candidate prefers the target condition over other reasonable alternatives. The method adds the same noise to a candidate and compares the diffusion model’s denoising errors under the target and competing conditions. The difference between these errors is used as a selectivity reward. The method first uses reward-weighted regression to learn from offline binding energy data, with rewards centered by subtracting the mean for each molecule. It then jointly optimizes the selectivity reward and a molecular quality reward. Computing the selectivity reward only requires forward passes. The authors report that it is about 1,056 times faster than a docking-based proxy evaluator, allowing its use in online reinforcement learning. On six held-out zeolite frameworks, SCORED achieves better competitive scores than OSDA Agent.

Critical strengths
  1. This paper uses the model itself to compare the target and competing conditions, and well-designed experiments support its effectiveness. The contribution is to use the generator’s own conditional denoising errors as a selectivity signal, without external docking or a separately trained affinity predictor. The competitor contrast ablation replaces the reward with the denoising error under the target condition alone while keeping everything else fixed. This leads to clearly worse competitive scores, showing that this component is effective.
  2. The proposed reward is fast and stable enough to make the selectivity signal useful for online reinforcement learning. Forward scoring reduces the computational cost by three orders of magnitude compared with a docking-based proxy evaluator. Using shared noise also reduces variance, with a clear drop in the estimator’s standard deviation across all diagnostic settings. The difference between conditions is much smaller than the random variation in individual denoising errors, so the signal is lost without paired noise. This variance reduction makes the speedup useful in practice.
Critical weaknesses
  1. The lack of a working expensive baseline makes it difficult to test the main claim that the method is both cheap and good enough. The paper argues that physics-based evaluation is too expensive and should therefore be replaced with a signal from the model itself. However, the only expensive reward baseline fails completely. In Table A.6, when the binding energy proxy is used as the reward, only six of the 3,000 generated molecules produce dockable conformations. All six fall within an unreasonable energy range, and none pass the physical validity filter, so no competitive score can be reported. This baseline also has much higher average molecular weight and number of rotatable bonds than the other settings. This pattern suggests reward hacking and could reflect poorly balanced quality rewards or an unsuitable training budget, rather than a fundamental problem with the proxy reward. Readers therefore cannot tell whether SCORED is both cheap and effective, or simply cheap. A more useful comparison would give both methods the same wall-clock budget, allow the expensive reward fewer training steps, and compare their final competitive scores. Without this comparison, the reported speedup establishes faster scoring but does not show whether it leads to better results for the time spent.

  2. The training labels and evaluation metrics use the same physical approximation, leaving open the possibility that the method learns its systematic biases. The offline binding energy labels used for RWR come from DREIDING calculations with fixed conformations. The final competitive scores also use fixed-conformation DREIDING calculations with Voronoi docking. Although all six evaluation frameworks are excluded from every training stage, the force field and treatment of conformations remain the same. Offline binding energies first enter the model through RWR. SCORED then uses the resulting model parameters to compute rewards, RL adjusts generation based on these rewards, and the final molecules are assessed using the same physical approximation. Systematic errors in the force field could therefore appear in both the training signal and the evaluation metric, making the results look stronger than they are. This is especially concerning because RL can exploit weaknesses in a scoring function. The paper does not address this possible source of bias or provide experimental synthesis validation.

Questions for authors

Q1. For the proxy reward baseline in Table A.6, were the quality reward weights and number of training steps exactly the same as those used for Full SCORED? The clear shift toward higher molecular weight and more rotatable bonds could reflect a configuration issue rather than a failure of the proxy reward itself. Could you add a comparison under the same wall-clock budget, allowing the proxy reward baseline fewer RL steps so that its total training time matches SCORED, and report its final competitive score?

Q2.What fraction of the offline binding energy labels used for RWR comes from the same fixed-conformation DREIDING protocol used in the final evaluation? Does the combined dataset contain labels obtained using other methods or experiments? Could you also plot both the internal SCORED reward and the physics-based competitive score against the number of RL steps?

Q3. Could you recalculate the binding energies of the top-N candidates from Full SCORED and at least one ablation using an independent evaluator, such as a different force field, allowing conformational relaxation, or DFT single-point calculations for a small subset?

8 comments
AuthorsOpenReview ↗

Response to Reviewer fXqi (revisions in purple)

We sincerely thank you for the positive assessment of our ablation and variance-reduction analysis and for three concrete and clear questions. We detail our response below point by point. Please kindly let us know if our response addresses the issues you raised in this paper.

W1 & Q1: Is SCORED Both Cheap and Effective? Matched Wall-Clock Comparison with a Direct GULP Reward Provides a Strong Baseline Yet Still Falls Behind Full SCORED

  • Hyperparameter of surrogate reward ablation:
    • We do not adjust any hyperparameters for the surrogate ablation in the original manuscript, i.e., the quality reward weights and the number of training steps are exactly the same with Full SCORED.
    • We agree that the reward hacking behaviour may source from misconfiguration and we provide additional experiment per your insightful comment in Table R1 with details as follows.
  • SCORED is both cheap and effective
    • Follow your valuable suggestion to provide a strong baseline from an expensive reward, we conduct additional experiment directly uses DREIDING-calculated competition as selectivity reward that is free from any surrogate bias in Table R1 and Table. A10.
    • With only 12 steps of RL using such oracle reward, it does not suffer from reward hacking behavior as the previous surrogate reward ablation and outperforms other ablations, providing a very strong baseline. Yet, Full SCORED is still 0.8820.882 kJ/mol Si better (2.6×2.6\times run-level SDs) with lower SCScore and higher overall scores (in harmonic means Hsel/HeqH_{\text{sel}}/H_{\text{eq}}).

Table R1 -- RL with Oracle Selectivity Reward Signal

MethodComp. ↓\downarrowValid. ↑\uparrowNov. ↑\uparrowDiv. ↑\uparrowSCScore ↓\downarrowHselH_{\text{sel}} ↑\uparrowHeqH_{\text{eq}} ↑\uparrow
Oracle reward RL (12 steps, 4.5 h)5.9085.908 (+0.882+0.882)91.93%91.93\%0.7980.7980.9230.9232.5492.5490.8650.8650.8730.873
Full SCORED (500 steps, 3 seeds)5.026±0.3345.026 \pm 0.33492.34%±1.76%92.34\% \pm 1.76\%0.720±0.0150.720 \pm 0.0150.889±0.0110.889 \pm 0.0111.908±0.0801.908 \pm 0.0800.9640.9640.9440.944

W2 & Q2: RWR Label Fraction, Reward and Competition Score during RL Training

  • Fraction of DREIDING in RWR labels (§5.1, Appendix C.5):
    • 100% of binding-energy labels in the merged corpus [1, 2, 3] come from DREIDING calculations using frozen-pose method.
  • Reward and Physics-based Competition Score during RL training:
    • Follow your valuable suggestion, we visualize the trend of both the reward and competition score during training in Figure A.3, and observe the following trends:
      • In the early steps, reward increases sharply while hurting the physical competition, suggesting an exploration phase;
      • The reward then decreases sharply again while optimizing physical competition, balancing exploration and exploitation;
      • In the late stage, the reward steadily increases and the physical competition gradually decreases (lower is better), leading to significant improvement at the end.
Reviewer_rAHKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

The paper introduces the SCORED framework to design organic structure-directing agents (OSDAs) that can selectively bind to a target zeolite framework among competing frameworks. Its main contributions to methodology are

  1. A model-intrinsic selectivity reward that compares the denoising error of a conditional diffusion model for the same candidate under target and competing conditions. This provides instant feedback during online optimization for the selectivity and can be combined with molecular validity, synthetic complexity, novelty, and diversity rewards in reinforcement learning.
  2. A binding-informed warm initialization of the model that utilizes reward-weighted regression (RWR) to establish conditional preferences on the binding energy. The authors carry out ablation studies on OSDA design and transfer analysis to structure-based drug design and MOF design scenarios. The main result indicates that the model-intrinsic selectivity reward can provide informative and cheap feedback for selectivity optimization when the underlying model already encodes relevant physical-property information.
Critical strengths

The scientific problem is important. Selectivity is an essential but usually ignored objective in binder design. The authors not only address this caveat in the templated synthesis of zeolite materials, but also show the potential in other host-guest systems, such as protein-ligand binding and gas capture in MOFs. As a model-agnostic framework, it is flexible enough to inspire more downstream applications and task-specific designs.

Critical weaknesses
  1. The contribution and novelty are not well validated or represented.
  • Binding-informed RWR is not a strong point of novelty, as the authors have acknowledged a similar idea in AliDiff compared to IPDiff. Given this fact, the contribution of this work looks more marginal.
  • The 1056× reduction in cost is not a fair comparison. A docking-based evaluator or a surrogate docking model has higher confidence than the generative model itself and can always be used as a post-validation step after generation. The more important question is whether the selectivity gain by SCORED is reliable enough to reduce the post-validation in a real design decision.
  • In Section 5.4, from a chemistry point of view, a selectivity increase of 0.21 or 0.16 kcal/mol doesn't make any sense in drug discovery; from a statistics point of view, it is hard to tell how this increase compares to the fluctuation in the generated samples.
  • In Section 5.4 and relevant appendices, the experimental settings of the MOFFUSION-H${}_2$ are very unclear. The gaps include the definition of the competitor conditions and selectivity, and an explanation of the "matched reward toggle".
  1. The manuscript is poorly written and hard to follow.
  • RWR happens logically before SCORED, but SCORED is written first.
  • Missing references for AutoDock Vina. MMFF and MOFFUSION-H${}_2$ are neither cited when they first appear.
  • Some wording is confusing or misleading.
    • The paragraph caption "From target confidence to selectivity" in Section 4.1 refers to the "conditional compatibility" as a "confidence". This is indeed a new statement rather than a rephrasing.
    • Readers have to infer from the context that MθM_\theta is called "margin" in Section 4.1, and RoffR_\mathrm{off} is called "utility" in Section 4.2. The "utility" term appears with a different meaning again in Section 5.1.
    • The authors use the words "outer iteration" in Section 4.3, but present no clean algorithm or scheme to clearly point out what it is.
  • The unit, kJ/mol Si, needs to be explained to a broader audience.
  • There is no need to explain Coff\mathcal C_\mathrm{off} at the end of the "Shared-corruption scoring" paragraph of Section 4.1, because it is not mentioned in this paragraph but has been explained in the previous one.
  • In Section 5.3, "a learned surrogate" should be further explained as the learned binding energy surrogate model as a reward.
Questions for authors
  1. Why does the denoising score imply "conditional compatibility"? Do the models in Section 5.4 with a low score-property correlation undermine the generalizability of your hypothesis?
  2. How can we stop the model from enhancing its own bias in the model-intrinsic selectivity reward?
12 comments
AuthorsOpenReview ↗

Response to Reviewer rAHK (revisions in blue)

We truly appreciate your constructive comments and recognition of the importance of our work as well as the flexibility of our method. We detail our response below point by point. Please kindly let us know if our response addresses the issues you raised in this paper.

W1.1: Positioning of the proposed SCORED Relative to AliDiff

We respectfully clarify that our contribution is neither binding-informed RWR nor an AliDiff-style alignment; it is orthogonal to both.

  • RWR and AliDiff belong to the same class: offline warm-start alignment techniques. As noted in our submitted version and recognized by the reviewer, both align the model's score with the target affinity using a fixed/pre-labelled dataset, before the generative loop runs.

    • Following the reviewer's feedback, we move the generic RWR formulation into Preliminaries (Sec. 2, Eq. 2).
    • AliDiff (Sec. 3.2 [1]) made this clear: evaluating reward values such as binding affinity is computationally expensive and we instead resort to aligning with a labeled offline dataset.
  • SCORED as our core contribution is an online RL selectivity signal, computed inside the loop from the denoiser’s own target-vs-competitor error. Because this signal is self-generated and cheap, SCORED first enables selectivity to be optimized online without the expensive physical evaluators that confine prior work (including AliDiff [1]) to offline training.

  • Offline alignment and the online signal are orthogonal and complementary: Offline alignment is the prerequisite; the online signal is what converts it into selectivity.

    • Offline warm-start alone does not translate into selectivity, as shown in (1) Table 1, where RWR+SCORED improves over RWR only by 70%/48% on AFI/LTA OSDA frameworks; and (2) Table R1 below, where AliDiff+SCORED outperforms AliDiff alone on SBDD generalization.
    • In turn, however, the online signal alone has nothing to act on. As shown in Table R1, on unaligned IPDiff backbone, SCORED yields no gain (2.57 →\rightarrow 2.54).
    • Moreover, offline alignment does not automatically converts to selectivity and it may actually trade selectivity for affinity. As shown in Table R1, AliDiff performs offline alignment on top of IPDiff but its selectivity drops (2.57 →\rightarrow 2.49).
    • Hence, selectivity emerges only when both are present, and it does so regardless of which offline method fills the warm-start slot.

    Table R1 -- SBDD Generalization Results (Vina selectivity gap, kcal/mol, higher is better)

    BackboneWithout SCOREDWith SCOREDΔVina\Delta \text{Vina} ↑\uparrow
    IPDiff (τb=−0.021\tau_b = -0.021)2.57±0.082.57 \pm 0.082.54±0.042.54 \pm 0.04−0.021±0.048-0.021 \pm 0.048
    AliDiff (τb=+0.126\tau_b = +0.126)2.49±0.082.49 \pm 0.082.69±0.112.69 \pm 0.11+0.198±0.034+0.198 \pm 0.034

W1.2: Fairness of the 1,056× Cost Comparison and Post-Validation Yield

We completely agree that physical docking remains a necessary post-validation step in any real discovery campaign, and we have clarified in the text that the 1,056× speedup (7.86 ms vs. 8.3 s; Table A.12) measures only the cost of the reward signal during online training, not end-to-end discovery cost.

To fully address the reviewer's concern whether the selectivity gain by SCORED is reliable enough to reduce the post-validation in a real design decision, we account for the end-to-end discovery, including (1) the online training spent once and (2) the generations plus the post-validation they trigger, and measure SCORED's reliability in each stage.

  • We answer whether SCORED is reliable enough to reduce the post-validation (evaluating the cost of stage (2) to achieve a fixed reliability) by comparing Gen./100: the expected number of raw generated candidates to obtain 100 qualifying OSDAs.

    • A candidate OSDA counts qualifying only if Voronoi docking + GULP confirm it: distinct, novel, binding the target within a physically reasonable energy range, and binding the target more strongly than any competitor framework.
    • Gen./100 measures post-validation cost because, to obtain 100 molecules of the same validated quality, a less reliable method must generate more candidates and post-validate every one.
    • Across the six held-out frameworks (Table 2; Table R3), SCORED outperforms matched-time Oracle RL, while cutting the post-validation docking workload by 1.39×–3.02× relative to all ablations.
    • On AFI and LTA (Table 1; Table R4), SCORED needs 2.7–8.3× smaller Gen./100 than SFT and RWR. Despite comparable Gen./100 on LTA with OSDA Agent, SCORED requires no in-loop docking during generation that OSDA Agent struggles, with substantially higher HselH_{\text{sel}} and HeqH_{\text{eq}}.
  • We answer whether SCORED as a signal is reliable (evaluating the reliability of generations given a fixed budget of (1)) by comparing against an oracle RL baseline rewarded directly by ground-truth DREIDING competition scores (with no surrogate error at all) under an identical 4.5h wall-clock budget on 1 A100.

    • Given the exact same wall-clock budget, the oracle completes only 12 policy updates and reaches 5.908 kJ/mol Si (Table R2).
    • In the same 4.5 hours SCORED enables 500 updates, reaching a markedly better competition score (5.026 ± 0.334) kJ/mol Si, accompanied by even higher synthetic accessibility (SCScore) and trade-off scores (HselH_{sel} and HeqH_{eq}).
    • Thus, reliability is set by exactness per update and update count together, not by exactness alone: the oracle maximizes the first and manages only 12 updates in 4.5h.

    Table R2 -- RL with oracle selectivity

    MethodComp. ↓\downarrowValid. ↑\uparrowNov. ↑\uparrowDiv. ↑\uparrowSCScore ↓\downarrowHselH_{\text{sel}} ↑\uparrowHeqH_{\text{eq}} ↑\uparrowGen./100 ↓\downarrow
    Oracle RL (12 steps, 4.5 h)5.9085.908 (+0.882+0.882)91.9391.93%0.7980.7980.9230.9232.5492.5490.8650.8650.8730.8732,0982{,}098
    SCORED (500 steps, 3 seeds)5.026±0.3345.026 \pm 0.33492.3492.34\\% \pm 1.76\\%0.720±0.0150.720 \pm 0.0150.889±0.0110.889 \pm 0.0111.908±0.0801.908 \pm 0.0800.9640.9640.9440.9441,6641{,}664

    Table R3 -- Gen./100 comparison of ablations (lower is better)

    SCOREDOracle RL−- denoising reward−- RWR−- competitor contrastfrozen scorer
    Gen./1001,6642,0982,3084,3064,9725,028

    Table R4 -- Gen./100 comparison of baselines on AFI and LTA (lower is better)

    FrameworkSCOREDSFTRWROSDA Agent
    AFI505238141671500
    LTA75821742083750

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision note

We sincerely thank the Area Chair and the reviewers for their valuable efforts in reviewing our paper. To facilitate tracking of the changes made in the revision, we provide the revision notes as follows (in the revised PDF, new or modified text is color-coded by reviewer: blue for rAHK, purple for fXqi, orange for 4TWB, and teal for 5jWH; newly added tables shifted some appendix table numbers, e.g., Tables A.6, A.9, and A.20 correspond to Tables A.5, A.6, and A.14 in the initial submission):

Independent physical evaluation: add frozen-pose method using MACE-MP-0 + D3, GFN2-xTB, and DFT (PBE + D3) on 81 shared candidates generated by Full SCORED and −- denoising reward ablation in Section 5.2, Appendix C.5, Table A.7, and Figure A.2. (Reviewer fXqi W2&Q3, Reviewer rAHK Q2, Reviewer 5jWH W1)

Matched wall-clock and surrogate baselines: add a wall clock training-budget matched RL with oracle selectivity reward comparison as a strong, expensive baseline; clarify the 500-step pose-based MLP surrogate collapse behavior in Sections 5.2–5.3, Appendix F.2, Table A.9, and Table A.10. (Reviewer fXqi W1&Q1, Reviewer rAHK W1.2)

End-to-end yield and evaluation funnel: add Gen./100 (expected number of generations per 100 hits of novel, distinct, binding, and selective OSDAs) as an end-to-end yield metric to Tables 1, 2, and A.10, and update Table A.3 (Appendix C.3) with actual per-run funnel counts across the three repeated runs. (Reviewer fXqi W1, Reviewer rAHK W1.2, Reviewer 4TWB W1&Q1)

Competitor-coverage control and seed identity: add ncompn_{\text{comp}}-stratified competition scores across all strata including complete five-competitor panels in Section 5.2, Appendix C.3, and Table A.4.

Scorer stability across RL steps: plot the training reward, DREIDING-calculated competition score, and extreme-quartile AUROC every 50 RL steps on both online-generated and fixed-cohort molecules (with RLHF τb\tau_b context) in Section 5.3, Appendix E.3, and Figure A.3. (Reviewer 5jWH W1&Q2, Reviewer fXqi W2&Q2, Reviewer rAHK Q2)

Sampling-time guidance ablation: add shared-noise DDIM gradient guidance (s∇ztRs\nabla_{z_t}R) experiments on frozen SFT and RWR checkpoints in Section 5.3, Appendix C.6, and Table A.8. (Reviewer 5jWH Q1)

Multi-seed SBDD replications: add three-seed TargetDiff replications with a matched target-only control (Appendix H.3, Tables A.17–A.18) and three-seed AliDiff/IPDiff evaluations (Appendix H.4, Table A.19) in Section 5.4. (Reviewer rAHK W1.3, Reviewer 4TWB W2&Q2)

MOFFUSION-H2 specification and scope framing: add complete MOFFUSION-H2 hyperparameters, target/competitor condition definitions, and evaluation details (all-80 outputs vs. CIF construction rate) in Section 5.4 and Appendix H.5 (Table A.20), and frame cross-domain transfer as a descriptive hypothesis in Sections 1 and 5.4. (Reviewer rAHK W1.4, Reviewer 4TWB W2&Q3)

Method organization: move generic RWR to Section 2 (Eq. 2), clarify conditional compatibility, MθM_\theta, RoffR_{\text{off}}, and outer iteration ss in Sections 4.1–4.3, and remove the redundant Coff\mathcal{C}_{\text{off}} definition. (Reviewer rAHK W1.1&W2)

Units, terminology, provenance, and citations: define kJ/mol Si at first mention (Sections 1 and 5.1, Appendix A.1), rename utility to normalized score (Section 5.1, Figure 3(b), Appendix G.2), specify that the surrogate scores docked poses (Abstract, Section 5.3), clarify DREIDING label provenance (Section 5.1), and add MMFF, AutoDock Vina, and MOFFUSION citations at first mention (Sections 5.1 and 5.4). (Reviewer rAHK W2, Reviewer fXqi W2)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.