acceptodds
Under review as a conference paper at ICLR 2027

AVID: Action-Value Guidance and Identity-Aware Learning for Candidate-Set Molecular Discovery

Abstract

Molecular optimization rewards individual molecules, but a finite-budget discovery campaign seeks a growing set of distinct, high-quality candidates. Repeatedly generating a high-scoring molecule can maintain high per-output reward without adding a candidate. We introduce AVID, which couples action-value guidance during generation with identity-aware learning after completion. A frozen, archive-trained prefix-action guide biases token selection toward high-reward continuations. Once a molecule is complete, an identity ledger assigns within-run repeats zero weight in the direct policy loss while preserving cached rewards and the optimizer's other learning terms. Across all 23 tasks of the Practical Molecular Optimization (PMO) benchmark and five seeds per task, AVID acquires an average of 7,114.1 distinct candidates meeting fixed archive-derived reward thresholds within 32,768 generation attempts and at most 10,000 new oracle calls. This is 45.5% more than Augmented Memory, with higher task means on 18 of 23 tasks. Component studies distinguish the two interventions: adding guidance to identity weighting improves mean and top-10 reward at 512 newly scored identities on every task, while long-horizon experiments expose the trade-off between candidate quality and acquisition. Guidance can raise the qualifying fraction yet reduce distinct yield if new identities arrive more slowly. These results show that reward quality and qualifying-set growth provide complementary views of molecular optimization under finite budgets.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular optimization, goal-directed molecular generation, reinforcement learning, action-value guidance, identity-aware learning, candidate-set discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uB5NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AVID, a two-stage RL framework for molecular discovery: a pre-scored archive guides prefix generation, followed by an identity-weighted PPO update. Across 23 tasks the authors report large improvements in candidate yield.

Critical strengths

The evaluation scale is unusual for this area and the paper honestly reports unfavorable results such as reduced pairwise fingerprint diversity.

Critical weaknesses

The main comparison is not like-for-like: AVID receives task-specific offline reward supervision while the baselines have no equivalent information, so the gains cannot be attributed to the proposed method.

Beyond that, the guided PPO objective is non-standard and updates may be clipped before taking effect; there is no ablation with the archive alone and without identity weighting to isolate the second component, and the headline metric H_32768 is coupled to a specific archive quantile threshold tau.

Questions for authors

Can the baselines be given equivalent offline reward information? How often are updates clipped in practice? Can you report the sensitivity of H_32768 to tau, together with paired significance tests?

3 comments

Thank you for your careful reading and thoughtful feedback. Below, we respond to your questions point by point and share the results of our additional analyses.

Q1: Access to offline reward information. When we charge the full archive on the saved runs, AVID finds 2,251 more qualifying molecules per task than standard PPO (Holm-adjusted p = 0.00609) and covers 1,476 more qualifying Bemis-Murcko scaffolds. Across 23 tasks and five paired seeds, we apply the same caps of 10,000 total labels and 32,768 proposals counted by the identity ledger. AVID pays for all 2,048 offline scores, leaving at most 7,952 new online scores; a baseline pays for an archived score the first time it uses it. The table gives the estimated differences at the paper's 90th-percentile cutoff.

ComparatorExtra qualifying molecules per task from AVID
Standard PPO+2,251
REINVENT+831
Augmented Memory (AM)+906

The table addresses label cost rather than training the external baselines on the same archive. To hold offline information fixed, we compare AVID with Q-Steer-only: both use the same archive-trained guide and loss reduction. Adding the mask yields 3,475 more qualifying candidates per task on average across the 23 tasks.

Q2: Actual PPO clipping frequency. Across 112 completed diagnostic runs spanning all 23 tasks, the median run clipped 1.664% of retained tokens (run-level range: 0.817% to 6.656%). Parameters changed in all 995 logged update checks. The ratio is the current base actor's token probability divided by the stored guided behavior probability. We counted clipping only when it took effect given the sign of the advantage, not every ratio outside the allowed range.

Q3: Threshold sensitivity and paired tests. Here, H32768H_{32768} counts distinct newly scored molecules with reward at least τ\tau, the task-specific archive reward cutoff (the 90th percentile in the paper); 32,768 is the cap on generation proposals counted by the ledger, not the number of oracle calls. AVID finds more qualifying molecules than AM at the 80th, 85th, and 90th percentiles after Holm correction. At the original online budget, we average the five paired seeds within each task and run a paired sign-flip test across the 23 tasks. The p-values below are adjusted across the threshold and structural comparisons.

Archive reward cutoffExtra qualifying molecules per task: AVID minus AMHolm-adjusted p-value
80th percentile+2,4920.00360
85th percentile+2,4200.00490
90th percentile+2,2250.01470
95th percentile+1,8560.13024
99th percentile+9431.00000

At the 90th percentile, the gain also holds against PPO (adjusted p = 0.00051) and REINVENT (adjusted p = 0.04172).

Thank you for the response and additional analyses, which help address my concerns about clipping and threshold sensitivity. However, differences in access to task-specific offline reward information remain a concern in the main baseline comparisons, limiting attribution of the reported gains to the proposed method. I therefore maintain my rating of 2.

Thank you for raising the question of access to offline rewards. When PPO and Augmented Memory (AM) learned from the same task-specific reward archive as AVID, AVID still found 3,480 more qualifying molecules per task than PPO and 2,287 more than AM, on average.

Comparisons with the same archive. We evaluated all 23 tasks with five paired seeds per task. The table averages the seeds within each task and gives each task equal weight.

BaselineBaseline mean per taskAVID mean per taskAVID gain per taskTasks with a higher AVID mean
PPO trained on the same archive3,6347,114+3,48022/23
AM with replay seeded from the same archive4,8277,114+2,28718/23

PPO trained its starting policy for eight epochs on the archive, giving higher-scoring examples more weight. AM seeded its replay buffer with molecules and scores from the same archive, using its usual buffer size. AVID used that archive to train its guide.

The common budget was 2,048 offline scores per task and up to 10,000 new online scores per run, for a total cap of 12,048 including the archive. A qualifying molecule is a distinct, newly scored molecule that meets the paper's task-specific cutoff (the 90th percentile of archive rewards). We count these molecules within a separate cap of 32,768 proposals recorded by the evaluation ledger.

What identity weighting adds to guided PPO. Both conditions used the same archive-trained guide and the same active-token loss normalization. We changed only whether repeat molecules contributed directly to the policy loss. Masking repeats added 3,257 qualifying molecules per task on average, raising the five-seed mean on all 23 tasks.

In the revision, we will report both archive-informed baselines in the main results and present the fixed-guide comparison alongside them.

Reviewer_DPoUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AVID, a molecular optimization framework combining prefix-level action-value guidance (Q-Steer) with identity-aware policy learning. Q-Steer uses a frozen reward predictor to guide token generation, while identity weighting suppresses direct policy updates from repeated molecules. Experiments on 23 PMO tasks show improvements in the number of distinct high-scoring candidates, although AVID underperforms Augmented Memory on the standard PMO Top-10 AUC metric.

Critical strengths
  1. The paper addresses a meaningful problem by considering the acquisition of distinct high-quality candidates rather than focusing solely on individual molecular rewards.

  2. The experiments cover all 23 PMO tasks with five seeds and provide useful analyses of candidate quality, acquisition efficiency, and their trade-offs.

Critical weaknesses

W1. Limited practical advantage and insufficient comparisons with existing methods.

Although AVID improves the proposed distinct-candidate yield metric, it underperforms Augmented Memory on the standard PMO Top-10 AUC (15.4526 vs. 16.1875). Since molecular optimization typically prioritizes finding high-quality candidates under limited oracle budgets, the practical significance of generating more molecules above an archive-derived threshold remains insufficiently justified. Moreover, uniqueness and diversity have been extensively studied, and existing methods such as Augmented Memory already incorporate diversity filters and memory-purging mechanisms. The paper lacks comparisons with simple duplicate-filtering variants and stronger recent methods such as Saturn, making it difficult to establish meaningful improvements over existing approaches.

W2. Insufficient ablations and unclear methodological novelty.

The proposed components largely build on existing ideas in reward-guided decoding and diversity-aware learning. However, the experiments do not convincingly establish their independent contributions. Early ablations show that Q-Steer drives quality improvements, while identity weighting contributes little to early reward quality. Complete long-horizon factorial experiments cover only three tasks, and the additional controls do not adequately distinguish identity weighting from alternative loss normalization strategies. Notably, an alternative normalization variant outperforms AVID across all five Amlodipine seeds. Without stronger ablations and comparisons with simpler alternatives, the necessity and novelty of the proposed combination remain unclear.

Questions for authors
  1. Given that AVID underperforms Augmented Memory on the standard PMO Top-10 AUC, what practical advantages justify prioritizing the proposed candidate-set yield metric? How sensitive are the results to different reward thresholds?

  2. Can the authors compare AVID against stronger recent molecular optimization methods, particularly Saturn, and simple duplicate-masking or replay-filtering variants of Augmented Memory under matched budgets?

  3. Can the authors provide full 23-task long-horizon ablations to establish the independent contributions of Q-Steer and identity weighting? In particular, can they isolate identity-aware learning from the effects of loss normalization?

  4. Since Q-Steer uses additional task-specific pre-scored molecules for training, how does the performance compare when this offline supervision and the guide's computational overhead are included in the overall budget?

  5. How does the proposed identity weighting fundamentally differ from existing diversity-aware memory and duplicate-handling methods? What evidence demonstrates benefits beyond these simpler techniques?

2 comments

Thank you for your careful reading and thoughtful feedback. Below, we respond to your questions point by point and share the results of our additional analyses.

Q1: Practical value and reward cutoff. At the paper's cutoff, AVID finds 7,114 distinct qualifying molecules per task versus 4,889 for Augmented Memory (AM), a 45.5% increase. AVID has higher mean yield on 18/23 PMO tasks with five paired seeds per task and the same caps of 32,768 proposals counted by the identity ledger and 10,000 new oracle scores. The share of distinct molecules that pass the cutoff is nearly the same (82.20% for AVID vs. 81.84% for AM), so the larger pool comes from finding more distinct molecules. AM's higher Top-10 AUC sum (16.1875 vs. 15.4526 for AVID) reflects the quality of the best ten as search progresses. Qualifying yield instead counts the different molecules that clear the reward cutoff and can go forward for screening.

The gain over AM survives Holm correction at the 80th, 85th, and 90th percentiles: 2,492, 2,420, and 2,225 more qualifying molecules per task, respectively (adjusted p = 0.01470 at the paper's 90th-percentile cutoff). At the stricter 95th and 99th percentiles, the point estimates are +1,856 and +943 per task, respectively.

Q2: Saturn and Augmented Memory variants. We have submitted runs for Saturn and two Augmented Memory variants: one with direct duplicate masking and one with diversity filtering and selective memory purging. We will compare them with AVID using the same generation-attempt and oracle-call caps as in Q1. If the results are verified before the discussion closes, we will post the comparison in this thread.

Q3: Components and normalization. Across all 23 PMO tasks with five paired seeds per task, the submitted AVID configuration finds 2,104 more qualifying candidates per task on average than the active-token normalization variant, with higher yield on 14/23 tasks (Holm-adjusted p = 0.01127). Both versions use the same guide and identity mask; only the loss denominator changes. On Amlodipine, the active-token variant yields more candidates in all five seed pairs.

With identity weighting held fixed, adding Q-Steer raises mean reward by 0.0907 and Top-10 reward by 0.0813 at 512 newly scored identities, with gains on all 23 tasks. At the longer horizon, adding the identity mask to Q-Steer while keeping the guide and loss reduction fixed yields 3,475 more qualifying candidates per task on average. This comparison measures the benefit of adding the mask; the random-mask and uniform-scaling controls do not establish an additional benefit from targeting repeats specifically.

Q4: Offline scores and computation. Under a common budget of 10,000 total labels, including the offline archive, AVID finds 2,251 more qualifying candidates per task than standard PPO on the saved runs (Holm-adjusted p = 0.00609). Under the same accounting, the estimated differences are +831 versus REINVENT and +906 versus AM. We charge all 2,048 offline scores to AVID and charge each baseline for an archived score the first time it uses it. The comparison matches proposal and label caps, not wall-clock time: the frozen guide adds computation during sampling but makes no additional oracle calls.

Q5: Difference from deduplication and memory filtering. AVID acts on the direct policy loss for the current rollout: a repeated molecule keeps its cached score, but its trajectory gets zero weight in that loss. Output deduplication changes the returned list, and memory purging changes which trajectories are replayed. By changing which current trajectories contribute, the identity mask can change the direction of the direct policy gradient; a positive scalar applied to that gradient changes only its size.

AuthorsOpenReview ↗

Follow-up: Normalization Control and Baseline Results

With the frozen guide and loss normalization rule held fixed, masking repeat trajectories in PPO yielded 3,257 more qualifying molecules per task on average and increased the five-seed mean on all 23 PMO tasks. The PPO control tests what repeat masking adds to the same optimizer, while the table below compares AVID with the three requested baselines as complete methods.

The PPO control and the baseline comparisons use all 23 PMO tasks and five paired seeds per task, with caps of 32,768 proposals counted by the evaluation ledger and 10,000 new online oracle scores per run. Qualifying molecules are distinct, newly scored molecules that meet each task's 90th-percentile archive reward cutoff.

Additional baseline results. The table compares the submitted AVID configuration with the three baselines you requested (Q2) under those same caps.

MethodMean qualifying molecules per task
AVID7,114
Augmented Memory with direct duplicate masking6,130
Saturn7,155
Augmented Memory with diversity filtering and selective memory purging8,763

Normalization control. In the PPO comparison (Q3), both conditions counted all active tokens before applying weights; the only change was whether repeat trajectories received zero weight in the direct policy loss.

Reviewer_o7XBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of goal-directed molecular generation. The authors argue for an evaluation setting and methods that generate many distinct, high-quality candidates instead of focusing only on the few molecules with the highest rewards. They propose AVID, which introduces two main changes to a standard LSTM-based PPO method. First, they introduce Q-Steer, a network trained on an offline archive of scored molecules and kept frozen during RL, which predicts terminal rewards from partial SMILES strings and guides next-token sampling. Then, after a molecule is generated, they apply identity-weighted policy updates, which remove the direct policy-loss contributions of molecules whose identities have already appeared within the same run. AVID is compared against existing molecular optimization algorithms on the established PMO benchmark. The method achieves improvements on the newly introduced distinct qualifying yield metric, which counts acquired molecular identities exceeding a fixed reward threshold, while results on standard PMO reward-quality and structural-diversity metrics are mixed.

Critical strengths

The paper addresses the practically important problem of acquiring distinct, high-quality candidates in goal-directed molecular generation, rather than focusing only on the few molecules with the highest rewards. The proposed methodological contributions in AVID are intuitively well-motivated for expanding the set of high-quality candidates. The authors evaluate the method on the established PMO benchmark [1] and include ablation experiments to investigate the contributions of introduced components.

Critical weaknesses

While the problem and proposed methodological approach are practically relevant and well-motivated, the paper in its current form has several critical weaknesses concerning the soundness of the evaluation setting and the strength of the experimental evidence. The main concern is that AVID performs best on its newly proposed measure of molecular discovery, but the authors do not sufficiently justify this evaluation setting relative to existing diversity-aware benchmarks or establish the robustness of their conclusions to its design choices.

Soundness – Evaluation

The paper introduces distinct qualifying yield to evaluate the acquisition of distinct, high-quality molecular candidates. While this is a well-defined and meaningful metric for measuring distinct molecular identities, it does not account for structural similarity between the acquired molecules. Existing work proposes complementary evaluation protocols for discovering high-scoring, structurally diverse candidates [2]. The authors acknowledge related evaluation approaches and provide additional structural-diversity analyses. However, the motivation for choosing identity-based qualifying yield as the primary evaluation metric, rather than existing diversity-aware alternatives, is not sufficiently developed. In particular, the paper does not clearly explain why this metric better captures the intended scientific objective, or its relationship to existing evaluations.

Furthermore, the proposed metric depends on a fixed reward threshold defined by the 90th percentile of an initial scored molecular archive. There are two related concerns. First, the archive is a fundamental component of both the method and the evaluation, yet the source of its molecules is never stated. Given that the archive determines both the offline supervision for Q-Steer and the reward threshold used in the main evaluation, its provenance is essential for reproducibility and interpretation of the results. Second, the choice of the 90th-percentile threshold is not justified or supported by sensitivity analyses. A stronger justification of this choice is required to support the evaluation setting, either through clear arguments and citations, or a robustness analysis varying the proposed threshold.

Experimental Results

The experiments support two main positive findings: Q-Steer consistently improves early reward quality, and complete AVID substantially increases distinct qualifying yield under the proposed evaluation protocol. However, the evidence for broader improvements in high-quality, diverse molecular discovery remains limited. AVID does not outperform Augmented Memory on the aggregate conventional PMO Top-10 AUC [1,3], while its structural-diversity results are mixed, including lower normalized fingerprint diversity than REINVENT [4].

Overall, the experimental results demonstrate improvements in identity-focused molecular acquisition under a particular archive-derived reward threshold. However, the paper does not sufficiently establish how these improvements support the broader goal of discovering diverse, high-quality molecular candidates. This could be addressed in two complementary ways. First, the authors could provide a clearer motivation for the chosen evaluation setting and stronger arguments explaining why the reported results sufficiently demonstrate the intended benefits of AVID. Alternatively, they could provide additional experimental evidence, such as comparisons under established diversity-aware evaluations [2] and robustness analyses of the proposed metric.

[1] Gao, Wenhao, et al. "Sample efficiency matters: a benchmark for practical molecular optimization." Advances in neural information processing systems 35 (2022): 21342-21357.

[2] Renz, Philipp, Sohvi Luukkonen, and Günter Klambauer. "Diverse hits in de novo molecule design: Diversity-based comparison of goal-directed generators." Journal of Chemical Information and Modeling 64.15 (2024): 5756-5761.

[3] Guo, Jeff, and Philippe Schwaller. "Augmented memory: Sample-efficient generative molecular design with reinforcement learning." Jacs Au 4.6 (2024): 2160.

[4] Blaschke, Thomas, et al. "REINVENT 2.0: an AI tool for de novo drug design." Journal of chemical information and modeling 60.12 (2020): 5918-5922.

Questions for authors
  1. Could you provide full details on the construction of the archive dataset, including the source of the molecules?

  2. Could you provide the motivation behind the chosen threshold for the distinct qualifying yield metric? Furthermore, could you provide robustness results when this threshold is varied?

  3. Could you provide further arguments for positioning your evaluation setup against existing diversity-aware evaluation settings or add further experimental results showing the strong benefits of AVID over the external baselines beyond the distinct qualifying yield metric?

5 comments

Thank you for your careful reading and thoughtful feedback. Below, we respond to your questions point by point and share the results of our additional analyses.

Q1: Where the archive comes from. Each of the 23 task archives contains 2,048 scored SMILES rows. We sampled the SMILES from a frozen ACEGEN LSTM prior and evaluated them with the corresponding PMO task's scoring function before online optimization. The 2,048 counts scored rows, not unique molecules. The guide learns from the resulting task scores.

Q2: Choice of cutoff. AVID's gain over Augmented Memory (AM) holds at the 80th, 85th, and 90th percentiles after Holm correction. An archive percentile sets a high-reward reference for each task without imposing one numerical score across different tasks. All methods on a task use the same cutoff, and it is used only to count qualifying outputs, not to train the guide or decide which trajectories to mask.

All 23 tasks and five paired seeds are included, with the original caps of 32,768 proposals counted by the identity ledger and 10,000 new scores:

Archive reward cutoffExtra qualifying molecules per task: AVID minus AM
80th percentile+2,492
85th percentile+2,420
90th percentile+2,225
95th percentile+1,856
99th percentile+943

We average the paired seeds within each task before testing. The adjusted p-values at the 80th, 85th, and 90th percentiles are 0.00360, 0.00490, and 0.01470, respectively, with Holm correction across the threshold and structural comparisons. The table also shows the point estimates at the two stricter cutoffs.

Q3: Candidate yield and structural coverage. At the paper's cutoff, AVID covers 2,004 more qualifying Bemis-Murcko scaffolds than PPO (Holm-adjusted p = 0.00051). To test structural separation, we use fixed MaxMin selection on radius-2 Morgan fingerprints, which addresses the question raised by Diverse Hits. AVID retains 1,424 more qualifying molecules at Tanimoto distance 0.5 and 802 more at distance 0.6 than PPO (both adjusted p < 0.001). At distance 0.7, REINVENT retains 962 more than AVID; the AVID differences versus PPO (+283) and AM (-144) do not pass correction at that distance.

Distinct qualifying yield measures how many different molecules are available for further screening; close analogues can count separately. The scaffold and fingerprint readouts tell us how many frameworks are represented and how many candidates remain when we require a minimum distance between them. We report both because a larger screening pool and a more structurally separated set answer different needs.

Thank you for your responses and additional experiments.

  • Answer to Q1: Thank you for clarifying this point.

  • Answer to Q2: Thank you for the additional results. To my understanding, they show that AVID's gains over Augmented Memory remain significant at the paper's 90th-percentile archive reward cutoff and below, while raising the cutoff to the 95th or 99th percentile does not maintain significant improvements.

  • Answer to Q3: Thank you for the additional results and discussion. I understand that a larger screening pool and a more structurally separated set answer different needs. My question, however, was about the motivation and positioning of distinct qualifying yield relative to existing diversity-aware evaluations. The response explains that the two differ and points to further screening as the setting where a larger pool matters. That addresses what the metric measures, but not why it should be the primary measure of success relative to existing diversity-aware evaluations. I believe that citing work with specific applications where larger pool of distinct identities is a key quantity would strengthen this motivation.

The clarification in Q1 made me understand that the paper in its current form does not provide the comparison under a common label budget. That is, the budget that counts the offline archive scores against the total label budget. I have noticed that a similar concern has already been raised by other reviewers and that you have provided some additional results, so I will not repeat the same question. However, the common budget comparison and the results varying the cutoff from Q2 raise the questions below.

The paper's empirical gains lead with the 45.5% improvement over the Augmented Memory baseline as one of the key results. However, the additional results show that: (1) the improvements over Augmented Memory do not remain significant at the higher 95th and 99th percentile cutoffs, and (2) under a common label budget the reported gain over Augmented Memory falls from +2,225 to +906 qualifying candidates per task. This leads to two questions:

  1. Could you please combine the two experiments to assess the robustness of the results under different archive reward cutoffs for the distinct qualifying yield metric, while giving all approaches a common label budget?

  2. Could you please indicate how these new results will be incorporated into the paper? I understand that the discussion period is short, and it is acceptable not to fully revise the paper. However, to my knowledge, the current version of the paper is unchanged from the submission, while the additional results concern some of the main empirical results in the paper, mentioned in the abstract. Could you please explain how the claims and contributions in the paper will be revised, given the new results? Could you provide a non-exhaustive list of changes that will be incorporated into the paper? I find it difficult to assess the paper without seeing how the revised version would incorporate some of the key results, which potentially require revising some of the main points.

I keep my score and I would consider raising after understanding how the revised manuscript would incorporate the new results and refine the main points.

AuthorsOpenReview ↗

Follow-up (1/2): Results and rationale

Thank you for your careful reading and thoughtful follow-up. We have completed the combined cutoff-and-total-budget comparison you requested.

1. Combined cutoff sweep under a common total-score budget. The comparison now includes all five cutoffs, all three baselines, all 23 PMO tasks, and five paired seeds, with the offline archive charged within the 10,000-score budget.

We recomputed the readouts from the saved trajectories under caps of 10,000 total scores and 32,768 generation proposals. AVID is charged all 2,048 archive scores upfront, leaving at most 7,952 new online scores. Each baseline is charged for every new score and for an archived score on first use. We count distinct, newly scored molecular identities meeting the same task-specific archive cutoff. The proposal cap counts generation attempts, not scoring calls.

Archive reward cutoffAVID minus PPOAVID minus REINVENTAVID minus AM
80th percentile+2,457+163+1,107
85th percentile+2,381+462+1,062
90th percentile+2,251+831+906
95th percentile+1,956+1,201+613
99th percentile+1,272+1,423−15

Entries give the mean extra qualifying molecules per task. We first average the five paired seeds within each task, then compare the resulting task means.

At the paper's 90th-percentile cutoff, the task means are 5,795 for AVID, 3,544 for PPO, 4,964 for REINVENT, and 4,889 for AM. Against AM, AVID acquires 906 more qualifying molecules per task, an increase of approximately 18.5%, with the archive included in the total budget. The 45.5% result uses the separate reusable-archive setting with matched online budgets. Paired task-level tests, corrected across all 15 comparisons, support the gains over PPO at the 80th–95th percentiles and over REINVENT at the 99th percentile.

2. Identity weighting under a fixed PPO configuration. Adding identity weighting gives +3,257 qualifying molecules per task on average, with higher task means on all 23 tasks. Both conditions use the same frozen guide and the same active-token loss-normalization rule; only identity selection changes. We average the same five paired seeds within each task and use the paper's 90th-percentile cutoff. This is a within-PPO comparison at 10,000 new online scores, distinct from the 10,000-total-score comparison above.

3. Why qualifying yield is primary for candidate acquisition. A screening reward is an initial filter. Each distinct compound that clears it adds an option for follow-up tests; a repeat of the same compound does not. For example, Lyu et al. [1] synthesized and tested 549 compounds selected from a docking screen against the D4 receptor.

At this stage, even close analogues can add useful options. Leach et al. [2] used matched molecular pairs to show how small structural changes can affect aqueous solubility, plasma protein binding, and oral exposure. A structural-distance requirement answers a different question: how many qualifying compounds can we select for a structurally separated shortlist? We count distinct qualifying molecules to measure the pool available for follow-up, and report scaffold coverage and the fingerprint-separation analysis motivated by Diverse Hits [3] to measure structural coverage. Top-10 AUC separately measures how well the search improves its best scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.