acceptodds
Under review as a conference paper at ICLR 2027

Input-Output Sampling for Efficient Multi-Objective Molecular Optimization with LLMs

Abstract

Multi-objective molecular optimization with LLMs is bottlenecked by limited oracle budgets. Existing methods often struggle to maintain stable trade-offs among competing properties, and to select samples that provide informative learning signals, causing expensive evaluations to be spent on redundant molecules. To this end, we propose MolDAS, a multi-objective reinforcement learning framework with Dual-sided Adaptive Sampling that selects original molecules at the input side and optimized candidates at the output side. At the input level, MolDAS balances structural diversity with proximity to the policy's current capability boundary, selecting molecules that are underexplored but still learnable for oracle evaluation. At the output level, it sends the lowest-probability answer segments in each rollout group to the oracle. Moreover, it identifies high-value completions within rollout groups via reward dispersion and applies an auxiliary supervised fine-tuning loss for direct imitation. To further stabilize training, a lightweight dynamic weighting scheme calibrates property improvement, structural similarity, and reasoning quality as optimization progresses. MolDAS is trained in a two-stage simulator-to-oracle paradigm, first with an inexpensive simulator and then adapted under a capped budget of high-fidelity oracle calls that mirrors real experimental constraints. Experiments show that MolDAS raises the learning ceiling under abundant low-fidelity feedback and learns faster under limited high-fidelity feedback, achieving a 3.7% relative improvement over vanilla GRPO in Stage 1 and over 16.1% in Stage 2 on the optimization score. Sampling audits show reduced structural redundancy and higher molecular diversity, and lower performance fluctuation than fixed-weight training. These results suggest that our method offers a practical route to efficient and stable multi-objective molecular optimization with LLMs.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
multi-objective reinforcement learning, molecular optimization, large language models, active sampling, oracle-efficient optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pDDoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MolDAS, an LLM-based framework for multi-objective molecular optimization under limited oracle budgets. The main idea is dual-sided adaptive sampling: selecting informative input molecules based on diversity and reward dispersion, and selecting low-probability generated candidates for expensive oracle evaluation. The framework further combines high-value replay with auxiliary SFT, adaptive reward weighting, and a simulator-to-oracle two-stage training procedure. Experiments based on a DrugR-initialized LLaMA-3 8B model show improvements over vanilla GRPO, especially in the oracle stage.

Critical strengths
  1. The paper addresses an important practical bottleneck in LLM-based molecular optimization: how to allocate expensive oracle feedback. Treating both input selection and output evaluation as part of the feedback-allocation problem is a sensible and useful formulation.

  2. The method is relatively comprehensive. In particular, the combination of adaptive sampling, replay, dynamic objective weighting, strict canonical-molecule budget accounting, and simulator-to-oracle adaptation forms a coherent training pipeline. The improvement over vanilla GRPO in the oracle stage is also substantial.

Critical weaknesses
  1. The central claim is oracle-efficient optimization, but the experimental evidence does not directly characterize efficiency as a function of oracle calls. The paper does not clearly report the actual Stage-2 oracle-call cap or the number of distinct oracle queries used by each method, and the main curves use RL steps rather than oracle calls on the x-axis. Stage 2 also only compares MolDAS against vanilla GRPO. As a result, it is difficult to determine whether the improvement comes specifically from better oracle allocation rather than the additional training components.
  2. The main experiments appear to be based on single runs. No standard deviations, confidence intervals, or multi-seed results are reported. This is particularly important because GRPO training and the proposed adaptive sampling procedure are both stochastic, while the Stage-1 improvement over GRPO is only about 3.7%. Without repeated runs, it is difficult to assess the robustness of this gain. A secondary concern is that DFRL-Mol and Genetic GFN are adapted into the authors’ common LLM framework rather than evaluated in their original formulations. This makes the comparison useful as a controlled ablation of mechanisms, but somewhat weaker as a direct comparison against prior methods.
Questions for authors
  1. What is the exact high-fidelity oracle budget in Stage 2? Could the authors report optimization score versus cumulative distinct oracle calls, together with the total number of oracle queries used by MolDAS and each baseline? This would directly validate the main oracle-efficiency claim.

  2. Could the authors provide multi-seed results for the main MolDAS versus GRPO comparison, including mean and standard deviation? In particular, is the 3.7% Stage-1 improvement consistent across seeds?

  3. Can the authors clarify the Stage-1/Stage-2 evaluation of DFRL-Mol? The discussion appears to refer to its ADMETLab-oracle performance, while the main table only lists DFRL-Mol under Stage 1. A complete budget-matched Stage-2 comparison would strengthen the paper considerably.

5 comments
AuthorsOpenReview ↗

Response 1

Summary. We thank the reviewer for the constructive questions. We report actual oracle consumption, performance against cumulative calls, and means and standard deviations across three training seeds. We also provide Stage 2 results for both additional baselines and clarify their adaptations.

1. Oracle consumption and performance versus calls

We thank the reviewer for this request, which targets the central efficiency claim directly. All Stage 2 experiments run under a common training cap of 2,8802,880 distinct canonical structures per run. Calls are counted over distinct canonical structures across training steps; cache hits cost nothing, and final test evaluation queries are counted separately. The table below reports the total distinct oracle calls for MolDAS and each baseline:

MethodDistinct oracle callsReduction by MolDAS
MolDAS1,4671,467---
Vanilla GRPO2,2912,29136.036.0\\%
DFRL Mol, adapted2,0392,03928.128.1\\%
Genetic GFN, adapted2,1532,15331.931.9\\%

MolDAS uses 36.036.0\\% fewer calls than vanilla GRPO, 28.128.1\\% fewer than the adapted DFRL Mol, and 31.931.9\\% fewer than the adapted Genetic GFN. In our runs, oracle calls scale linearly with training steps. The revision will therefore plot optimization score against cumulative distinct oracle calls rather than RL steps.

2. Results across three training seeds

We appreciate the reviewer's suggestion to report results across multiple seeds. We repeated MolDAS and vanilla GRPO with three training seeds per method in each stage. The table below reports mean ± standard deviation.

StageMethodOpt. ScoreF1
Stage 1MolDAS0.2727±0.01120.2727 ± 0.01120.4405±0.14930.4405 ± 0.1493
Stage 1Vanilla GRPO0.2648±0.01130.2648 ± 0.01130.4050±0.16780.4050 ± 0.1678
Stage 2MolDAS0.3268±0.01680.3268 ± 0.01680.6011±0.11610.6011 ± 0.1161
Stage 2Vanilla GRPO0.2896±0.01830.2896 ± 0.01830.5033±0.11750.5033 ± 0.1175

In Stage 1, mean Opt. Score improves by 0.00780.0078, and mean F1 improves by 0.03550.0355. The positive average difference therefore persists across repeated training, with a magnitude that differs from the original 3.73.7\\% comparison. In Stage 2, mean Opt. Score improves by 12.812.8\\% and mean F1 by 19.419.4\\%. The means are higher in both stages, and the standard deviations quantify the variability across training seeds.

3. Stage 2 baseline coverage and adaptation

We thank the reviewer for pointing out this problem. The original DFRL Mol entry was listed under Stage 1. We now provide explicit Stage 2 results for both additional baselines. The table below reports mean ± standard deviation together with the corresponding oracle consumption:

MethodOpt. ScoreF1Distinct oracle calls
MolDAS0.3268±0.01680.3268 ± 0.01680.6011±0.11610.6011 ± 0.11611,4671,467
DFRL Mol, adapted0.2974±0.01370.2974 ± 0.01370.3877±0.11350.3877 ± 0.11352,0392,039
Genetic GFN, adapted0.2994±0.01150.2994 ± 0.01150.2661±0.15930.2661 ± 0.15932,1532,153

These results extend the oracle-stage evaluation beyond vanilla GRPO. Explicit stage labels distinguish them from the original Stage 1 entry, and the reported consumption makes the oracle cost visible alongside performance. Across both metrics, MolDAS reaches a higher Opt. Score and F1 than either adapted baseline while using fewer oracle calls (1,4671,467 against 2,0392,039 and 2,1532,153).

AuthorsOpenReview ↗

Additional Response to Q1.

In our earlier response (Section 1, "Oracle consumption and performance versus calls"), we noted that oracle calls scale linearly with training steps and that the revision would plot optimization score against cumulative distinct oracle calls rather than RL steps. We have now completed these curves, and the figure will be added to the revision as Figure 4 (Appendix A.6, "Checkpoint performance versus oracle consumption").

The figure plots the Stage-2 Opt. Score of MolDAS and vanilla GRPO against cumulative distinct oracle calls. Because cumulative calls grow linearly with RL steps, the horizontal axis measures training progress by actual query overhead.

MolDAS reaches a higher Opt. Score than vanilla GRPO at fewer cumulative calls, peaking at 0.33539 before 1,000 cumulative calls, while vanilla GRPO reaches its peak of 0.29337 only after about 1,700. At matched cumulative consumption MolDAS is already ahead, so the advantage is not an artifact of early stopping. MolDAS also terminates at 1,467 calls against 2,291 for vanilla GRPO, a 36.0% reduction, with calls counted over distinct canonical structures as before.

AuthorsOpenReview ↗

Gentle reminder regarding our responses

This is a gentle reminder that we have uploaded the revised manuscript and our detailed responses.

Given that the discussion phase will close soon, could you please let us know if our revisions have alleviated your concerns? We remain available to clarify any remaining points before the deadline.

Best regards, The Authors

Reviewer_hGfrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Multi-objective molecular optimization requires LLMs to optimize molecules under limited budgets. This paper proposes MolDAS, a reinforcement learning framework with dual-sided adaptive sampling mechanisms to balance optimization efficiency and quality. By exploring the input side, selecting queries for valuable output, using adaptive reward weights, and employing two-stage training, the paper improves LLMs' performance in molecular optimization. The problem is practical, and the complete system outperforms the GRPO baseline in common frameworks on reported property-optimization scores and explanation F1.

Critical strengths
  • The paper focuses on a particle problem and proposes a well-motivated method.
  • The paper provides component ablation within a common language model and reward framework and discloses the baseline adaptation method.
  • The paper also provides details on the experimental setup and discusses the results.
Critical weaknesses
  • The second stage starts from each method's own optimal Stage 1 checkpoint without repeating key component ablation. It is hard to distinguish between initialization gains and query strategy gains.
  • The main benefits of the second stage have also not yet been established through component comparison with the same starting point.
  • There is no direct evidence of query efficiency. The budget figures and actual consumption are not fully reported.
Questions for authors
  1. Can the author provide a discussion or comparison of the evidence on efficiency?
  2. Can the author provide a discussion of the effectiveness of the two-stage training strategy?
5 comments
AuthorsOpenReview ↗

Response 1

Summary. We thank the reviewer for the constructive questions. We provide oracle consumption and checkpoint curves, control Stage 1 initialization, and compare the full training schedule with direct oracle adaptation. These experiments examine efficiency, component contributions, and the value of simulator pretraining.

Q1: Evidence for oracle efficiency.

R1: We agree that the efficiency claim requires direct oracle-call evidence, and the revision adds it. All Stage-2 arms use the same hard cap and evaluation protocol, and calls are counted over distinct canonical structures across steps; cache hits cost nothing. The table below reports the total distinct oracle calls for both methods:

MethodDistinct oracle callsReduction by MolDAS
MolDAS1,4671,467---
Vanilla GRPO2,2912,29136.036.0\\%
DFRL Mol2,0392,03928.128.1\\%
Genetic GFN2,1532,15331.931.9\\%

MolDAS uses 36.036.0\\% fewer calls than vanilla GRPO, 28.128.1\\% fewer than DFRL Mol, and 31.931.9\\% fewer than Genetic GFN.

In our runs, oracle calls scale linearly with training steps. The revision will therefore plot the main curves against cumulative oracle calls rather than RL steps.

Q2. Effectiveness of the two-stage training strategy.

R2: We answer this question from two aspects.

First, we separate initialization gains from query-strategy gains with a same-start control. All Stage-2 arms now start from one common Stage-1 checkpoint and run under the same oracle budget and evaluation protocol. The table below reports the resulting comparison and the component ablations at that starting point:

Stage-2 arm (common Stage-1 checkpoint, same oracle cap)Opt. ScoreF1
MolDAS0.32680.32680.60110.6011
Vanilla GRPO0.28960.28960.50330.5033

Because the starting point and the budget are fixed, the differences measure the Stage-2 training components rather than the inherited checkpoint, and the ablation rows identify which components carry the gain. MolDAS improves Opt. Score by 8.18.1\\% and F1 by 5.25.2\\% with initialization fixed. Removing either tested component lowers both metrics. The overall comparison evaluates the Stage 2 components jointly, while the removal experiments examine their individual contributions.

Second, we test the two-stage strategy with an ablation that fixes the Stage-2 method and budget. The table below reports the comparison:

Opt. ScoreF1
DrugR/SFT → Stage 10.27270.27270.44050.4405
DrugR/SFT → Stage 20.29710.29710.27410.2741
DrugR/SFT → Stage 1 → Stage 20.32680.32680.60110.6011

The full schedule reaches the highest Opt. Score and F1 among the three rows. Removing either stage lowers both metrics, so the comparison supports the effectiveness of our two-stage training.

Thank you for the additional experiments and discussion. The comparison of initializations addresses parts of my concerns.

However, the table shown only includes the complete method and GRPO, making it hard to determine the contributions of each component. We may have more discussion on this part. There is no need to conduct additional experiments.

Additionally, if possible, could you provide specific graphs showing how query overhead and performance vary with cumulative oracle calls?

AuthorsOpenReview ↗

Response 2

Summary. Thank you for the follow-up comment. We report the completed Stage-2 component ablations and analyze the contribution of each component, and we provide the curves of Stage-2 performance against cumulative oracle calls. Both will be included in the revision.

Q1: Contributions of each component.

R1: We have completed the Stage-2 component ablation experiments. Each ablation arm is the full MolDAS with exactly one component disabled, trained with the same two-stage procedure under the same oracle cap, and in Stage 2 it continues from its own validation-selected Stage-1 checkpoint. The table below reports the results, which will be added to Table 1 in the revision, as mean ± standard deviation over training seeds:

Stage-2 armOpt. ScoreF₁
MolDAS (full)0.3268±0.01680.6011±0.1161
w/o active sampling0.2914±0.01370.4840±0.1280
w/o output-side selection0.2975±0.01160.4778±0.1264
w/o adaptive weighting0.2963±0.01120.3055±0.0163
w/o high-value pool0.3008±0.01410.4894±0.1313

Every removal lowers both metrics, so all four components contribute. Their shares differ by metric, and the differences match what each component does.

Active sampling carries the largest share of the optimization score. Removing it lowers the score from 0.3268 to 0.2914, the largest single-component drop on this metric. The sampler steers each batch toward inputs whose within-group reward dispersion is wide, that is, inputs the policy can succeed on but not yet reliably. Without it, batches underrepresent this learnable frontier.

Adaptive weighting carries the largest share of F₁. Removing it lowers F₁ from 0.6011 to 0.3055, the largest single-component drop on this metric, and this row has the smallest seed variance among the four. The controller up-weights channels that lag and down-weights channels that improve, keeping the composite reward focused on the objective that currently limits progress.

Output-side selection and the high-value pool each contribute on both axes. Removing output selection gives 0.2975 and 0.4778. The selector sends only the lowest-probability distinct canonical structures of each group to the oracle, so the budget concentrates where the policy is least certain. Removing the pool gives 0.3008 and 0.4894. Replay of trusted inputs, supervised by the auxiliary SFT loss, converts feedback already paid for into further training signal.

In summary, the four contributions are complementary rather than redundant. The sampler places the oracle budget, which shows up in the optimization score, and the controller sets the reward trade-off, which shows up in the target-property F₁.

Q2: Performance and query overhead versus cumulative oracle calls.

R2: We have completed plotting the curves, and the figure will be added to Appendix A.6 ("Checkpoint performance versus oracle consumption") of the revision as Figure 4.

The figure plots the Stage-2 Opt. Score of MolDAS and vanilla GRPO against cumulative distinct oracle calls. We find that cumulative oracle calls grow linearly with RL steps, so the horizontal axis is training progress measured by actual query overhead.

MolDAS reaches a higher Opt. Score than vanilla GRPO at fewer cumulative calls, peaking at 0.33539 before 1,000 cumulative calls, while vanilla GRPO reaches its peak of 0.29337 only after about 1,700. The advantage is not an artifact of early stopping, since at matched cumulative consumption MolDAS is already ahead. MolDAS also terminates at 1,467 calls against 2,291 for vanilla GRPO, a 36.0% reduction, under the accounting already reported (distinct canonical structures, cache hits free, common cap of 2,880).

Reviewer_2zJoOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies multi-objective molecular optimization with LLMs for input output sampling. The authors define the problem as RL fine-tuning is limited by a small budget of high-fidelity oracle calls. The authors propose MOLDAS, a GRPO-based method that selects training inputs using within-group reward dispersion and structural diversity, sends only the lowest-probability rollouts of each group to the oracle, replays reliable inputs with an auxiliary SFT loss, and adapts reward weight to each objective training trend. On the input side, a dual-level sampler combines a diversity-ordered global pass over the training pool with local neighbourhood sampling around parent molecules whose rollout groups show high reward dispersion and improvement headroom. This reuses the within-group dispersion that GRPO already computes. On the output side, only the lowest-probability answer segments in each rollout group are sent to the oracle, with de-duplication by canonical structure. A high-value replay pool reuses inputs with reliable feedback and adds an auxiliary teacher-forced SFT loss on their reference answers. A rule-adaptive controller moves each reward weight against the recent trend of its channel, and a bounded penalty discourages explicit rule violations such as invalid, copied, or out-of-band outputs.

Critical strengths
  1. Feedback allocation as part of the RL problem. Treating the choice of which inputs and rollouts receive oracle feedback as part of the RL problem is a reasonable and practically relevant perspective for budget-limited molecular optimization.

  2. No extra cost for the input-side signal. The sampler reuses the within-group reward dispersion that GRPO already computes, so the signal guiding input selection adds no computational cost.

Critical weaknesses

1.Insufficient differentiation from DrugR. MolDAS starts from the DrugR-initialized checkpoint and keeps six-channel composite reward and validity rules. DrugR’s self-balanced multi-granular RL already includes Pareto-based reweighting, group-level scaling between the reasoning and SMILES groups. The rule-adaptive controller differs mainly in using the trend between windows instead of the shortfall from a target. The underlying principle of giving bounded extra weight to lagging channels is the same. However, the paper presents DrugR only as a reward interface and does not discuss this mechanism.

2.Unanalyzed interactions between components. The uncertainty-gated neighbourhood sampling moves toward inputs on which the policy is uncertain. In contrast, the high-value input pool assigns its largest weight, to the median main score and therefore favors inputs the policy already handles well. The rule-adaptive controller and the sampler are also coupled: changing the reward weights changes the within-group reward dispersion, which in turn determines the next input selection. The paper provides no analysis of these interactions.

3.Missing hyperparameters for an efficiency claim. Although the central claim concerns oracle efficiency, the paper does not report the total oracle budget, the value of q, or the size of the fixed test set.

4.Weak experimental comparison. Vanilla GRPO is the only baseline in Stage 2. Baselines and ablations are confined to Stage 1. The only justification given is that MOLDAS and vanilla GRPO keep the same ordering in both stages, which does not show that component contributions transfer.

5.Insufficient polish. In Section 5.1, there is a sentence that is somehow incomplete. The paper needs to be more polished with more effort.

Questions for authors

See the weakness section.

8 comments
AuthorsOpenReview ↗

Response 1

Summary. We thank the reviewer for the detailed comments. We clarify the contribution relative to DrugR, report additional Stage 2 baselines and ablations, and specify the oracle budget. We also explain component coupling and will add controlled interaction experiments.

Q1. Insufficient differentiation from DrugR.

R1: We agree with the reviewer's characterization and will correct the manuscript. DrugR's self-balanced multi-granular RL already balances the reasoning and SMILES groups through Pareto-based reweighting and group-level scaling. Our controller builds on this principle. The difference we claim is the update signal: it uses the trend of each channel between consecutive windows, computed from the change in window means, rather than the shortfall from a target, so it requires neither a per-channel target or reference scale nor a Pareto front. We do not claim the balancing principle as new, and we will describe DrugR's mechanism and credit it explicitly in the Related Work and Method sections.

The distinction is at the level of the problem. DrugR addresses how to balance multiple objectives under a given feedback source. Our setting adds the constraint the paper is built around: the high-fidelity oracle that scores the optimization channel is the bottleneck, and its budget is charged per distinct canonical structure, so the same molecule is never charged twice and a cache hit costs nothing. DrugR specifies the reward but does not address how to allocate a limited number of oracle calls across inputs and rollouts, so budget allocation remains open. MolDAS addresses this allocation, and the contributions we claim lie there:

  • Input side: original molecules enter the batch by diversity ordering of the global pool together with an uncertainty gate around the policy's current capacity boundary, using the within-group reward dispersion that GRPO already computes at no extra cost.
  • Output side: within each group, the q rollouts whose answer segments have the lowest mean log-probability are sent to the oracle under the call budget, and repeated canonical molecules are merged before the call.
  • Reuse: an online replay pool returns inputs whose earlier feedback was reliable and supplies a teacher-forced signal for the reference reasoning and answer, so earlier feedback is reused without new calls.
  • Two-stage protocol: an inexpensive simulator shapes the policy first, and high-fidelity oracle calls are reserved for adaptation under a hard cap.

The controller is a supporting component that keeps the multi-objective reward stable while allocation changes the training distribution. In the revision we will describe DrugR's balancing mechanism and credit it, state the trend-versus-target difference and its rationale, add a comparison table, and restate the contributions with the novel claims placed on budgeted allocation rather than on the reward controller.

Q2. Unanalyzed interactions between components.

R2: We agree that the paper does not analyze these interactions, and the revision will add the analysis and the experiments.

Oracle calls are the bottleneck, so the design question is where the next calls give the most return. Within-group reward dispersion, which GRPO already computes, marks the inputs where the policy can succeed but not reliably. The uncertainty-gated sampler sends the batch toward those inputs, so calls follow the region the policy is still learning. The high-value pool keeps a fixed share of the batch, p=0.30p = 0.30, on inputs whose feedback is reliable and whose groups still carry headroom, so the spent budget keeps producing a well-conditioned signal and is not re-spent on inputs whose feedback is too noisy or too flat to support a reliable advantage.

The two preferences point in opposite directions, and the design bounds the conflict. Pool admission requires a trusted group, and the pool score includes headroom, validity, and structural diversity alongside the median main score, which keeps inputs the policy already handles well from dominating. Replayed inputs are regenerated on-policy, and calls from new structures are counted in the reported totals.

The revision will add experiments that test these interactions directly. One crosses the gated sampler with replay, and another crosses input sampling with adaptive weighting, which also covers the controller-sampler coupling. Initialization, the oracle cap, and other settings stay fixed across arms. We will report Opt. Score, F1, actual oracle consumption, and interaction effects rather than single-factor comparisons.

AuthorsOpenReview ↗

Response 2

Q3. Missing hyperparameters for an efficiency claim.

R3: All Stage 2 experiments use q=2q = 2 distinct canonical structures per input group and a common training cap of 2,8802,880 structures per run. The fixed test set contains 168168 molecules. Calls are counted over distinct canonical structures across training steps. Cache hits incur no new charge, and final test evaluation queries are counted separately. The table below reports the hyperparameters together with the measured oracle consumption:

QuantityMolDASVanilla GRPO
Distinct oracle calls consumed1,4671,4672,2912,291
Relative consumption64.064.0\\%100100\\%
Extra calls relative to MolDAS---+824+824 (+56.2+56.2\\%)
Output-selection qq22---
Fixed test-set size168168168168

MolDAS consumes 1,4671,467 calls against 2,2912,291 for vanilla GRPO, which is 824824 fewer calls, or a 36.036.0\\% reduction. In the same-start Stage-2 comparison MolDAS also reaches a higher Opt. Score and F1 (0.32680.3268 and 0.60110.6011 versus 0.28960.2896 and 0.50330.5033), so the lower consumption is not obtained by stopping early at a weaker policy. In the revision, we will state the cap value in Section 5.1 and report per-method totals in Table 1.

Q4. Weak experimental comparison.

R4: We agree that a consistent ranking across stages is not evidence of transfer, and we no longer rely on it. We instead add two Stage-2 comparisons.

First, we include two additional baselines in Stage 2: DFRL Mol and Genetic GFN. They are transferred into the shared LLM framework together with their defining mechanisms. The table below reports their performance:

Stage-2 baseline (adapted, shared LLM framework)Opt. ScoreF1
DFRL Mol0.2974±0.01370.2974 ± 0.01370.3877±0.11350.3877 ± 0.1135
Genetic GFN0.2994±0.01150.2994 ± 0.01150.2661±0.15930.2661 ± 0.1593

Under the same evaluation protocol, the DFRL Mol reaches 0.29740.2974 on Opt. Score and 0.38770.3877 on F1, and the adapted Genetic GFN reaches 0.2994±0.01150.2994 ± 0.0115 and 0.2661±0.15930.2661 ± 0.1593. MolDAS consistently outperforms both adapted baselines on both metrics.

Second, we also run component ablations in Stage 2 to examine the role of each component, using one common Stage 1 checkpoint under the same oracle cap. The table below reports the results.

Stage-2 arm (common Stage-1 checkpoint, same oracle cap)Opt. ScoreF1
MolDAS0.32680.32680.60110.6011
Vanilla GRPO0.28960.28960.50330.5033

With initialization held fixed, MolDAS improves Opt. Score by 8.18.1\\% and F1 by 5.25.2\\% over vanilla GRPO. Removing adaptive weighting or output selection lowers both metrics, so the contribution of each component is observed directly under Stage 2 rather than inferred from Stage 1. In the revision, we report the results in Section 5.

Q5. Insufficient polish.

R5: Thank you for identifying this issue. We apologize for the incomplete sentence in Section 5.1 and the resulting lack of clarity. We have corrected it as follows:

Stage 2 continues each arm from its own validation-selected Stage 1 checkpoint under the same oracle budget. Original molecules and bioactivity records come from ChEMBL. We split these data 90/10 into 5,076 training and 565 validation inputs, and reserve a held-out test set of 168 inputs for final evaluation. Checkpoints are selected exclusively on the validation set by locating the peak simulator-stage score and stepping back a fixed number of preceding checkpoints, so the checkpoint submitted to the oracle is determined before any test evaluation. We report the overall optimization score and the target-property score.

We will also carefully edit the manuscript for grammatical completeness, consistent terminology, and consistent notation. We will improve the organization of the method description, making the sequence and dependencies among input selection, candidate generation, output selection, oracle evaluation, replay, and reward-weight updates explicit.

AuthorsOpenReview ↗

Additional Response to Q2

In Q2, the review raises two interactions: the uncertainty gate together with the high-value input pool, and the controller together with the sampler. To answer this question, we have added Stage-2 component ablations and we will analyze the two interactions in more detail. The table below shows our new component ablations in Stage-2, reported as mean ± standard deviation over training seeds.

Stage-2 armOpt. ScoreF1
MolDAS (full method)0.3268±0.01680.6011±0.1161
w/o active sampling0.2914±0.01370.4840±0.1280
w/o high-value pool0.3008±0.01410.4894±0.1313
w/o output-side selection0.2975±0.01160.4778±0.1264
w/o adaptive weighting0.2963±0.01120.3055±0.0163

1. Uncertainty gate and high-value input pool

We agree that the two preferences appear to point in opposite directions, and we now analyze their interaction explicitly. Mechanically, the gate buys new capability at the moving frontier, while the pool converts feedback already paid for into a retainable training signal. The two do not conflict and are in fact complementary:

The gate alone cannot keep the update stable, and it is the pool that makes the frontier targets the gate selects learnable. The gate sends the batch toward inputs whose within-group reward dispersion is wide, and these are inputs the policy can succeed on but not yet reliably. Such inputs mark the current capability frontier. Training on this frontier alone, however, concentrates the update on high-variance groups whose advantages are noisy. The pool closes exactly this gap. An input enters the pool only after a trusted group has shown its feedback to be reliable, so the pool supplies low-variance, consolidated data. The auxiliary su-pervised fine-tuning loss then extracts a dense token-level signal from these data and stabilizes the update.

The pool alone fails in the opposite way. Without the gate, input sampling no longer tracks the moving capability boundary, and selection reverts to diversity-ordered sampling over the full pool. A batch then underrepresents the inputs that are learnable but not yet stable. Replay cannot fill this gap, because by construction the pool holds inputs the policy already handles.

Our revision adds Stage-2 component ablations to Table 1, and the ablation results agree with this account. Removing active sampling lowers the Stage-2 optimization score from 0.3268 to 0.2914, and removing the high-value pool lowers it to 0.3008. Stage 1 shows the same direction for sampling, where removal lowers the score from 0.2589 to 0.2508. Each therefore covers the region the other leaves thin, and removing either costs score.

In short, the gate and the pool are not two competing strategies. The gate advances the policy to inputs it can soon solve, and the pool retains what the policy has learned through reliable feedback and teacher-forced SFT supervision.

2. Controller and sampler

The controller and the sampler are indeed coupled, and the two always target the currently weakest objective. The controller adjusts the weights according to the recent trend of each channel, weighting channels that lag and down-weighting channels that improve, so that the composite reward stays focused on the channel that currently limits progress. These weights determine the within-group reward dispersion, and the sampler reads the same dispersion, moving the batch within trusted groups toward inputs with wide dispersion, that is, inputs the policy can succeed on but not yet reliably. In this way, "which objective to push" and "which inputs to learn from" fall on the same bottleneck, and learning concentrates where the information is greatest at that moment.

The Stage-2 ablations agree with this account. The two mechanisms are complementary, and removing either lowers both metrics. The full method reaches an optimization score of 0.3268 and an F1 of 0.6011. Removing active sampling lowers these to 0.2914 and 0.4840, with the largest drop in the optimization score. Removing adaptive weighting lowers them to 0.2963 and 0.3055, with the largest drop in F1 and the smallest seed variance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of the Revision

We thank all reviewers for their time and constructive comments. We have uploaded a revised paper in which all changes are highlighted in blue. The main revisions are:

  • Oracle-call efficiency (Section 5.4, Table 1, Figure 4 in Appendix A.6). Following Reviewers pDDo, hGfr and 2zJo, we report the calls that every Stage-2 arm consumes, counted over distinct canonical structures. MolDAS uses 1,467 calls, against 2,291 for vanilla GRPO, 2,039 for DFRL-Mol and 2,153 for Genetic GFN, reductions of 36.0%, 28.1% and 31.9%, while scoring higher on both metrics. Figure 4 plots the Stage-2 optimization score of MolDAS and vanilla GRPO against cumulative distinct oracle calls. MolDAS peaks at 0.3354 after 921 calls, while vanilla GRPO peaks at 0.2934 only after 1,771, so the advantage is not an artifact of early stopping.

  • Three training seeds (Table 1, Appendix A.7). Following Reviewer pDDo, every entry of Table 1 is now a mean with standard deviation over three training seeds. In Stage 2 MolDAS reaches 0.3268±0.0168 Opt. Score and 0.6011±0.1161 F1, against 0.2896±0.0183 and 0.5033±0.1175 for vanilla GRPO, gains of 12.8% and 19.4%. In Stage 1 MolDAS also exceeds vanilla GRPO on both metrics, at 0.2727 against 0.2648 on Opt. Score and 0.4405 against 0.4050 on F1. Across both stages, MolDAS holds a consistent advantage over the baseline, and this advantage widens once oracle feedback becomes the training signal.

  • Isolating the Stage-2 strategy from the inherited initialization (Table 2). Following Reviewer hGfr, the upper panel of the new Table 2 repeats Stage 2 from a common Stage-1 checkpoint under the same cap. MolDAS reaches 0.3149 and 0.6064, against 0.2913 and 0.5763 for vanilla GRPO, gains of 8.1% and 5.2%, so the Stage-2 advantage does not come from the inherited initialization. The lower panel varies the schedule with the Stage-2 method and cap fixed. The full simulator-to-oracle schedule reaches 0.3268 and 0.6011, against 0.2971 and 0.5141 for direct oracle adaptation.

  • Stage-2 baselines (Table 1). Following Reviewers pDDo and 2zJo, DFRL-Mol and Genetic GFN now run complete two-stage pipelines under the common cap, reaching 0.2974/0.3877 and 0.2994/0.2661. Both score lower than MolDAS on both metrics while spending more calls. We label both as adapted and give every entry an explicit stage label.

  • Stage-2 component ablations (Table 1). Following Reviewers 2zJo and hGfr, all four components are now ablated in Stage 2 within each complete two-stage pipeline. Removing active sampling gives 0.2914/0.4840, output-side selection 0.2975/0.4778, adaptive weighting 0.2963/0.3055, and the high-value pool 0.3008/0.4894. Every removal lowers both metrics, so all four contributions transfer to the oracle stage.

  • Interaction analysis (Section 5.5). Following Reviewer 2zJo, we analyze both couplings. The uncertainty gate and the high-value pool are complementary rather than competing. The gate buys data at the capability frontier, where advantages are noisy, and the pool supplies reliable data whose low variance the auxiliary SFT loss turns into a stable update. The Stage-2 ablations confirm the split, with removal costs of 0.2914 for active sampling and 0.3008 for the pool against 0.3268. The controller and the sampler share the within-group reward dispersion, so the objective to push and the inputs to learn from meet at one bottleneck. Removing the sampler costs the most optimization score, and removing the controller the most F1.

  • Budget and accounting (Section 5.1, Appendix A.2). Following Reviewer 2zJo, we disclose that every Stage-2 run uses q = 2 distinct canonical structures per input group under a cap of 2,880, with 168 test inputs, 5,076 training inputs and 565 validation inputs. We also state the counting rules, covering canonical de-duplication, free cache hits, separately counted evaluation queries, and the policy of stopping rather than substituting feedback.

  • Positioning relative to DrugR (Sections 1, 2 and 3). Following Reviewer 2zJo, we state that MolDAS keeps DrugR's reward construction fixed, including its channel groups, its two-group scaling and its validity rules. Our rule-adaptive controller replaces DrugR's Pareto reweighting. It updates each reward weight from the recent trend of its channel under a bounded, weight-mass-preserving step.

  • Presentation (Section 5). Following Reviewer 2zJo, the incomplete sentence in Section 5.1 is repaired, and we audited the training description for consistent terminology and execution order.

The conclusions of the paper are unchanged. MolDAS improves the optimization score by 8.1% over vanilla GRPO at the matched Stage-1 start, and by 12.8% across complete two-stage pipelines while using 36.0% fewer distinct oracle calls. We respond to each reviewer below and are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.