- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether improvements in local molecular diversity translate into more efficient discovery in multi-turn molecular agents. The authors introduce EpisodeScope, an evaluation protocol that measures the number of distinct qualifying molecules per charged oracle call while accounting for realised spending and coverage at common call counts. Experiments show that farthest-first (MaxMin) selection improves within-episode structural diversity but underperforms uniform selection in qualified discovery efficiency. Further analyses examine repeated evaluations, anchor selection, and fixed-pool replay. The paper also reports substantial improvements from persistent molecular memory.
Critical strengths
- Meaningful evaluation perspective. The paper highlights the distinction between local structural diversity and long-term discovery efficiency. Accounting for actual oracle usage is practically relevant to molecular optimization.
- Careful empirical investigation. The authors provide matched comparisons, anchor analyses, fixed-pool replay experiments, and memory ablations. Their transparent reporting of experimental limitations and implementation deviations is also commendable.
Critical weaknesses
- Limited novelty and broader scientific significance of the findings. The paper's central observation is that increasing within-episode structural diversity does not necessarily improve the number of distinct high-scoring molecules discovered per oracle call. However, these objectives are not inherently aligned, and the importance of oracle efficiency, avoiding repeated evaluations, and persistent molecular memory has already been extensively explored in molecular optimization. EpisodeScope mainly combines established evaluation quantities rather than introducing a fundamentally new evaluation principle. The most distinctive empirical contribution is the observed underperformance of MaxMin selection relative to uniform selection. However, this reversal is primarily established on a single molecular optimization objective under a specific soft-budget agent configuration. The second molecular objective mainly examines the effectiveness of memory rather than reproducing the MaxMin reversal. Consequently, it remains unclear what sufficiently generalizable insights the study provides beyond demonstrating that local diversity is not always a reliable proxy for discovery efficiency in this particular setting.
- Insufficient explanation and experimental isolation of the main performance reversal. The paper reports substantial online performance degradation for MaxMin selection, but the underlying cause remains insufficiently established. First, the main experiments use a fixed first-position anchor rather than the random initialization of the referenced MaxMin-A method. Although the additional random-start control still underperforms uniform selection, the fixed-pool replay in Appendix B.3 shows that random-start MaxMin and fresh uniform selection achieve nearly identical qualified discovery rates, differing by at most 0.010 per call. This contrast raises an important unresolved question: why does MaxMin exhibit substantially worse performance during online training when its direct selection performance on fixed candidate pools is comparable to uniform selection? The observed anchor repetitions explain part of the behavior, but do not fully account for the random-start results. Interactions between selection, policy adaptation, and evolving proposal distributions may be important, yet these mechanisms are not systematically analyzed. Moreover, the realised candidate pools contain only four or five valid proposals at the median, making the selection regime substantially different from the referenced diversity-selection setting. As a result, the paper documents an interesting empirical reversal but does not sufficiently establish its underlying mechanism or the conditions under which it should be expected.
Questions for authors
1.Given that random-start MaxMin and uniform selection perform similarly in fixed-pool replay but differ substantially in online training, what evidence can distinguish the effects of selector choice from those of policy adaptation and changing proposal distributions? 2.What broader conclusions can be drawn from the observed MaxMin reversal beyond the specific molecular objective and experimental configuration? In particular, how might the substantially different candidate-pool sizes and selection ratios affect the comparison with the referenced diversity-selection method? 3.Could the authors further justify the choice of an unenforced oracle budget with zero call-cost penalty? Since training and validation calls differ substantially across methods, how should the reported efficiency results be interpreted in practical molecular optimization settings with limited total oracle budgets?
4 comments
Corrected 22-Cell Paired Study Results for Reviewer hxGz
Thank you again for your review. The frozen 22-cell paired study on Qwen3-4B and Osimertinib_MPO (25 GRPO updates) has finished. All 22 cells passed acceptance by reconciling requests, cache hits, and actual backend evaluations, with decision rules fixed before any result was examined. New Appendix H and the revised supplementary material report the full design, per-cell counts, and paired contrasts.
1. Fixed-pool and online differences. Your first question noted that random-start MaxMin and fresh uniform selection had small fixed-pool differences yet consistently negative contrasts in our submitted online runs. The corrected paired study uses stable per-request random draws and trains each block on identical hardware. It does not reproduce the submitted selector reversal. On the primary endpoint (distinct canonical molecules with raw score at least 0.567 first evaluated in the run's final 128 validation episodes, per actual backend evaluation in that window), MaxMin minus uniform is +0.3551, −0.0810, and +0.2085 across three seed blocks with caching off, and +0.0295, −0.0278, and −0.1232 with caching on. On the pre-specified secondary endpoint (final-window qualifying distinct molecules per selected score position), the contrast also changes sign under both caching settings (two positive and one negative in each). These three-block results leave the selector comparison inconclusive rather than proving equivalence, and they do not reproduce the submitted reversal, which we withdraw. A fixed-policy selector crossover would still be needed to separate policy adaptation from pure selection on evolved policies, and we do not claim to have isolated adaptation.
Caching and memory. Enabling raw-score caching raises the primary endpoint in all three blocks for both selectors (uniform +0.6208, +0.3947, +0.5246, and MaxMin +0.2952, +0.4479, +0.1929), partly because cache hits leave the denominator, whereas its secondary contrast changes sign (one positive and two negative under uniform, two positive and one negative under MaxMin). In the five cached memory blocks, at least 99% of the memory-off policy's final-window selections were cache hits, leaving zero actual evaluations in two blocks and one or two in the remaining three. The primary contrast is undefined in two blocks and negative in three (−0.0734, −0.0437, −0.0764). In the latter three, the memory-off rate is 1.0 based on one or two qualifying molecules. This denominator degeneracy makes the primary comparison uninformative about the memory effect. Following the pre-specified rule, the memory claim is restricted to the secondary endpoint, which increases with memory in all five blocks (median +0.5724, range +0.4422 to +0.7143). Directly addressing your budget question, memory also spends substantially more total evaluations across training and validation. Before the final window, the memory-on arms had actually evaluated 5,273 to 7,392 distinct molecules against 681 to 1,953 without memory, so the secondary gain does not show more discovery at a fixed total oracle budget (Appendix H).
Scope. The corrected study covers one model and one objective at 25 updates across three selector blocks and five memory blocks, reporting paired directions without significance tests (with at least 156,014 actual backend evaluations and 99,094 cache hits logged across all campaign stages). It neither reproduces nor refutes the published 640-to-64 MaxMin-A regime, and a fixed-policy crossover, a selection-ratio sweep, and a second-target selector comparison remain open. What the revised paper contributes is the EpisodeScope accounting protocol, which exposed the post-terminal charging defect, together with the corrected online evaluation, including an empirical regime where near-total cache reuse makes per-evaluation validation rates degenerate unless paired with cumulative spend and per-selection reporting. We claim neither a new diversity principle nor a general selector ranking in the revised abstract, introduction, Section 4.1, and conclusion.
Maintain my Weak Reject rating.
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address the experimental issues and improve the reliability of the study. However, the revised experiments also substantially narrow the paper's original findings, particularly regarding the central MaxMin performance reversal. The remaining contributions, which mainly focus on an evaluation and accounting framework, still lack sufficiently demonstrated novelty and broader applicability beyond existing molecular optimization benchmarks. Therefore, I maintain my Weak Reject rating.
Thank you for reviewing our additional 22-run campaign. Rather than merely narrowing our study, investigating the exact discrepancy you flagged in Question 1 (between fixed-pool replay differences of at most 0.010 in Appendix B.3 and the submitted online differences of −0.289, −0.162, and −0.156) uncovered two consequential failure modes in how multi-turn RLVR agents with costly oracles are evaluated, alongside a clean online test of local batch dispersion versus stateful exploration (Appendix H, pp. 47–49 of the 15 November revised PDF via the PDF icon beside the title, changes in blue):
-
Exposing off-record oracle activity in batched multi-turn RLVR: Replay and online runs already differ in source policies, proposal streams, purchase histories, and novelty windows, and reconciling episode records against service logs further revealed
673,923, and1,344excess pre-scoring selected positions over recorded episode charges across three3,968-episode runs, together with a reproducible batched-runner bug where terminated environments kept querying the oracle and mutating shared memory. Enforcing absorbing terminal states and exact reconciliation across requests, selected positions, cache hits, and backend evaluations (all 22/22 corrected runs) eliminates an execution hazard that arises whenever variable-length multi-turn RLVR rollouts share a stateful tool or oracle service. -
Uncovering a striking failure mode of standard per-call efficiency under caching ("100% precision on 1 call"): Standard cost-aware benchmarks rank agents by campaign-new qualifying discoveries per actual backend oracle call (our primary endpoint, ). In our corrected
GRPOruns with caching enabled (Table H.2, p. 48 and per-cell supplement records), removing memory causes the policy to repeat cached molecules on 99%–100% of final-window selections. Because cache hits leave the paid-call denominator,Memory OFFmakes0backend calls in two blocks (0/0) and just1or2calls in Blocks 100, 103, and 104, scoring1/1 = 1.0000and2/2 = 1.0000and outscoringMemory ON(303/327 = 0.9266,197/206 = 0.9563,145/157 = 0.9236) on per-backend-call efficiency despite discovering only5, 8, and 9window-qualifying molecules versus362, 250, and 202. Pairing with per-selection yield (, whereMemory ONwins all 5/5 blocks by+0.4422to+0.7143) and prior unique exploration (5,273–7,392vs.681–1,953actual evaluations) is essential to keep cached benchmarks from misranking collapsed policies. -
Clarifying what actually drives multi-turn RLVR discovery: Under reconciled online training, enforcing within-turn structural dispersion (
MaxMin) gives no consistent gain over fresh uniform selection at this 3-block scale (MaxMin − Uniformflips sign under both caching OFF and ON, Table H.1, leaving the selector contrast inconclusive rather than showing a reversal), whereas online score caching (+0.1929to+0.6208on across all 6/6 pairs) and persistent cross-episode memory (5/5 blocks on ) govern the consistent efficiency and exploration shifts.
Although our 22 prospective runs empirically evaluate Qwen3-4B with GRPO on Osimertinib_MPO (and do not test the 640-to-64 MaxMin-A regime or non-molecular tasks), the protocol requirements demonstrated here (terminal-boundary reconciliation, guarding against cache-induced denominator collapse, and separating window efficiency from cumulative training spend) apply whenever multi-turn LLM agents train against cached or stateful oracles. We hope this highlights the substantive value of the revised contribution.
Author Response to Reviewer hxGz
Thank you for the review. Your first question identified the tension between small fixed-pool differences and consistently negative submitted online contrasts. A frozen 22-cell paired study on Qwen3-4B has now finished, and our companion results comment reports Appendix H's paired results. The corrected study uses stable per-request seeds and fresh paired controls. Random-start MaxMin minus fresh uniform selection changes sign across three seed blocks, so the submitted selector reversal is not reproduced and we withdraw that claim. Isolating policy adaptation from pure selection on evolved policies would still require a fixed-policy selector crossover, which we have not run. Qualified yield, oracle efficiency, persistent memory, and separating local from cross-episode diversity build on established molecular optimization practice rather than introducing a new optimization principle.
1. Fixed-pool and online differences. On the four fixed candidate streams in Appendix B.3, random-start MaxMin and fresh uniform selection differ by at most 0.010 new qualifying molecules per call, whereas the submitted online comparisons differed by −0.289, −0.162, and −0.156. Those two sets of numbers cannot be subtracted to estimate policy adaptation because their source policies, proposal streams, purchase histories, and novelty windows differ (full-stream purchase history in replay versus within-window distinct molecules in the submitted online endpoint). Anchor repetition explains part of the first-start gap, not the random-start gap.
Reconciling episode records with service logs after submission identified a specific competing explanation. Across three complete random-start runs of 3,968 training and validation episode records each, service selection logs contain 673, 923, and 1,344 more selected score positions than recorded episode charges (full-run counts, distinct from the final window, and logged before scoring rather than counting completed backend calls). We reproduced a batched-runner defect where an already-terminated environment can keep requesting scores while another batch member continues, updating shared run memory while omitting those requests from the terminated episode's recorded outcome and cost. Missing historical identifiers prevent assigning every discrepancy to this cause, and adjusting denominators cannot repair altered memory state or feedback. We therefore do not interpret the submitted online-versus-replay gap as evidence of policy adaptation or proposal drift.
2. Generalization and the selection regime. In the logged random-start runs, the median realized pool contains four or five valid proposals with two selected for scoring, which is far less selective than the 640-to-64 regime in the referenced MaxMin-A study. Our first-start rule also used a fixed anchor rather than MaxMin-A's random start, our original uniform control seeded draws from the candidate-list hash, and the random-start follow-up reused earlier controls. Neither the submitted runs nor the corrected 22-cell study reproduces or refutes 640-to-64 MaxMin-A, and no general claim against diverse selection is supported. Likewise, the Perindopril_MPO extension and Qwen3-1.7B replication test memory rather than selector effects, while DPP and memory-plus-IPS remain mixed.
3. Unenforced budget and zero call-cost penalty. Using an unenforced budget with zero call-cost penalty exposes how interventions shift agent-chosen spending rather than fixing expenditure by construction. It estimates a soft-budget operating point rather than performance under a hard total campaign cap. Common-call validation curves adjust for unequal validation spending without equalizing training calls, and omitted post-terminal requests further prevented historical recorded-charge comparisons from showing equal actual spend. For a hard total oracle budget, validation rates must be read alongside cumulative training and validation expenditure. The corrected study logs both, and Appendix H reports the memory arms' earlier evaluation counts.
Corrected runner and open questions. The corrected implementation uses stable per-request seeds, separate selected-position, cache-hit, and backend-evaluation counters, and absorbing terminal states, reconciling all 652 study requests across 384 episodes to 933 actual evaluations in a one-update cache-off Qwen3-4B check. Our companion comment and Appendix H report the completed 22-cell study, while a fixed-policy selector crossover, a selection-ratio sweep, and a second-target selector comparison remain open.