acceptodds
Under review as a conference paper at ICLR 2027

BioDyad: Synchronize Biomedical Discovery and Machine Learning Engineering

Abstract

Agentic biomedical machine learning (ML) draws on complementary advances in biomedical evidence acquisition and executable program search. Existing systems connect aspects of these capabilities, but coordinating them throughout program search remains challenging. New evidence must guide candidate construction, execution outcomes must inform subsequent discovery and reuse, and validation demands must fit the search budget. We introduce BioDyad, which couples biomedical discovery and ML engineering through two hierarchies within Monte Carlo graph search. Its scientific hierarchy combines prior biomedical guidance with iterative discovery, then links biomedical plans to execution outcomes in memory for reuse across candidates. Its engineering hierarchy moves candidate programs from smoke execution, through train/validation evaluation, to full-data retraining. We evaluate BioDyad on the 76-task BioXArena benchmark under a two-hour per-task budget with three matched LLM backends. It achieves the highest penalized all-task score and task success rate among four agent methods and a one-shot baseline under each backend. These results support coordinating biomedical discovery and ML engineering to integrate external knowledge into executable programs across heterogeneous biomedical tasks. Code Available: https://anonymous.4open.science/r/BioDyad-7755/

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM agents, Automated machine learning, Biomedical discovery, Hierarchical agentic search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nRRgOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
3(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces BioDyad, a hierarchical agentic search framework that coordinates biomedical discovery and machine learning engineering under a fixed wall-clock budget. Within a shared Monte Carlo graph search (MCGS) backbone, its scientific hierarchy combines task-level prior guidance, node-level discovery, and biomedical memory linking candidate plans to execution outcomes. Its engineering hierarchy advances candidate programs through smoke execution, train/validation evaluation, and final full-data retraining and submission. The authors evaluate BioDyad on the 76-task BioXArena benchmark across nine biomedical domains under a two-hour per-task budget and three matched LLM backends. BioDyad achieves the highest penalized all-task score and task success rate under each backend. Component ablations, repeated runs, candidate-level diagnostics, and case studies provide additional evidence for the framework’s effectiveness.

Critical strengths
  1. The framework addresses a concrete coordination problem, and the component analyses support its design. Biomedical evidence can improve predictive representations while introducing integration errors and additional execution costs. BioDyad connects biomedical discovery to program generation and execution through plan-outcome memory and staged evaluation. In Table 4, the full framework achieves a penalized average score of 0.763, compared with 0.697 for MLEvolve w/ discovery. Removing biomedical memory or staged evaluation reduces the score to 0.736 and 0.733, respectively. Table 7 further shows that the full framework produces the largest number of valid full-execution runs within the fixed budget. These results support a useful system contribution: effective biomedical resource integration depends on how discovery, memory, and execution feedback are coordinated, rather than resource access alone.

  2. The broad evaluation and consistent aggregate gains across backends strengthen the empirical contribution. Under the same task interface, hardware allocation, and two-hour limit, BioDyad achieves penalized all-task scores of 0.738, 0.705, and 0.701 across the three backends. The improvements over the strongest matched alternatives are 0.023, 0.042, and 0.086. BioDyad also leads perturbation dynamics under every backend. Three independent runs on a fixed, domain-balanced 18-task subset provide preliminary support for robustness. For a paper whose main contribution is an integrated system, these results provide meaningful empirical support.

Critical weaknesses
  1. The component ablations support the framework, but finer-grained attribution remains incomplete. In the primary comparison, MLEvolve and BioXArena Harness do not access the biomedical resources available to BioDyad, although Biomni accesses its own data lake and tools. The resource-matched MLEvolve w/ discovery control therefore provides an important comparison, but it covers only two domains and 16 tasks using GPT-5.4, rather than one of the three main evaluation backends. The existing ablations support the contributions of biomedical memory and staged evaluation, but do not separately establish the benefit of node-level discovery over prior guidance alone, or the additional value of biomedical plan-outcome memory over a resource-matched generic execution-feedback memory. Broader resource-matched comparisons and targeted controls would strengthen the attribution of the observed gains.

  2. The budget-extension analysis characterizes candidate-pool improvement, but does not establish the corresponding benefit under the actual model-selection procedure. Section 5.5 and Appendix D explicitly report retrospective maxima of normalized hidden-test scores among completed candidates. This appropriately supports the stated conclusion that extended search can uncover better candidates. However, the deployed procedure selects a program using its pre-retraining validation score, so the analysis does not show whether the system can identify and submit those better candidates. In addition, because search selection depends on the elapsed budget fraction, the two-hour checkpoint of a 12-hour run occupies a different exploration/refinement stage from an independent two-hour run, even under the same scheduling rules. Reporting hidden-test performance for the validation-selected incumbent at each checkpoint would make the practical benefit of additional search time clearer. This limitation concerns the budget-extension analysis and does not directly undermine the two-hour main results.

Questions for authors
  1. Could you extend the resource-matched MLEvolve w/ discovery comparison to at least one main evaluation backend, and report repeated-run results or paired confidence intervals for the key ablations?

  2. With resources and budgets held fixed, how much does node-level discovery improve over using only the initial BIO_PLAN? How much additional benefit does biomedical plan-outcome memory provide over generic execution-feedback memory?

  3. Within each task, do candidates share fixed validation examples, splits, and metric aggregation rules? If candidate plans can change the validation strategy, how are scores kept comparable for search and final program selection?

4 comments
AuthorsOpenReview ↗

Response to Reviewer nRRg (1/2)

Thank you for your time and valuable feedback. Our replies to the weaknesses/questions are as follows.


W1a + Q1: The resource-matched control covers only two domains with GPT-5.4. / Could you extend it to at least one main evaluation backend, and report repeated-run results or paired confidence intervals?

Thank you for this precise assessment. We agree that the resource-matched control should be tested under a main backend. We ran it on all 8 perturbation-dynamics tasks, the domain where BioDyad leads under every backend, with DeepSeek-V4-Flash Preview, one of our three main backends. MLEvolve w/ discovery uses the same cache, tools and discovery components as BioDyad. Since biomedical discovery is itself part of BioDyad's scientific hierarchy, this control isolates the contribution of the remaining components, memory and staged evaluation. Each configuration was run three times (mean ± sample SD over runs); paired 95% CIs are bootstrapped over the 8 per-task means.

Configuration8-task penalized score
MLEvolve0.591 ± 0.011
MLEvolve w/ discovery (resource-matched)0.677 ± 0.011
BioDyad (full)0.723 ± 0.006
ComparisonMean difference95% CI
BioDyad − MLEvolve+0.132[0.047, 0.238]
MLEvolve w/ discovery − MLEvolve+0.087[0.024, 0.187]
BioDyad − MLEvolve w/ discovery+0.045[−0.004, 0.127]

From these results:

  • BioDyad outperforms MLEvolve on all 8 tasks (+0.132, 95% CI [0.047, 0.238]), so its advantage on perturbation dynamics in Table 2 holds under this main backend.
  • Discovery provides most of this gain (+0.087, higher on 7 of 8 tasks and tied on the other). As part of BioDyad's scientific hierarchy, it is what makes the shared resources usable.
  • The full framework adds a further +0.045, which comes from tasks where discovery alone fails to produce a valid or stable submission.

We have added these results to Sec. 5.3 and Appendix C.4 of the revised manuscript.

W1b + Q2: The ablations do not separate node-level discovery from prior guidance, or biomedical memory from generic execution-feedback memory. / With resources and budgets held fixed, how much does node-level discovery improve over using only the initial BIO_PLAN? How much additional benefit does biomedical plan-outcome memory provide over generic execution-feedback memory?

Thank you for this constructive question.

We apologize for the confusion caused by our description of memory. Generic execution-feedback memory is MLEvolve’s contribution rather than ours, so we retain it in all configurations and do not ablate it separately. Biomedical memory builds on this mechanism and extends it when enabled. BioDyad w/o memory (we now name it "w/o bio. memory") therefore falls back to MLEvolve’s generic memory while keeping the same resources and discovery, i.e., the resource-matched generic memory you describe. Comparing the full framework with this variant measures the benefit of the biomedical extension. We have clarified this in Sec. 4.1.2 of the revised manuscript.

Biomedical memory records are keyed by the NODE_PLAN from biomedical discovery, so the failures it retains and the repeats it avoids concern biomedical plans. This matters because integrating biomedical evidence introduces execution failures: adding discovery without biomedical memory raises invalid full runs from 248 to 358, and adding biomedical memory raises the valid share from 66.5% to 80.9% (Appendix C.3).

Following your suggestion, we ran both controls on all 8 perturbation-dynamics tasks with DeepSeek-V4-Flash Preview, one of our three main backends, with the same two-hour budget as Table 4 and three independent runs per variant (mean ± sample SD over runs).

ConfigurationPrior guidanceNode discoveryBio. memoryStaged eval.Penalized score
MLEvolve✗✗✗✗0.591 ± 0.011
MLEvolve w/ discovery✓✓✗✗0.677 ± 0.011
BioDyad w/o NODE_PLAN (prior-only)✓✗✓✓0.653 ± 0.017
BioDyad w/o bio. memory✓✓✗✓0.671 ± 0.011
BioDyad (full)✓✓✓✓0.723 ± 0.006

From these results:

  • Node-level discovery adds value beyond the initial BIO_PLAN. With resources and budget fixed, keeping only prior guidance lowers the score from 0.723 to 0.653. The same holds in Table 4 (GPT-5.4, chemical biology and phenotype-disease): 0.723 vs. 0.763.
  • Biomedical memory adds value beyond generic memory. Falling back to MLEvolve's generic execution-feedback memory lowers the score from 0.723 to 0.671.

We have added these results to Sec. 5.3 (Tables 4 and 5) of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer nRRg (2/2)

W2: The budget-extension analysis characterizes candidate-pool improvement, but does not establish the corresponding benefit under the actual model-selection procedure.

Thank you for raising this insightful point.

We reported retrospective maxima of hidden-test scores in Table 6 (Table 5 in the original submission) to show that gains emerge from candidate exploration, but we agree that these candidates are not the ones the system would finally submit. In order to verify practical gains to a larger budget, we re-select candidates by validation scores and compare them in the following table:

Domain (GPT-5.6-sol)2h4h6h8h10h12hGain 2 to 12h
Chemical biology, validation-selected0.9000.9230.9240.9240.9310.929+0.029
Chemical biology, retrospective maximum0.9270.9320.9360.9360.9360.937+0.010
Phenotype-disease, validation-selected0.5800.6240.6130.6130.6200.631+0.051
Phenotype-disease, retrospective maximum0.6400.6630.6880.6900.6900.690+0.050

From this table:

  • The most critical observation is that the gains still exist when selecting candidates by validation scores (+0.029/+0.051 on chemical biology and phenotype-disease), so the system does identify and submit much of the improvement found by extended search.
  • As you rightly pointed out, the 2h checkpoint of a 12-hour run is at a different exploration stage from an independent two-hour run, which is why it falls below Table 2. Relative to Table 2, the gains are +0.003/+0.037 on chemical biology and phenotype-disease. The gain depends on headroom: chemical biology already starts high (0.926 in Table 2), while phenotype-disease has more room to improve.

We have added the validation-selected rows to Table 6 and revised Sec. 5.5 accordingly.

Q3: Within each task, do candidates share fixed validation examples, splits, and metric aggregation rules? If candidate plans can change the validation strategy, how are scores kept comparable for search and final program selection?

Thank you for this constructive question.

Within a task, candidates share the metric and its aggregation rule, but not the validation examples or split: following MLEvolve, each candidate defines its own hold-out or CV split, and a candidate plan can change it. Validation scores are therefore not strictly comparable across candidates, either during search or at final selection. We kept MLEvolve's protocol so that BioDyad and the MLEvolve baseline select programs under the same rule.

This affects which program is selected, not how results are scored: all reported numbers are hidden-test scores from the official BioXArena evaluator. Validation scores remain informative: in the 12-hour runs of Table 6, the median Spearman correlation between candidates' validation and hidden-test scores is 0.64 across the 16 tasks. A split shared across candidates would make validation scores directly comparable, and we note this in Appendix B.2.

We have clarified the validation protocol in Appendix B.2 and added the validation–test correlation to Appendix D.1.


We hope these responses address your concerns, and we would be happy to discuss any further questions.

Thank you for the detailed response and additional experiments. The new experiments strengthen the paper and sufficiently address my concerns. I appreciate the authors’ careful revisions and will maintain my rating of 3 (Weak Accept).

Thank you for your positive feedback and for recognizing our revisions. We are glad that the additional experiments have sufficiently addressed your concerns. We sincerely appreciate your thoughtful review and your support for our work.

Reviewer_rsyLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents BioDyad, a framework that extends MLEvolve with biomedical resource discovery, memory of plans and execution outcomes, and staged program evaluation. It uses execution feedback to guide subsequent biomedical planning, aiming to improve predictive performance and execution reliability. The authors evaluate the framework on 76 BioXArena tasks across nine domains and three LLM backends, with component ablations and execution analyses examining the proposed design.

Critical strengths
  1. The paper addresses a relevant problem by connecting biomedical knowledge acquisition with program search and execution feedback.

  2. BioDyad achieves the highest overall mean score across all three evaluated backends under a common hardware and time budget. The coverage of 76 tasks across nine domains provides substantial evidence for the practical value of the complete system.

  3. Table 4 includes a resource-matched baseline and multiple component ablations, providing useful evidence for the combined design.

Critical weaknesses
  1. The paper uses biomedical information both before and during the search, but Table 4 does not separate these two steps. The example showing improved prediction also uses annotations included in the first draft. It is therefore unclear how much continued discovery adds beyond the initial guidance.

  2. BioDyad performs best on perturbation prediction under all three models in the main experiments. However, the component analysis covers two other task groups and uses a different model. It therefore does not show which components account for this consistent advantage.

Questions for authors
  1. How does performance change if biomedical guidance is provided only at the start, while memory and staged execution are retained? Can the authors show an example where earlier execution results lead to a useful change in the biomedical information used later?
  2. MLEvolve already stores and reuses experience from earlier attempts. What changes does BioDyad make to the information stored and the way it is reused? In Table 4, is MLEvolve's original memory retained when biomedical memory is removed?
  3. Could the authors test the contributions of discovery, memory, and staged execution on perturbation prediction using one of the models from the main experiments? This would help explain which components drive the improvement.
4 comments
AuthorsOpenReview ↗

Response to Reviewer rsyL (1/2)

Thank you for your time and valuable feedback. Our replies to the weaknesses/questions are as follows.


W1 + Q1a: The paper uses biomedical information both before and during the search, but Table 4 does not separate these two steps. / How does performance change if biomedical guidance is provided only at the start, while memory and staged execution are retained?

Thank you for raising this constructive point.

We agree that Table 4 did not separate prior guidance from candidate-level discovery. We also agree that the annotations in the predictive-gain case (Fig. 4a) were introduced by prior guidance in the first draft, so that case does not show what node-level discovery adds. We address this in two ways: a variant that keeps only prior guidance (below, and on perturbation dynamics in W2+Q3), and a new example in which an earlier execution result changes the biomedical information used later (Q1b).

Following your suggestion, we add BioDyad w/o NODE_PLAN to Table 4, which keeps prior guidance, biomedical memory, and staged evaluation but performs no node-level discovery:

HierarchyMechanismMLEvolveMLEvolve w/ discoveryBioDyad w/o bio. memoryBioDyad w/o Sci.BioDyad w/o Eng.BioDyad w/o NODE_PLANBioDyad (full)
Sci. hierarchyPrior guidance✗✓✓✗✓✓✓
Node-level discovery✗✓✓✗✓✗✓
Bio. memory✗✗✗✗✓✓✓
Eng. hierarchyStaged evaluation✗✗✓✓✗✓✓
Performance ↑Chemical biology0.8980.7950.9130.8610.8960.9060.919
Phenotype-disease0.5380.5990.5600.4680.5700.5400.606
Average0.7180.6970.7360.6650.7330.7230.763

From this table, the average score of BioDyad w/o NODE_PLAN drops from 0.763 to 0.723 compared with BioDyad (full). The reasons are twofold: 1) Biomedical memory remains active in this variant, but what it stores changes: without node-level discovery, it records plans written under the initial guidance instead of node-level biomedical plans. 2) Without a node-level biomedical plan, the program search is restricted to the initial guidance.

We have added this analysis to Sec. 5.3. We also compare them using DeepSeek-V4-Flash Preview backend in our answer to your W2+Q3.

Q1b: Can the authors show an example where earlier execution results lead to a useful change in the biomedical information used later?

Example (breast-cancer-subtype task from phenotype-disease domain, GLM-5.1 backend). One candidate’s NODE_PLAN proposed adding MSigDB hallmark pathway scores, computed as the mean expression of each pathway’s genes among the 200 measured genes. Its execution log reported “Added 0 MSigDB hallmark pathway scores”: the gene lists, stored as serialized lists in the data-lake table, were not parsed, so the program ran without any pathway information (validation macro F1 0.619). The next candidate was generated from this program and its execution record. It parsed the gene lists correctly, so the biomedical information actually used changed from no pathways to 47 hallmark pathway scores. Validation macro F1 rose to 0.641, hidden-test macro F1 rose from 0.719 to 0.728, and this candidate produced the submitted program.

We have added this example to Appendix E.3.4 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer rsyL (2/2)

W2 + Q3: The component analysis covers two other task groups and uses a different model. It therefore does not show which components account for this consistent advantage. / Could the authors test the contributions of discovery, memory, and staged execution on perturbation prediction using one of the models from the main experiments? This would help explain which components drive the improvement.

Thank you for raising this constructive point.

We originally ran the ablations with GPT-5.4 to decouple the component analysis from the three main evaluation backends, but we fully agree that the strongest-gain domain under a main backend better explains where the gains come from. Therefore, following your suggestion, we ran the full component ablation on all 8 perturbation-dynamics tasks with DeepSeek-V4-Flash Preview, one of our three main backends, with the same two-hour budget as Table 4 and three independent runs per variant (mean ± sample SD over runs).

ConfigurationPrior guidanceNode discoveryBio. memoryStaged eval.Penalized score
MLEvolve✗✗✗✗0.591 ± 0.011
MLEvolve w/ discovery✓✓✗✗0.677 ± 0.011
BioDyad w/o Sci.✗✗✗✓0.598 ± 0.012
BioDyad w/o NODE_PLAN✓✗✓✓0.653 ± 0.017
BioDyad w/o bio. memory✓✓✗✓0.671 ± 0.011
BioDyad w/o Eng.✓✓✓✗0.698 ± 0.009
BioDyad (full)✓✓✓✓0.723 ± 0.006

From these results:

  • The scientific hierarchy is the main source of the advantage on perturbation dynamics. Removing it causes the largest drop (0.723 → 0.598), and adding discovery to MLEvolve alone already raises the score from 0.591 to 0.677.
  • Continued discovery adds value beyond the initial guidance. Keeping only prior guidance (w/o NODE_PLAN) lowers the score from 0.723 to 0.653, so candidate-level discovery during search contributes beyond the plan made before search.
  • Memory and staged evaluation also contribute: removing either lowers the score (0.723 → 0.671 and 0.698).

We have added this ablation to Sec. 5.3 (Table 5) of the revised manuscript.

Q2a: MLEvolve already stores and reuses experience from earlier attempts. What changes does BioDyad make to the information stored and the way it is reused?

Thank you for this constructive question.

In BioDyad, we made the following major changes compared to MLEvolve’s generic memory mechanism:

  1. Add biomedical node plans to the memory. MLEvolve stores the planner’s text plan and a code summary. BioDyad additionally stores the node plan produced by biomedical discovery, linked to its execution outcome (parent and child scores, or the error summary).
  2. Add buggy executions to the memory. MLEvolve only stores nodes that run successfully and produce a score, so failed executions are discarded. BioDyad also records buggy and smoke-failed nodes with their error messages, so later nodes can see which integrations failed and why.
  3. Avoid repeated plans before a new node is expanded. Each plan is recorded as pending before code generation and updated after execution. Before expanding a new node, the discovery agent retrieves similar successful, pending and failed plans and is asked to propose a different one. MLEvolve, in contrast, uses memory only after a plan has been drafted, to refine it or to guide debugging.

These three changes reflect the characteristics of biomedical tasks: 1) new biomedical knowledge enters the search at the node level; 2) integrating this knowledge often introduces new execution errors; and 3) with parallel expansion and a fixed budget, repeating an already failed or ongoing plan wastes budget.

We have clarified these differences in Sec. 4.1.2 of the revised paper.

Q2b: In Table 4, is MLEvolve's original memory retained when biomedical memory is removed?

Yes. In Table 4, BioDyad w/o bio. memory (named "w/o memory" in the original submission) removes only biomedical memory and keeps MLEvolve's generic memory, which is retained in all configurations. We apologize for the confusion caused by our description of memory, and we have clarified this in Sec. 4.1.2 of the revised manuscript.


We hope these responses address your concerns, and we would be happy to discuss any further questions.

Thank you for the additional experiments and clarifications. The new ablations and the explanation of the memory mechanism have largely addressed my main concerns.I will keep my rating at 3, but your response has strengthened my support for acceptance. If finer-grained ratings were available, my score would be around 3.5.

Thank you for your follow-up and for your strengthened support for acceptance. We are glad that the new ablations and the clarification of the memory mechanism have addressed your main concerns. We greatly appreciate your constructive feedback, which has helped us improve the work.

Reviewer_4vMBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates biomedical ML program search under a fixed budget. BioDyad combines biomedical prior guidance, candidate-level discovery and plan–outcome memory with staged smoke testing, train and vlaidation execution and final retraining. Its graph search and budget-dependent selection strategy build on MLEvolve. The evalaution is on 76 BioXArena tasks.

Critical strengths
  1. Consistent evaluation: Matched backends and two-hour limits across 76 heterogeneous tasks, plus a predefined repeated-run subset, make the system result materially more credible than isolated success cases.
  2. Useful component controls and failure analysis: The resource-matched discovery control, memory/staging removals and a negative feature-integration case show that merely providing more biomedical information is insufficient in these experiments. This provides actionable understanding of the integration problem.
Critical weaknesses
  1. Generalization of the framework:. The strongest general MLE baseline lacks the biomedical cache in Table 2, while the resource-matched control is limited to 16 tasks and a different backend (Sec. 5.1; Table 4). If comparable resource access eliminates the advantage in the strongest-gain domains, the case for a general framework contribution weakens substantially.
  2. The additional-search evidence does not measure model's improvement. Tables 5 and 8 select retrospectively by hidden-test score, whereas the method selects by validation score (Sec. 4.2; App. D.1). Without validation-selected checkpoint results, these plots do not support practical returns to a larger budget. This limits one supporting claim rather than overturning the main two-hour comparison.
Questions for authors
  1. Does a resource-matched MLEvolve + discovery control retain the main advantage on perturbation-dynamics tasks, with repeated task-level results (Table 2 versus Table 4)?

  2. How are candidate validation folds/preprocessing standardized, and how do you exclude benchmark target records from the local data lake?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 4vMB (1/2)

Thank you for your time and valuable feedback. Our replies to the weaknesses/questions are as follows.


W1 + Q1: Generalization of the framework. / Does a resource-matched MLEvolve + discovery control retain the main advantage on perturbation-dynamics tasks, with repeated task-level results (Table 2 versus Table 4)?

Thank you for raising this concern.

MLEvolve has no mechanism to search or integrate the biomedical cache, so providing the cache alone would not enable MLEvolve to use it. Making such resources usable is itself part of BioDyad's contribution: biomedical discovery, a component of BioDyad's scientific hierarchy, derives a task-level plan from the cache before search (prior guidance) and candidate-specific plans as each node is expanded (node-level discovery). The resource-matched control, MLEvolve w/ discovery, attaches both discovery components to MLEvolve with the same cache and tools. It is therefore a partial BioDyad rather than an external baseline, and comparing it with the full framework isolates memory and staged evaluation.

In addition, in Table 2, Biomni uses the same data lake with its own agent and trails BioDyad on perturbation dynamics under all three backends (0.404 / 0.618 / 0.571 vs. 0.776 / 0.737 / 0.717).

We agree that this control should be tested on the strongest-gain domain under a main backend. Following your suggestion, we ran this control on all 8 perturbation-dynamics tasks with DeepSeek-V4-Flash Preview, one of our three main backends, with three independent runs per configuration (mean ± SD over runs; 0.000 indicates no valid submission).

TaskMLEvolveMLEvolve w/ discoveryBioDyad (full)
Cancer drug sensitivity0.960 ± 0.0130.969 ± 0.0020.972 ± 0.002
CRISPR perturbation prediction0.525 ± 0.0220.946 ± 0.0070.944 ± 0.003
Drug transcriptional response0.514 ± 0.0120.559 ± 0.0040.558 ± 0.002
ECCITE multimodal perturbation0.659 ± 0.1310.726 ± 0.0060.730 ± 0.008
Gene regulatory network inference0.000 ± 0.0000.000 ± 0.0000.316 ± 0.024
Multi-timepoint perturbation0.920 ± 0.1350.937 ± 0.1050.997 ± 0.000
RNA velocity cell transition0.540 ± 0.0450.582 ± 0.0050.566 ± 0.034
SPEAR-ATAC perturbation0.607 ± 0.0990.699 ± 0.0110.696 ± 0.007
8-task mean0.591 ± 0.0110.677 ± 0.0110.723 ± 0.006

Number of generated candidates per run, summed over the 8 tasks:

ConfigurationBio. discoveryBio. memoryStaged eval.Run 1Run 2Run 3Mean ± SD
MLEvolve✗✗✗423615770603 ± 174
MLEvolve w/ discovery✓✗✗12113492116 ± 22
BioDyad (full)✓✓✓157123161147 ± 21

From these results:

  • Discovery provides most of the gain over MLEvolve (+0.087), even though it reduces the number of candidates by about 80%. Biomedical information outweighs the extra search it displaces.
  • The full framework adds a further +0.045, concentrated in two tasks: gene regulatory network inference and multi-timepoint perturbation. On the other six tasks, the two configurations are comparable.
  • Across domains, discovery alone is not a reliable substitute for the full framework. In Table 4 (GPT-5.4), the same control falls below MLEvolve on chemical biology (0.795 vs. 0.898) and on average (0.697 vs. 0.718), while the full framework reaches 0.919 and 0.763. Over the three domains tested, only the full framework is best in every domain.
  • Perturbation dynamics is thus the domain where discovery alone already works well. Memory and staged evaluation are what make the gain hold across domains, consistent with their effect on execution reliability: the valid share of full runs rises from 46.7% for MLEvolve w/ discovery to 80.9% for the full framework (Table 8, Appendix C.2).

We have added these results to Sec. 5.3 and Appendix C.4 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 4vMB (2/2)

W2: The additional-search evidence does not measure model's improvement.

Thank you for raising this insightful point.

We reported retrospective maxima of hidden-test scores in Table 6 (Table 5 in the original submission) to show that gains emerge from candidate exploration, but we agree that these candidates are not the ones the system would finally submit. In order to verify practical gains to a larger budget, we re-select candidates by validation scores and compare them in the following table:

Domain (GPT-5.6-sol)2h4h6h8h10h12hGain 2 to 12h
Chemical biology, validation-selected0.9000.9230.9240.9240.9310.929+0.029
Chemical biology, retrospective maximum0.9270.9320.9360.9360.9360.937+0.010
Phenotype-disease, validation-selected0.5800.6240.6130.6130.6200.631+0.051
Phenotype-disease, retrospective maximum0.6400.6630.6880.6900.6900.690+0.050

From this table:

  • The most critical observation is that the gains still exist when selecting candidates by validation scores (+0.029/+0.051 on chemical biology and phenotype-disease).
  • The 2h results are not consistent with those in Table 2 because the exploration strategy is affected by the total budget (Eq. 8). The real gains over those in Table 2 are +0.003/+0.037 on chemical biology and phenotype-disease. The gain depends on headroom: chemical biology already starts high (0.926 in Table 2), while phenotype-disease has more room to improve.

We have added the validation-selected rows to Table 6 and revised Sec. 5.5 accordingly.

Q2a: How are candidate validation folds/preprocessing standardized?

Thank you for this constructive question.

What is standardized. Each task fixes the metric, and every candidate reports the official task metric on its held-out data, with validation and test inputs passing through the same preprocessing.

What is not standardized. Following MLEvolve, each candidate defines its own hold-out or CV split, so validation scores from different candidates are not strictly comparable. We kept this protocol so that BioDyad and the MLEvolve baseline select programs under the same rule.

This affects which program is selected, not how results are scored: all reported numbers are hidden-test scores from the official BioXArena evaluator. Validation scores still rank candidates reasonably: in the 12-hour runs of Table 6, the median Spearman correlation between candidates' validation and hidden-test scores is 0.64 across the 16 tasks. A split shared across candidates would make validation scores directly comparable; we note this in Appendix B.2 as a way to improve selection.

Q2b: How do you exclude benchmark target records from the local data lake?

Thank you for raising this important point.

We do not remove records from the cache; instead, we prevent the system from using target records. Each candidate program is checked by a code-review agent (see ./agents/BioDyad/agents/code_review_agent.py in our anonymous repository). It removes any use of cache records that duplicate benchmark labels or are target-equivalent, while public covariates keyed by non-label identifiers (e.g., genes, compounds, SMILES) remain allowed. The same check is enabled in every configuration with access to the cache, including MLEvolve w/ discovery and Biomni, and runs within the two-hour budget. We have described this in Appendix B.1.


We hope these responses address your concerns, and we would be happy to discuss any further questions.

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer 4vMB,

As the discussion period ends in less than 20 hours, we would greatly appreciate it if you could take a look at our response when time permits. It aims to address your two main concerns: the resource-matched comparison on perturbation dynamics and the budget extension under validation-based selection. The new experiments are now included in the revised manuscript. If our response resolves your concerns, we would appreciate it if you could take this into account in your final assessment; otherwise, we are happy to answer any remaining questions.

Thank you for your time.

Best regards,
Authors of Submission 10634

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

Dear Area Chairs and Reviewers,

We sincerely thank the Area Chairs for organizing the review process, and all reviewers for their time, valuable comments, and constructive suggestions.

The initial reviews (ratings: 2, 3, 3) recognize several strengths of BioDyad. The reviewers recognize that it addresses a relevant and concrete problem, connecting biomedical knowledge acquisition with program search and execution feedback (rsyL, nRRg), and that the evaluation is consistent and broad, covering 76 heterogeneous tasks with matched backends and a common two-hour budget (4vMB, rsyL, nRRg). BioDyad achieves the highest overall score under all three backends (rsyL, nRRg), and the component controls and failure analysis show that providing more biomedical information is not enough on its own, which gives actionable insight into the integration problem (4vMB, nRRg).


During the discussion, we focused on addressing the following primary concerns, which have been incorporated into our revised manuscript (changes marked in blue):

  • Resource-matched comparison on a main backend (4vMB, nRRg): We clarified that biomedical discovery is itself one of BioDyad's contributions. MLEvolve has no mechanism to search or use the biomedical cache, so the resource-matched control, MLEvolve w/ discovery, is a partial BioDyad rather than an external baseline. We then extended this control to all eight perturbation-dynamics tasks with DeepSeek-V4-Flash Preview, one of our main backends, over three independent runs. BioDyad outperforms MLEvolve on every task (0.723 vs. 0.591). Beyond discovery, biomedical memory and staged evaluation still bring a consistent gain over MLEvolve w/ discovery, both on these tasks (+0.045) and in Table 4 (+0.066), where discovery alone even falls below MLEvolve. The full framework is therefore the best configuration in both settings (Sec. 5.3, Appendix C.4).

  • Contribution of each component (rsyL, nRRg): We ablated each component on all eight perturbation-dynamics tasks with DeepSeek-V4-Flash Preview, with three runs per variant (new Table 5). Removing the scientific hierarchy causes the largest drop (0.723 → 0.598). Keeping only the initial prior guidance, without node-level discovery, lowers the score to 0.653 (and from 0.763 to 0.723 in Table 4), so continued discovery adds value beyond the initial BIO_PLAN. Removing biomedical memory, which falls back to MLEvolve's generic execution-feedback memory, lowers the score to 0.671, and removing staged evaluation lowers it to 0.698. We also clarified how biomedical memory extends MLEvolve's memory: it stores the node-level biomedical plans together with buggy and smoke-failed attempts, and it is retrieved before a new node is expanded so that failed or ongoing plans are not repeated (Sec. 4.1.2).

  • Budget extension under the deployed selection rule (4vMB, nRRg): Besides the retrospective best candidate, we now report the hidden-test score of the program the system would actually submit, i.e., the candidate with the best validation score at each checkpoint. Extending the search from 2 to 12 hours still improves the submitted program by 0.029 in chemical biology and 0.051 in phenotype–disease (Table 6, Sec. 5.5). We also note that the 2-hour checkpoint of a 12-hour run differs from an independent 2-hour run, because the search schedule depends on the elapsed fraction of the budget.

Other discussions/suggestions, including the validation protocol across candidates (4vMB, nRRg), the prevention of target leakage from the data lake (4vMB), and an example in which execution feedback corrects a biomedical integration (rsyL), have also been integrated into the revised manuscript (Appendices B.1, B.2, D.1, and E.3.4). We appreciate these valuable comments, which make our work more complete.


We thank all reviewers again for their constructive feedback. We hope this work offers a useful perspective on coordinating biomedical discovery and machine learning engineering under a fixed budget.

With two days remaining in the discussion phase, we would be happy to address any further questions or concerns. We would also greatly appreciate it if the reviewers could let us know whether our responses have resolved the issues they raised.

Best regards,
Authors of Submission 10634

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.