acceptodds
Under review as a conference paper at ICLR 2027

Hypothesis Portfolio Search: Shared Feedback for Genetic Perturbation Discovery

10 views

Abstract

LLM agents can effectively link biological knowledge to genetic perturbation experiments, yet tight budgets leave most plausible interventions untested. Active hit discovery aims to maximize the recovery of assay-specific hits within these limits. However, standard scoring methods select candidates purely by global rank, risking over-concentration on dominant pathways while neglecting under-explored biological mechanisms, whereas prompt-based LLM agents struggle to convert accumulating feedback into quantitative, assay-specific value estimates. To address this, we formulate discovery as two coupled decisions: allocating testing budgets across hypothesis-defined candidate pools, and selecting specific genes within each pool. We propose **Hy**pothesis **P**ortfolio **S**earch, termed **HyPS**, a novel framework that treats hypotheses as revisable search units with dynamic budget allocations. Feedback iteratively updates these units, while a shared lightweight scorer integrates historical responses with biological knowledge to rank untested candidates. We evaluate HyPS retrospectively on six single-gene perturbation screens using nine LLM backbones at matched testing budgets. Averaged equally across screens and backbones, HyPS raises overall hit recall from 8.40% to 12.47%, outperforming the same-backbone retrieval-augmented agent in 53 of 54 comparisons. On a two-gene perturbation benchmark, HyPS increases mean hit-pair recovery by 72.4% over the same comparator across three LLM backbones. Our code will be made publicly available upon publication.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Large Language Models, LLM Agents, AI for Scientific Discovery, Genetic Perturbation Discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_np2LOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces Hypothesis Portfolio Search (HyPS), a framework for selecting genetic perturbation experiments under a limited testing budget. The method is split in two submethods that interact hierarchically. The top method supervises K individual methods that perform individual searches. An LLM uses the scientific goal, the stated hypothesis and the history of each specific assay as input to decide whether each individual search will stop and propose budget and certain genes that will serve as anchors for each hypothesis. The anchors proposed will be used to derive the candidate pool, which are basically the closest genes to the anchors using their Achilles representation for similarity. A module updates the quotas for each hypothesis throughout the rounds. Inside each candidate level search, there isa score which is a model trained on the historical assays. They are small logistic-regressions with three features per gene, representing the experimental effects already observed in genes based on three different similarities. The whole method runs in a loop and is updated based on the update of the history.

Critical strengths

The work tackles a very important and interesting problem on building assays for genomic experiments to increase the discovery rate.

The inclusion of an LLM as supervisor and a common regression for each search is an interesting idea.

Critical weaknesses

To my understanding, the internal threshold is derived from the complete benchmark hit set. This means that both allocation and scorer training use information beyond completed experiments.

The description of the methodology is hard to follow Sections 3 and 4 could improve in clarity significantly. For example slots and anchors are not clearly defined. The whole paragraph on Bounded adaptive quotas is hard to follow.

The method is not compared to TYpiclust or ACS-FW or most importantly, random selection [1].

[1] Huang, Kexin, et al. "Sequential optimal experimental design of perturbation screens guided by multi-modal priors." International Conference on Research in Computational Molecular Biology. Cham: Springer Nature Switzerland, 2024.

The code is not available to reproduce the results, which, for a non-theretical paper, is vital.

The theoretical or technical novelty of the work is limited.

Questions for authors
  • How much do the reported gains depend on deriving τ\tau from the complete benchmark hit set, and do they persist when the threshold is specified independently

  • Can you provide results for random selection, TypiClust and ACS-FW under the same experimental budgets?

  • Can you provide the implementation, configurations, and run records?

  • What is the principal technical advance beyond combining LLM based retrieval with scoring based on similarity?

10 comments
AuthorsOpenReview ↗

1. τ encodes the hit criterion, and HyPS's gains persist with a label-blind threshold

To my understanding, the internal threshold is derived from the complete benchmark hit set. This means that both allocation and scorer training use information beyond completed experiments.

We thank the reviewer for raising this important point.

  • Labels only after testing: τ\tau is a single scalar fixed before search that encodes the hit criterion; it contains no gene identities. HyPS labels a gene only after testing it and never observes any untested outcome.

  • τ\tau conveys the hit definition: On four of six screens, the label of a tested gene is identical to its hit status (new Table 4). On Sanchez, τ\tau carries no information at all, and HyPS still outperforms BDA for all nine backbones.

  • Label-blind experiment: We removed τ\tau entirely. The threshold is computed each round from tested genes only (robust z ≥ 2, fixed before any run). On a representative subset fixed in advance (IFNG, IL2, Carnevale × GPT-5.6 Sol, GPT-5.6 Terra, Claude Sonnet 5; ten runs each):

BDAHyPS (original)HyPS (label-blind)
Mean All / N/E recall (%)10.20 / 5.2715.81 / 8.2315.38 / 7.72

Label-blind HyPS outperforms BDA in all 9 pairs on both metrics and retains 92% / 83% of the gain. Per-pair results are given in our response to Reviewer p8h2 (Comment 2).

Revision in paper: §3, ¶"Interactive protocol" and ¶"Hit criterion and internal feedback labels"; Table 4 (App. A.3); §5.2, ¶"Independent threshold"; Table 20 (App. C.10).

AuthorsOpenReview ↗

2. Slots, anchors, pools, and quotas are now defined with an example and Algorithm 1

The description of the methodology is hard to follow Sections 3 and 4 could improve in clarity significantly. For example slots and anchors are not clearly defined.

We thank the reviewer for this helpful feedback. §4.1 now defines each term with a concrete example:

  • Slot: One of KK persistent positions in the portfolio.

  • Unit: The content of a slot, i.e., a natural-language hypothesis plus three anchor genes. For example, "TCR proximal signaling loss reduces IL-2 output" with anchors LCP2, ZAP70, and LAT.

  • Pool: The untested genes retrieved by Achilles similarity to the anchors.

  • Quota: The number of tests the slot receives in the round.

  • Algorithm 1: Shows how these elements interact in one complete round.

  • Explicit protocol: §3 now defines what the agent observes in each round.

  • Clarification of the summary: In HyPS, the LLM keeps, revises, or replaces each unit, while testing budgets are assigned by the bounded allocation rule (Eq. 2–3) from observed outcomes. This keeps the budget decision quantitative and feedback-driven.

Revision in paper: §3; §4.1; §4.2; Algorithm 1.

AuthorsOpenReview ↗

3. The quota rule is now an explicit three-step procedure

The whole paragraph on Bounded adaptive quotas is hard to follow.

We thank the reviewer for this comment. The paragraph is rewritten as an explicit procedure:

  1. Priority: Each slot receives a priority (Eq. 2, moved from the appendix): a smoothed event rate plus a low-observation bonus, in the spirit of upper-confidence-bound allocation.
  2. Floor: Every slot first receives qmin⁡q_{\min} tests.
  3. Assignment: The remaining tests are assigned one at a time by priority, subject to the cap qmax⁡q_{\max} (Eq. 3).
  • Design rationale: The floor keeps every direction explored, and the cap prevents over-concentration on early successes. With K=4K=4 and b=128b=128, the bounds are 16–48 and the initial quotas are equal.

Revision in paper: §4.2, ¶"Bounded adaptive quotas", Eq. (2)–(3); Algorithm 1, line 8.

Reviewer_FLFHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a bi-level method for improving search efficiency in genomic perturbation studies.The method consists of two pieces: a high-level strategic search strategy that uses LLMs to operate on broad gene / pathway-level hypotheses, and a low-level search that fits a linear logistic model on feature embeddings and previous experiment results to select which genes within a portfolio slot to test next.

Critical strengths
  • The paper proposes an intuitively reasonable method for conducting genetic perturbation search by designing a scaffold that leverages a biologically-grounded inductive bias, having the LLM reason over a mechanistic hypothesis space, while a fast logistic regression algorithm performs the low-level search over genes within each mechanistic category.
  • The empirical evaluations compare against both LLM and non-LLM-based search strategies, and consider a variety of backbones.
  • The paper conducts several ablations over algorithmic choices. Although some of these ablations could be better-analyzed, they give helpful insight into the method.
  • The figures are readable and clearly annotated.
  • Although many details are missing from the main paper, the appendix included most of the missing information I looked for.
Critical weaknesses
  • The paper is grammatically correct but written in a way that makes it extremely difficult to understand the technical details of the method. The writing alone is poor enough to justify rejection in my view, as I don’t think a reader can easily extract the important information about the method and its potential weaknesses from the paper in its current form. Many important details are deferred to appendices and not referenced in the main paper, and the descriptions of the method that remain in the main paper are often oblique. Section 4.2 is particularly impenetrable, and I’ve included a few examples here to illustrate.

    • Example: “Robustly normalized response magnitudes propagate through fixed dependency, pathway, and regulatory relations to form standardized features zv,t(x), v ∈ {A, R, S}, for measured and untested genes”. What does it mean for a response magnitude to propagate through fixed relations to form standardized features? Is there a vector over which some transition matrix defined by these relations is applied? What are these magnitudes tracking responses to? I have read this sentence several times and I still had no idea how exactly the features are constructed until I read to the end of the paper and saw there was an appendix with these details that section 4.2 failed to reference. I also had to ask an AI what A, R, and S are as it was not immediately clear from context that these were meant to be shorthand for datasets.
    • Then immediately after: “One intercept and three view weights minimize regularized binary crossentropy over measured genes.” The equation I had assumed this referred to has no regularization term.
    • In the same paragraph: “Here, feedback-calibrated denotes assay-adapted ranking.” The term ‘assay-adapted’ does not tell me anything about what type of information is received or what is done with that information to update the ranking beyond what I could have inferred from 'feedback-calibrated'. I searched through the pdf and it is only used in one other place, where it is not provided with a more technical definition that would make its use in this paragraph informative.
  • Table 1 is suspicious for multiple reason. The bolding strategy of ‘highlighted methods’ is confusing, as bolded numbers are generally assumed to indicate maxima and it’s not clear why the selected methods were highlighted. In part (b), the effect of HyPS is suspiciously uniform across LLM backbones and datasets. In almost all backbone-dataset combinations, it attains 1.55-1.58x improvement. It is extremely unusual for the relative effect of a method to be so consistent across such a wide variety of settings, and the paper does not comment on or try to explain why this might be the case.

  • The results show that this approach is better than searching directly over genes (whether with a parametric model or with an LLM), but don’t provide much insight into e.g. whether the low-level tactical gene selection mechanism is optimal. The discussion in Section 5.3 also suffered from difficult writing. The paper does not synthesize from the ablations any insights into what the mechanistic effects of the algorithm components were. There was also no discussion of the surprisingly positive effect of removing hypothesis adaptation from the method on N/E.

  • The red column in 4 (b) if anything suggests to me that the helpful inductive bias that is seeded in the portfolio at the start of the search by the portfolio is playing a large role in the method’s success, rather than having a more efficient search mechanism.

  • In figure 3: it looks like GPT-5.6 Sol performs significantly worse than Terra, despite being a larger same-generation model which, one would expect, would perform better at the same task.

Questions for authors

See weaknesses

10 comments
AuthorsOpenReview ↗

1. The method is now self-contained in the main text, with a new Algorithm 1

The paper is grammatically correct but written in a way that makes it extremely difficult to understand the technical details of the method.

We thank the reviewer for this candid feedback and for the concrete examples, which were extremely valuable in rewriting the method section. We restructured §3–§5 so that the method can be fully understood from the main text:

  • New Algorithm 1: One complete round of HyPS, from hypothesis update to assay feedback.
  • Explicit protocol: §3 now defines what the agent observes in each round.
  • Definitions with an example: §4.1 defines slot, unit, anchor, pool, and quota, illustrated by an IL2 hypothesis with its anchors.
  • Key equations in the main text: The quota priority (Eq. 2), feature propagation (Eq. 4), and full regularized objective (Eq. 6) moved from the appendix into §4.
  • Controls defined in the main text: §5.1 defines every within-HyPS control.
  • Appendix references: Every remaining appendix detail is referenced from the main text.

Revision in paper: Algorithm 1; §3; §4.1–4.3, Eq. (2)–(6); §5.1.

AuthorsOpenReview ↗

2. Feature construction, notation, and the scorer objective are now explicit in §4.3

What does it mean for a response magnitude to propagate through fixed relations to form standardized features? Is there a vector over which some transition matrix defined by these relations is applied?

We thank the reviewer for these precise questions. The reviewer's interpretation is exactly right: a relation matrix is applied to a vector of observed signals. We have rewritten this part of the method, and each example the reviewer raised is now addressed in the main text.

Propagation in two explicit steps (Eq. 4):

  1. Each tested gene gg receives a robust signal st(g)s_t(g): its absolute response minus the median of tested responses, divided by 1.4826×1.4826\timesMAD, clipped to [−3,3][-3,3].
  2. Each relation matrix κv\kappa_v propagates these signals to every gene, rv,t(x)=∑g tested, g≠xκv(x,g) st(g)r_{v,t}(x)=\sum_{g\ \text{tested},\,g\neq x}\kappa_v(x,g)\,s_t(g), followed by standardization to zv,t(x)z_{v,t}(x).

Hence zv,t(x)z_{v,t}(x) is high when genes related to xx under view vv have produced strong responses in the current assay. This is how evidence gathered anywhere in the search informs every candidate.

A, R, and S defined at first use ("I also had to ask an AI what A, R, and S are"): A = nonnegative cosine similarity of Achilles dependency profiles; R = Reactome pathway co-membership, down-weighted for large pathways; S = degree-normalized SIGNOR regulatory adjacency.

Full regularized objective (Eq. 6) ("The equation I had assumed this referred to has no regularization term"): The ridge penalty 12∥w−wt0∥Λ2\tfrac12\|\boldsymbol w-\boldsymbol w_t^0\|_\Lambda^2 with Λ=diag(1,3,3,3)\Lambda=\mathrm{diag}(1,3,3,3), together with the definition of the reference vector wt0\boldsymbol w_t^0, is now in the main text.

"Assay-adapted" replaced by a direct definition ("does not tell me anything about what type of information is received"): The text now states that the four scorer weights are re-estimated from the current assay's own outcomes after every batch, so the contribution of each relation view adapts to the assay. This is what "feedback-calibrated" means.

Revision in paper: §4.3, ¶"Relation-propagated response features" and ¶"Feedback-calibrated scorer", Eq. (4) and (6); App. A.3, Eq. (9).

AuthorsOpenReview ↗

3. Table 1 now follows standard bolding, and HyPS's gains are reproduced by an independent run group

Table 1 is suspicious for multiple reason.

We thank the reviewer for this careful examination of Table 1. We address both points.

Bolding now marks column maxima ("bolded numbers are generally assumed to indicate maxima"): Bold now marks the column maximum over all methods, including the traditional baselines, and the highlight markers are removed. Table 1 also now includes random selection, TypiClust, and ACS-FW, so every column compares HyPS with a broader set of baselines.

Relative gains vary across settings ("the effect of HyPS is suspiciously uniform across LLM backbones and datasets"): Each entry in Table 1 is the mean of ten independent five-round runs with batch size 128 (32 on Scharenberg). Across the 108 HyPS/BDA comparisons, the ratio ranges from 0.96 to 1.83. It is consistently lower on Steinhart (0.99–1.31 across all backbones), and it varies widely for MiMo-V2.5-Pro (0.99–1.83) and GLM-5.3 (0.96–1.60).

Confirmed by an independent run group: The label-blind experiment (new Table 20) is a completely separate set of runs. Its HyPS recall lies within 1.6 pp of the Table 1 values on every evaluated pair, and its gains over BDA range from 1.37× to 1.75× across the 18 comparisons. The size of the improvement is therefore reproduced by independent runs.

Per-run records: Per-run hit counts for every dataset, backbone, and seed will be released together with the code.

Revision in paper: Table 1 and caption; §5.2, ¶"Single-gene discovery" and ¶"Independent threshold"; Table 20 (App. C.10).

Reviewer_p8h2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Hypothesis Portfolio Search (HyPS) for genetic perturbation discovery under a fixed experimental budget. An LLM maintains hypotheses and anchor genes that define candidate pools, an adaptive allocator distributes tests across these pools, and a shared response-based scorer ranks candidates using observed outcomes and biological relations. The main contribution is the combination of revisable search directions, bounded allocation, and quantitative feedback scoring. Evaluation covers six single-gene screens with nine LLM backbones, a two-gene benchmark, and component and longer-horizon studies. The paper reports an increase in average single-gene recall from 8.40% to 12.47% over retrieval-enabled BioDiscoveryAgent and a 72.4% increase in mean recovered hit pairs.

Critical strengths
  1. The separation between candidate admission and selection is useful. Sections 3–4 clearly explain why improving a ranking model is insufficient when promising candidates never enter the search pool. Combining editable pools with a shared, four-parameter scorer is a practical way to connect LLM-generated proposals with experimental feedback. This is the most interesting aspect of the method.

  2. The comparison against BDA is broad and generally favorable. The main study covers six screens and nine backbones, with matched assay budgets and dependency-profile retrieval. Reporting both All and N/E recall is particularly useful because it exposes differences that would be hidden by overall recovery alone. The allocation and scoring ablations also provide evidence that the improvement involves more than better initial proposals.

Critical weaknesses
  1. The central comparison uses a threshold derived from the complete benchmark hit set. Appendix A.3 states that τ\tau is the minimum absolute response among all benchmark hits. This threshold affects both the allocation statistics and scorer-training labels. Although untested responses are excluded from feature propagation, the method still receives information derived from outcomes beyond its observed history. The authors acknowledge this, but its effect on the reported improvement is not measured. I therefore cannot tell how much of the advantage would remain with a threshold fixed independently of the target benchmark. The distinction between internal strong-effect labels and benchmark hits also needs empirical examination, particularly when response direction matters or the hit list cannot be reproduced by an absolute-score cutoff.

  2. The experiments do not sufficiently establish the contribution of hypothesis management beyond existing hybrid search methods. LLMNN (Gupta et al., 2025), discussed in Section 2, already combines LLM-proposed centers, neighbor retrieval, and experimental feedback, making it a particularly relevant missing comparison. Within HyPS, the evidence for adaptation is mixed: Figure 4/Table 8 show improved All recall but reduced N/E recall for six of seven backbones. Revision also changes anchors and resets allocation counts, so the existing control does not separate improved candidate admission from the renewed exploration bonus. These results support feedback scoring and allocation more clearly than the proposed hypothesis-management mechanism. A matched LLMNN comparison or a control that equips a simpler center-based method with the same scorer and allocator would help establish the additional contribution.

Questions for authors
  1. Can the authors report a representative comparison using a threshold specified independently of the complete benchmark hit set? How closely do the internal strong-effect labels agree with benchmark membership, and does this agreement differ across screens?

  2. How does HyPS compare with LLMNN under matched backbone, retrieval resources, and experimental budget? What evidence distinguishes the benefit of persistent hypotheses from that of the shared scorer and adaptive allocation?

11 comments
AuthorsOpenReview ↗

1. HyPS observes only tested outcomes: τ encodes the hit criterion, not hit identities

The central comparison uses a threshold derived from the complete benchmark hit set.

We thank the reviewer for this comment and for pointing out that our original wording was ambiguous. We have clarified the protocol and quantified exactly what τ\tau conveys.

A single scalar, no gene identities: τ\tau is one number fixed before search. It encodes the screen's hit criterion, as in threshold-exceedance formulations of active hit discovery such as Probability-of-Hit, and contains no information about which genes are hits.

Labels only after testing: In each round, the assay returns the responses of the selected batch only. A gene is labeled yτ(x)=1[∣f(x)∣≥τ]y_\tau(x)=\mathbf 1[|f(x)|\ge\tau] only after it has been tested. The response or hit status of an untested gene is never visible to the planner, allocator, or scorer.

τ\tau conveys the hit definition: We compared the rule ∣f∣≥τ|f|\ge\tau with benchmark membership on every screen:

ScreenHits (+/−)Genes with ∣f∣≥τ\lvert f\rvert\ge\tauPrecisionRecall
IFNG920 (431/489)9201.0001.000
IL2654 (484/170)7200.9081.000
Carnevale943 (452/491)9431.0001.000
Sanchez924 (0/924)18,4690.0501.000
Scharenberg53 (53/0)531.0001.000
Steinhart149 (149/0)1491.0001.000

On four of six screens, the label of a tested gene is identical to its hit status, so τ\tau conveys exactly the information an experimenter specifies before a screen.

Gains without any information from τ\tau: On Sanchez, τ\tau labels every gene positive and carries no information about hits. HyPS nevertheless outperforms BDA for all nine backbones in both metrics (e.g., Claude Sonnet 5: 4.6→7.1 All, 2.3→3.6 N/E).

Revision in paper: §3, ¶"Interactive protocol" and ¶"Hit criterion and internal feedback labels"; Algorithm 1, line 14; new Table 4 (App. A.3).

AuthorsOpenReview ↗

2. HyPS's gains persist with a label-blind threshold computed from tested genes only

I therefore cannot tell how much of the advantage would remain with a threshold fixed independently of the target benchmark.

We thank the reviewer for this valuable suggestion. We conducted a new experiment that measures this directly.

Label-blind HyPS: After each round, the threshold is recomputed from tested genes only. A gene is a strong-effect event if its robust z-score (median/MAD of observed response magnitudes, Eq. 9) is at least 2. The cutoff was fixed before any run, the agent never loads the hit list or τ\tau, and all other components are identical to Full HyPS.

Representative subset: We used the three screens and three backbones of our portfolio-size and thirty-round studies, fixed in advance. The subset includes IL2, whose asymmetric hit criterion makes it the most demanding case. Each pair uses ten five-round runs.

All / N/E recall (%); label-blind as mean ± SD:

ScreenBackboneBDAHyPS (original)HyPS (label-blind)
IFNGGPT-5.6 Sol8.1 / 4.012.5 / 6.313.17±1.69 / 6.37±0.59
IFNGGPT-5.6 Terra8.7 / 4.313.6 / 6.812.90±1.26 / 6.12±0.96
IFNGClaude Sonnet 59.1 / 4.114.0 / 6.412.43±1.79 / 5.94±0.58
IL2GPT-5.6 Sol18.1 / 9.828.1 / 15.327.13±1.17 / 13.87±0.79
IL2GPT-5.6 Terra17.4 / 9.427.1 / 14.626.02±1.94 / 13.62±1.14
IL2Claude Sonnet 517.8 / 9.627.6 / 14.927.25±1.18 / 14.07±0.80
CarnevaleGPT-5.6 Sol4.2 / 2.26.5 / 3.56.54±0.74 / 3.24±0.41
CarnevaleGPT-5.6 Terra4.4 / 2.16.7 / 3.36.69±0.43 / 2.91±0.42
CarnevaleClaude Sonnet 54.0 / 1.96.2 / 3.06.30±0.49 / 3.32±0.30
Mean10.20 / 5.2715.81 / 8.2315.38 / 7.72

Results:

  • Label-blind HyPS outperforms BDA in all 9 pairs on both metrics and retains 92% of the All gain and 83% of the N/E gain (All retention: 89% IFNG, 92% IL2, 102% Carnevale).
  • In four of eighteen comparisons, label-blind HyPS even slightly exceeds the original protocol.
  • The modest difference on IL2 N/E is expected, since the original threshold partly encodes IL2's asymmetric hit criterion.

On the evaluated subset, these results show that HyPS's improvement comes from its decision structure and does not depend on benchmark-derived information.

Revision in paper: §5.2, ¶"Independent threshold"; new Table 20 and the full rule (App. C.10); §6.

AuthorsOpenReview ↗

3. HyPS performs well whether or not internal labels match benchmark hits

The distinction between internal strong-effect labels and benchmark hits also needs empirical examination, particularly when response direction matters or the hit list cannot be reproduced by an absolute-score cutoff.

We thank the reviewer for this insightful comment. New Table 4 provides exactly this examination, and both cases the reviewer anticipates appear in our benchmarks.

When response direction matters (IL2): Hits are defined by asymmetric cutoffs (above +0.341 or below −0.324). The absolute rule adds 66 positive-score non-hits (precision 0.908), and HyPS still improves over BDA for all nine backbones on IL2.

When no absolute cutoff reproduces the hit list (Sanchez): All scores are negative and hits lie in both tails, so the rule labels every gene positive. HyPS still outperforms BDA for all nine backbones on Sanchez.

When no benchmark cutoff is used at all: The label-blind variant (Comment 2) needs no cutoff from the hit list and retains most of the gain, including on IL2.

Together, these results show that HyPS is robust to how its internal labels relate to benchmark membership.

Revision in paper: Table 4 and accompanying text (App. A.3); §3, ¶"Hit criterion and internal feedback labels"; Table 20 (App. C.10).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Summary

We sincerely thank all reviewers for their valuable feedback. It has substantially improved our approach and strengthened the rigor of our analysis and evaluation.

Improvements in the revised submission

In response to reviewer feedback:

  • New results: New tables for the label-blind threshold, the added baselines, and label–hit agreement; Table 1 and Figure 2 updated with the new baselines.
  • New baselines: Random selection, TypiClust, and ACS-FW, following Huang et al. (2024), under the same budgets.
  • New experiments: A label-blind threshold computed from tested genes only, and a 2×2 experiment that separates hypothesis revision from count resetting.
  • New analyses: Round-wise decomposition of initialization versus search, backbone sensitivity, and the consistency of relative gains.
  • Presentation: Algorithm 1, definitions of every component with an example, and the key equations moved from the appendix into the main text.

Task

  • Active hit discovery under a fixed budget: An agent selects batches of genetic perturbations over several rounds, observes the outcomes of tested genes only, and aims to recover as many assay-specific hits as possible. We evaluate six single-gene screens with nine LLM backbones and a two-gene benchmark with over 100,000 candidate pairs.

Contributions in our work

Problem: two bottlenecks that existing methods do not address together

  • A valuable gene must first enter the search scope, and then receive one of the few available tests.
  • Global-ranking methods (Bayesian optimization, active learning) prioritize candidates but do not manage which biological directions are explored.
  • Prompt-based LLM agents BDA propose biologically plausible directions, but hand the search space, the budget, and the use of feedback to a single free-form reasoning process, which struggles to turn numerical feedback into quantitative decisions.

Approach: HyPS decomposes exploration into three decisions, each with its own feedback mechanism

  • Search-space management: Revisable hypotheses with anchor genes define which genes can be tested, and are kept, revised, or replaced as evidence accumulates.
  • Budget allocation: Bounded adaptive quotas decide how many tests each direction receives, with a floor that preserves exploration and a cap that prevents over-concentration.
  • Candidate prioritization: A lightweight scorer, re-fitted to the assay's own outcomes every round, propagates measured responses through biological relations to rank untested genes.
  • Design principle: The LLM does what it is good at, proposing and revising mechanistic hypotheses, while quantitative feedback is handled by explicit statistical components. All observations train one shared scorer, so evidence transfers across directions, while each direction keeps its own budget.

HyPS thus transforms LLM-driven exploration from prompt-mediated decision-making into an explicit, feedback-driven decision process.

Evaluation

  • Consistent gains at matched resources: With the same backbone, retrieval, and budget as BDA, HyPS raises average recall from 8.40% to 12.47% (All, +48% relative) and from 5.57% to 8.15% (non-essential hits, +46% relative), improving 53 of 54 dataset–backbone comparisons.
  • Two-gene perturbations: HyPS increases recovered hit pairs by 72.4% across three backbones.
  • Stronger than non-LLM baselines: HyPS's average recall is 2.3× that of random selection and exceeds all seven non-LLM baselines, including TypiClust and ACS-FW.
  • Robust without benchmark information: With a threshold computed from tested genes only, HyPS retains 92% / 83% of its gain and outperforms BDA in all 9 evaluated pairs.
  • Every component matters: Removing the scorer or the adaptive quotas lowers recall for all 7 backbones; hypothesis revision alone accounts for 82% / 95% of the adaptation gain.
  • Gains come from search, not initialization: The advantage over BDA grows as feedback accumulates (e.g., from 0.5 to 9.8 pp on IL2 over five rounds); initialization explains only 5–18% of the final gap.
  • Less dependent on the backbone: On the two-gene task, the spread across backbones shrinks from 2.6× under BDA to 1.7× under HyPS.

Commitments for the final version

  • Adopt the label-blind threshold as the default and rerun all comparisons on all datasets.
  • Release the implementation, run configurations, and per-run records.
  • Include the 2×2 analysis, the distinction from LLMNN, and the contribution framing above.

Implications

LLM agents for scientific discovery benefit substantially from explicit control over where to search, how much to test, and which candidates to prioritize. Letting the LLM reason about mechanisms while feedback-driven components manage the budget offers a practical path to more reliable closed-loop experiment design.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.