acceptodds
Under review as a conference paper at ICLR 2027

Beyond Fixed Acquisition Functions: Temporal Difference Active Learning for Perturbation Discovery

23 views

Abstract

Genetic and drug perturbation screens are important tools for elucidating disease mechanisms and validating therapeutic targets, but experimental costs limit the number of candidates that can be tested. Existing perturbation screening methods typically update the surrogate model with feedback while keeping the acquisition function fixed, limiting their ability to adapt selection to different response landscapes. Experimental feedback reveals current hits and informs subsequent selection by updating candidate predictions, providing a training signal for learning how to select experiments. We formulate budgeted perturbation discovery as a sequential decision problem and propose Temporal Difference Active Learning (TDAL), which learns an acquisition function online through temporal difference learning from a single screening trajectory, accounting for current hits and future discoveries. TDAL uses the Hellinger distance between hit probabilities for unmeasured candidates before and after an experiment to weight successor value, and reuses measured outcomes through leave-batch-out (LBO) to construct virtual transitions and alleviate training data scarcity. Across eight public screens spanning single, double, and triple perturbations under varied experimental configurations, TDAL achieves an average rank of 1.63 among eleven methods and ranks first on four tasks, demonstrating its adaptability across response landscapes.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
active learning, experimental design, dynamic acquisition function, perturbation discovery, AI for science

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8G8dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies active learning in perturbation design with limited budgets. Specifically, this paper proposes a dynamic acquisition function TDAL that incorporates future actions to go beyond standard myopic acquisitions using temporal difference learning. Moreover, the authors propose to use leave-batch-out reconstruction to create augmented trajectories to learn TDAL from a single online perturbation screening trajectory. The proposed TDAL is evaluated on several single and multiple genetic perturbation screening datasets and it outperforms other classical baseline approaches, whose performance are often dataset-specific.

Critical strengths

The paper studies an important question within a growing field in drug discovery and the paper is well motivated by the fact that different perturbation datasets favour different acquisition functions in active learning. The proposed approach is novel and the paper is well-written.

Critical weaknesses

Some of the design choices of the TDAL are not well-motivated. The experimental setup can be improved to further show the potential impact of the proposed approach in real-world applications. Please see my questions below.

Questions for authors
  1. The authors propose to use a discount factor w_j to down-weight the learned future values using Hellinger distance, which implies a utility with discounted number of hits where the discount factor is trajectory-depend. Why does this design choice make sense? The authors did ablation studies by setting w_j to 0 or 1 and both ablations can reduce the performance. Why does the model perform better with w_j = 0 (discount future hits to 0) than w_j = 1 (consider all hits equally)?

  2. How well the proposed method perform within a smaller budget (e.g., <500)? The method is tested with 2k-52k perturbations which are not considered as budgeted in practice. Moreover, the proposed method does not performance well in the beginning of the experimental design loops (Figure 3), which are more budgeted regimes, and it has either similar or worse performance than PoH across all datasets.

3 comments

Thank you for your questions about the role of discounting and how TDAL performs under a smaller screening budget.

Q1. What role does discounting play, and why does w=0w=0 outperform w=1w=1?

During training, TDAL evaluates a batch using the discounted return Rj+wjRj+1+wjwj+1Rj+2+⋯R_j+w_jR_{j+1}+w_jw_{j+1}R_{j+2}+\cdots. We observe the current batch hit count RjR_j directly. We must bootstrap the successor value from a single screening trajectory that cannot be reset, so its estimate can have substantial error. As Proposition 3 shows, a smaller weight ww introduces bias by discounting distant rewards, but it also limits how much value estimation error propagates into the current TD target. With w=1w=1, future hits count in full, and errors in successor value estimates can propagate along the trajectory and influence batch selection when observations are limited. Setting w=0w=0 removes the bootstrap error from the successor term. The acquisition network fθf_\theta still learns from the difference between observed and predicted hits in the current batch, Rt−R^t(At)R_t-\widehat R_t(A_t). Updating the selection rule after each batch preserves the central idea of a dynamic acquisition function. This can explain why w=0w=0 outperforms w=1w=1 in Table 2 of the revised manuscript [1][2].

Hellinger weighting and fixed discount factors. The best fixed weight varies across response landscapes and is difficult to determine before screening. We tested 11 fixed weights (revised Table 3). The best values were 0.1 on IFN-γ, 0.2 on IL-2, and 0.5 on NK. TDAL sets the current batch weight wjw_j online using the Hellinger distance between predicted hit probabilities for the unmeasured candidates before and after feedback [3]. When feedback barely changes the predictions, wjw_j is close to zero, and training relies mainly on the observed batch hit count RjR_j. A larger change may indicate that the batch revealed information unavailable from earlier observations. A larger wjw_j assigns more successor value to that batch, allowing its contribution to subsequent exploration to enter the training target. On all three datasets, adaptive TDAL achieved a higher mean cumulative hit count than the best fixed weight selected in hindsight. Values are means ± sample SD across five seeds.

wwIFN-γIL-2NK
0.0260.0±11.7361.4±3.4101.4±14.0
0.1268.2±10.8357.0±11.4104.0±9.1
0.2265.0±22.0362.0±4.2103.6±7.0
0.3249.2±13.8360.2±8.6102.8±7.0
0.4259.6±7.8358.2±3.298.4±7.7
0.5257.2±9.7352.6±5.6106.0±5.9
0.6235.6±18.1347.4±8.8103.4±9.9
0.7215.6±35.2348.2±8.7105.0±11.8
0.8200.8±46.0330.8±34.7104.8±8.3
0.9211.6±24.4327.6±24.596.8±4.0
1.0191.8±41.1319.6±16.894.0±6.8
Adaptive TDAL270.6±6.1367.4±3.0109.4±9.3

Q2. How large are the screening budgets relative to the candidate pools?

Across the eight datasets, the total budget covers 11.68%–29.14% of each candidate pool (revised Table C3). The numbers of possible gene pairs and triples grow quadratically and cubically, respectively, with the number of genes, so combinatorial screens have larger absolute budgets. Our budget of 2,048 measurements per single-gene dataset is comparable to GeneDisco's 2,560 measurements (64 per batch) [4] and PoH's 2,000 active queries per real dataset [5].

A dedicated evaluation with a budget of 480 measurements. In Figure 3, the IFN-γ and IL-2 screens run for 60 rounds. The model state explicitly includes the fraction of the budget remaining and the number of rounds remaining (Appendix A.6, Eq. 19). With many rounds left, the policy can query candidates with moderate immediate hit probabilities if their outcomes could improve later predictions. A shorter screen calls for earlier use of the predictions already available. To evaluate discovery when the budget is small from the outset, we set the total budget to 480 measurements on both datasets (revised Table 1). Both methods started with the same 128 observations and allocated the remaining 352 queries across 11, 22, or 44 rounds. Small-budget TDAL achieved a higher mean cumulative hit count than PoH in all six settings. Values are means ± sample SD across ten paired seeds.

RoundsQueries per roundIFN-γ TDALIFN-γ PoHIL-2 TDALIL-2 PoH
113274.3 ± 21.360.4 ± 6.3117.1 ± 32.199.0 ± 7.6
221666.9 ± 21.961.6 ± 5.6114.9 ± 32.398.2 ± 3.4
44875.2 ± 22.360.3 ± 2.8117.0 ± 32.9100.7 ± 3.1

References

[1] Jiang et al. The Dependence of Effective Planning Horizon on Model Accuracy. AAMAS, 2015.

[2] Amit et al. Discount Factor as a Regularizer in Reinforcement Learning. ICML, 2020.

[3] White. Unifying Task Specification in Reinforcement Learning. ICML, 2017.

[4] Mehrjou et al. GeneDisco. ICLR, 2022, §5.

[5] Rubbi et al. Many Needles in a Haystack. ICML, 2026, §5.2.

Dear Reviewer,

Thank you again for your thoughtful review. Your comments helped us improve the paper and guided the additional experiments we reported in our response.

As the discussion period closes on November 18 (AoE), we would appreciate it if you could take a look at our response and let us know whether it addresses your concerns. We would be happy to clarify any remaining points.

Reviewer_diQoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TDAL, an online acquisition-learning framework for perturbation discovery. It combines a predicted hit-probability anchor with a learned correction, trains acquisition values through temporal-difference updates, and uses prediction-change-dependent successor weighting and leave-batch-out transition reconstruction. Experiments evaluate cumulative hit discovery across eight retrospective screening tasks.

Critical strengths
  1. The framework integrates immediate hit prediction, successor-value learning, and reuse of measured outcomes in a coherent design.

  2. The evaluation covers multiple perturbation datasets and includes ablations of the main components.

Critical weaknesses
  1. Missing comparison with GLARE. Although GLARE focuses on molecular screening, it already learns which candidates to select using screening feedback. Comparing against an adapted GLARE would clarify whether TDAL’s additional mechanisms offer an advantage.

  2. The benefit of learning future rewards is unclear. Removing the future-value term preserves much of TDAL’s performance. Paired comparisons and confidence intervals are needed to show whether this central component consistently improves discovery.

  3. Larger prediction changes do not necessarily mean more useful learning. TDAL gives future value more weight when predictions change substantially after an update. However, these changes could reflect noise rather than improvement. The paper should validate this assumption and compare against tuned fixed weights.

  4. It is unclear how the design specifically addresses perturbation discovery. The same framework could be applied to other screening tasks. The paper should explain which challenges of perturbation discovery motivate its components and show that those components address them.

Questions for authors

see weakness

4 comments

Thank you for suggesting the GLARE comparison and for your questions about successor value and the design of TDAL for gene perturbation screens.

W1. Comparison with adapted GLARE

GLARE was originally developed for molecular virtual screening using molecular fingerprints and binary activity labels. To compare the two methods directly on gene perturbation screens, we replaced the molecular fingerprints in the authors' implementation with TDAL's gene perturbation embeddings, adjusted the input layer dimensions, and converted continuous phenotypes to binary labels using each dataset's hit threshold. TDAL achieved a higher mean final hit count than adapted GLARE on all four datasets (mean ± sample SD across five seeds).

Two design differences are relevant. First, TDAL trains its surrogate on continuous phenotypes and then estimates the probability of crossing the hit threshold. This preserves effect-size information that would be lost by binarizing phenotypes at the outset. Second, TDAL reuses measured batches through leave-batch-out reconstruction (LBO) to construct virtual transitions. GLARE updates its policy along the single screening trajectory it executes.

DatasetTDALAdapted GLARE
IFN-γ270.6 ± 6.1245.2 ± 7.2
IL-2367.4 ± 3.0321.2 ± 5.8
Carnevale161.2 ± 8.6118.4 ± 28.7
SCHEMATIC160.4 ± 14.2122.4 ± 6.9

W2. What does successor value contribute beyond w=0w=0?

At w=0w=0, TDAL still updates its acquisition network fθf_\theta using training examples constructed through LBO. The network learns the difference between observed and predicted hits in the current batch, Rt−R^t(At)R_t-\widehat R_t(A_t). Using each batch’s feedback to update the acquisition rule preserves the central idea of a dynamic acquisition function. The surrogate can persistently underestimate hits in underexplored regions (Figure 5b). Compared with the surrogate-only baseline, learning this residual correction online increased the mean hit count from 186.4 to 260.0 on IFN-γ and from 253.8 to 361.4 on IL-2.

Adding Hellinger-weighted successor value to the immediate residual further increased the mean hit count on both datasets. Giving successor value its full weight (w=1w=1) reduced mean hit counts, which may reflect bootstrap error when successor value is estimated from a single trajectory. The table reports the mean difference in final hit counts (full TDAL minus w=0w=0) across five seeds.

DatasetMean difference in hits ± estimated SD95% sensitivity interval
IFN-γ+10.6 ± 6.2[2.9, 18.3]
IL-2+6.0 ± 1.2[4.5, 7.5]

W3. How do high-weight batches guide subsequent exploration, and how does adaptive weighting compare with a tuned fixed weight?

Changes in predictions can arise from refitting noise or reflect new information. To test whether high-weight batches were associated with improvements in subsequent rankings, we examined rounds 1, 10, and 40 for each of five seeds on both IFN-γ and IL-2, giving 15 screening states per dataset. At each state, we sampled 16 alternative batches of 32 genes uniformly at random from the then-unmeasured pool without consulting their phenotypes. Starting from the same surrogate model, we updated it separately with each batch's measured outcomes, reranked a common set of untested genes by predicted hit probability, and used their measured phenotypes to check which genes newly entering the top 32 were hits (revised Section 5.6, Table 4). On average, the four highest-weight batches brought more genes into the top 32 than the four lowest-weight batches. The number of true hits among these new entries increased by 50.3% on IFN-γ and 10.5% on IL-2 on average (mean ± sample SD across five seeds).

DatasetDifference in new top-32 entries (high − low)Hits among new entries (low weight)Hits among new entries (high weight)Relative increase in hits among new entries
IFN-γ6.86 ± 4.361.57 ± 0.532.55 ± 1.59+50.3% ± 64.1%
IL-24.56 ± 5.383.22 ± 0.673.37 ± 1.22+10.5% ± 58.8%

We also compared 11 fixed weights (Table 3) and used leave-one-seed-out tuning, selecting the weight on four seeds and evaluating it on the held-out fifth seed. This procedure yielded mean cumulative hit counts of 262.2 ± 21.5 on IFN-γ, 360.2 ± 4.5 on IL-2, and 99.4 ± 9.6 on NK. On all three datasets, adaptive TDAL achieved a higher mean cumulative hit count than both the fixed weight chosen by leave-one-seed-out tuning and the best fixed weight chosen in hindsight. Values are means ± sample SD across five seeds.

wwIFN-γIL-2NK
0.0260.0±11.7361.4±3.4101.4±14.0
0.1268.2±10.8357.0±11.4104.0±9.1
0.2265.0±22.0362.0±4.2103.6±7.0
0.3249.2±13.8360.2±8.6102.8±7.0
0.4259.6±7.8358.2±3.298.4±7.7
0.5257.2±9.7352.6±5.6106.0±5.9
0.6235.6±18.1347.4±8.8103.4±9.9
0.7215.6±35.2348.2±8.7105.0±11.8
0.8200.8±46.0330.8±34.7104.8±8.3
0.9211.6±24.4327.6±24.596.8±4.0
1.0191.8±41.1319.6±16.894.0±6.8
Leave-one-seed-out262.2±21.5360.2±4.599.4±9.6
Adaptive TDAL270.6±6.1367.4±3.0109.4±9.3

W4. Which features of gene perturbation screens shaped the design of TDAL?

We see it as a strength that TDAL’s temporal difference learning and LBO can extend to other batch screening problems with a limited budget. Its design directly addresses three features of gene perturbation screens.

Local hit clustering varies across phenotypic readouts. In representations such as DepMap CERES profiles and STRING network embeddings, the hit rate among the ten nearest neighbors of a hit ranged from 0.83 times the overall hit rate on NK to 6.27 times that rate on Simpson K562 (Figure 4a). The performance of fixed acquisition rules also varied across datasets. PoH ranked first on Jurkat and fourth on NK and Carnevale. CoreSet ranked first on NK and eighth on IL-2 and Simpson K562. TDAL learns a correction to its acquisition scores online from each batch's feedback, adapting to different hit distributions. It ranked among the top three methods on all eight datasets.

Screens measure continuous phenotypes, and follow-up validation targets candidates whose effects cross a threshold. Gene perturbation experiments measure continuous phenotypes such as log fold changes in cytokine levels or cell growth rates. The screening objective is to find candidates whose effects exceed a threshold τ\tau. TDAL fits a regression surrogate to the continuous phenotypes, predicts each candidate's probability of exceeding the threshold, and trains its acquisition network using batch hit counts and Hellinger-weighted successor value. This allows it to prioritize true hits that the surrogate underestimates. All five TDAL runs found the 85 IL-2 hits in Figure 5b, and all five runs that ranked candidates solely by surrogate hit probability missed them. When TDAL selected these genes, their mean rank under the concurrent surrogate predictions was 9,780.

Combinatorial perturbations require a representation that is invariant to gene order. Double- and triple-gene CRISPR screens measure the phenotype of a joint perturbation, which can include non-additive genetic interactions. For an unordered set of m∈{2,3}m\in\{2,3\} genes, TDAL concatenates the mean embedding of the member genes, the mean of the pairwise elementwise absolute differences, and the mean of the pairwise elementwise products, averaging over all unordered gene pairs for the latter two terms (Eq. 13). The representation retains features of both the member genes and their pairs and is invariant to the order in which the genes are supplied.

Dear Reviewer,

Thank you again for your thoughtful review. Your comments helped us improve the paper and guided the additional experiments we reported in our response.

As the discussion period closes on November 18 (AoE), we would appreciate it if you could take a look at our response and let us know whether it addresses your concerns. We would be happy to clarify any remaining points.

Reviewer_CYciOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the problem of discovering perturbations with desired phenotypic effects when the experimental budget is constrained, using temporal difference active learning.   The core idea is to train a gene perturbation model as a surrogate and an acquisition neural network to optimize the selection of perturbations.  The surrogate model is trained using the experimental data and the observed phenotypes to predict the phenotype of each candidate perturbation, along with its uncertainty. The model is an MLP with MC to quantify uncertainty. The acquisition network is a double DQN that learns scores to update the predicted probabilities using temporal-difference learning.  This means, similar to reinforcement learning, the network is trained to receive the history and the surrogate model, and instead of learning to optimize for the next selection, to learn the action (perturbation) that  maximizes the weighted cumulative number of discoveries from that point and until the end of the horizon. The weight for each step is based on the difference between predicted probabilities at that step before the surrogate model is trained on the chosen experiments and after. These changes are measured using Hellinger distance. The intuition is that changes in predictions for the remaining candidates are a proxy for the informational value of that change, and hence they are used to scale the future reward.  They also use leave-batch-out to generate additional training transitions for the acquisition network by hiding batches of already measured outcomes, refitting the surrogate on the remaining data, and restoring the hidden outcomes to reconstruct before-and-after states without requiring additional experiments, akin to an adaptation of memory replay. The experiments include 60 or 200 sequential selection rounds and are repeated for 5 random seeds.

Critical strengths

The evaluation covers diverse perturbation settings with phenotypic outcomes and candidate pools with thousands of interventions in total.

The benchmarks are extensive in terms of methods and datasets.

The experiments use five paired random seeds and include component ablations that distinguish between updating the surrogate, learning an immediate-hit correction, incorporating future value, and constructing additional training transitions.

Critical weaknesses

TDAL’s effectiveness with only a few acquisition rounds is insufficiently established. Huang et al. (2024) emphasize that practical constraints severely limit the number of experimental rounds in specific single-cell experiments. If a cycle takes months in the wet lab, developing a model that works for 60 or 200 cycles might have too small a scope in real life.

The evaluation assumes a hit threshold tautau derived, to my understanding, from the phenotype values of the complete candidate pool. However, by definition, the phenotypes of all the candidates are not available at the beginning of the experiment.

The theoretical contribution appears limited. To my understanding Proposition 1 adapts Bellman optimality to the weighted objective, while Proposition 2 assumes true conditional probabilities rather than the learned MC dropouts used in practice. Proposition 3 bounds performance through value-estimation error and objective-weighting bias without establishing that these quantities are surely small.

Questions for authors

Appendix G.1 shows a substantial performance drop on IL-2 with 15 rounds, but this setting also removes the initial LBO. Can the authors disentangle these effects and evaluate TDAL with substantially fewer acquisition rounds, for example, 5–10, under matched budgets?

In Appendix C.1 tautau is chosen from the complete candidate pool. We can assume that this will not be available in a real screening. Do the reported improvements persist when the selection threshold is estimated only from the initial observations?

Can Proposition 2 be extended to the MC-dropout probability estimates used in TDAL?

How do surrogate model's error or the MC sampling noise affect the weight computed from changes in predicted hit probabilities before and after an experiment?

What indicates that the weighting bias in Proposition 3 remains small?

3 comments

Thank you for your careful review of our experimental design and theoretical analysis.

W1/Q1. Assay turnaround time and performance over 5–15 screening rounds

Huang et al. studied Perturb-seq [1]. For the scalar IFN-γ and IL-2 readouts we use, detection began about a week after the library was introduced, well short of several months [2]. Following the approach to retrospective batch evaluation used by GeneDisco (up to 40 rounds) [3] and PoH [4], we used 60 rounds on five datasets and 200 rounds on three larger combinatorial perturbation datasets to evaluate how well the method continues to use feedback over time. On the latter three datasets, the budgets covered 18.57%–29.14% of the respective candidate pools.

For screens that allow only a few rounds of feedback, we examined why TDAL's performance decreased in the 15-round IL-2 setting in Appendix G.1. The original setting used an initial set of n0=128n_0=128 and a batch size of B=128B=128. Holding out a full batch left no observations with which to fit the pre-feedback surrogate, so the algorithm skipped initial leave-batch-out reconstruction (LBO) (Algorithm 1, line 8). To compare TDAL with and without initial LBO at the same total budget of 2,048 measurements, we increased n0n_0 to 608 and ran screens of 5, 10, and 15 rounds (Appendix Table G3).

DatasetRoundsTDALPoHTDAL without initial LBO
IFN-γ5231.2±30.4221.4±24.4152.6±14.9
IFN-γ10239.8±22.2230.2±25.2176.8±17.7
IFN-γ15219.4±11.6213.8±2.3143.0±6.6
IL-25300.4±26.2299.0±23.6234.0±54.7
IL-210318.6±18.0306.2±15.7210.2±54.8
IL-215303.8±22.2304.4±11.8194.2±20.2

Values are means ± sample SD across five paired seeds. In the cases shown in Table G3, enabling initial LBO increased the mean hit count by 63.0–109.6 across the six settings. TDAL had a higher mean hit count than PoH in five of these settings.

W2/Q2. Estimating the hit threshold τ\tau from initial observations alone

In a prospective screen, experimenters can specify a biologically meaningful fold-change threshold relative to a control or estimate the threshold from pilot measurements. In our main retrospective evaluation, we used the 95th percentile of phenotype scores in the full pool to define a reference hit set comprising approximately 5% of candidates, consistent with the rank-based targets used in GeneDisco and PoH [3][4]. To evaluate performance when the full-pool quantile is unavailable before screening, we estimated τ\tau as the 95th percentile of the absolute phenotype scores in the first 128 observations. We evaluated all methods against the same full-pool reference hit set (Appendix Table C2). On IFN-γ and IL-2, TDAL retained 97.2% and 99.1%, respectively, of its mean hit count in matched runs using the full-pool threshold. TDAL achieved the highest mean hit count on both datasets. Values are means ± sample SD across five paired seeds.

DatasetTDALPoHRandom
IFN-γ261.2 ± 11.5258.6 ± 18.194.0 ± 6.0
IL-2359.4 ± 4.4350.6 ± 9.6102.2 ± 7.0

W3/Q3. Extending Proposition 2 to MC dropout probability estimates

Proposition 2 extends to approximate probabilities when we account for post-feedback estimation error (Appendix B.2). Let zz be the binary hit label and qq the true conditional hit probability after feedback. Let p^,q^\hat p,\hat q be the fractions of K=50K=50 MC dropout predictions that meet or exceed the threshold before and after feedback, respectively. Fix the screening history and current batch, and let ⟨⋅⟩\langle\cdot\rangle denote an average over the remaining candidates, feedback outcomes, and model randomness. Define the mean squared change in predicted probability DD, the reduction in squared loss Δ^\widehat\Delta, and the post-feedback mean squared estimation error ee by

D=⟨(q^−p^)2⟩,Δ^=⟨(p^−z)2−(q^−z)2⟩,e=⟨(q^−q)2⟩.D=\langle(\hat q-\hat p)^2\rangle,\qquad \widehat\Delta=\langle(\hat p-z)^2-(\hat q-z)^2\rangle,\qquad e=\langle(\hat q-q)^2\rangle.

Assume that the model uses only observed information and that, conditional on the feedback, dropout randomness is independent of the unmeasured labels. Taking the conditional expectation over zz gives

Δ^−D=2⟨(q^−p^)(q−q^)⟩.\widehat\Delta-D=2\langle(\hat q-\hat p)(q-\hat q)\rangle.

The Cauchy–Schwarz inequality then gives ∣Δ^−D∣≤2De|\widehat\Delta-D|\le 2\sqrt{De}. The Hellinger weight computed from the estimated probabilities still satisfies E[w^4]≤D≤2E[w^2]\mathbb E[\hat w^4]\le D\le 2\mathbb E[\hat w^2].

W3/Q4. Sensitivity of Hellinger weights to MC sampling and model refitting

Fix the remaining candidate pool and both fitted models, one before and one after feedback. Let w∗,w^w_*,\hat w be the Hellinger weights computed from the true conditional probabilities and from probabilities estimated with KK MC dropout draws, respectively. Let ε−,ε+\varepsilon_-,\varepsilon_+ be the pre- and post-feedback root mean squared Hellinger errors between the models' infinite-sample dropout hit probabilities and the true probabilities. For a model hit probability pˉ\bar p and a hit fraction p^\hat p estimated with KK independent masks, Eq. (47) gives HBer2(p^,pˉ)≤∣p^−pˉ∣H_{\mathrm{Ber}}^2(\hat p,\bar p)\le|\hat p-\bar p|. The binomial variance gives EMC∣p^−pˉ∣≤1/(2K)\mathbb E_{\mathrm{MC}}|\hat p-\bar p|\le1/(2\sqrt K). Averaging over candidates and applying Jensen's inequality, followed by the triangle inequality for the pre- and post-feedback models, yields

EMC∣w^−w∗∣≤ε−+ε++2 K−1/4.\mathbb E_{\mathrm{MC}}|\hat w-w_*|\le\varepsilon_-+\varepsilon_++\sqrt 2\,K^{-1/4}.

The sampling term approaches zero as KK increases. In the first IL-2 screening round, we used K=50K=50 draws on the same candidate pool. Resampling dropout masks on a fixed model and refitting on the same data produced mean weights of about one-fifth and one-third, respectively, of the mean weight obtained after refitting with new feedback (Appendix B.2, Table B1). Values are means ± sample SD across five seeds.

Resample onlyRefit on the same dataRefit with feedback
0.0269±0.00200.0443±0.00920.1364±0.0616

W3/Q5. The role of the weighting bias ΔC(w)\Delta_C(w) in Proposition 3

Proposition 3 characterizes the trade-off between the objective weighting bias ΔC(w)\Delta_C(w) and the successor value estimation error εtw\varepsilon_t^w without assuming that the bias is small for arbitrary weights. Smaller weights discount future rewards, increasing the upper bound on ΔC(w)\Delta_C(w) while limiting how much successor value estimation error propagates along the trajectory. When value estimates are noisy, a shorter effective planning horizon can therefore act as a regularizer and may improve the undiscounted cumulative return [5]. Revised Table 2 illustrates this trade-off. At w=1w=1, future hits count in full, the weighting bias is zero, and successor value estimation error enters the update in full. At w=0w=0, future hits are fully discounted. The current training target retains only the immediate residual, and removing the successor term also removes its bootstrap error. On all four ablation datasets, adaptive TDAL achieved a higher mean final hit count than either endpoint.

References

[1] Huang et al. Sequential Optimal Experimental Design of Perturbation Screens Guided by Multi-modal Priors. RECOMB, 2024.

[2] Schmidt et al. CRISPR activation and interference screens decode stimulation responses in primary human T cells. Science, 2022.

[3] Mehrjou et al. GeneDisco: A Benchmark for Experimental Design in Drug Discovery. ICLR, 2022.

[4] Rubbi et al. Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments. ICML, 2026.

[5] Amit et al. Discount Factor as a Regularizer in Reinforcement Learning. ICML, 2020.

Dear Reviewer,

Thank you again for your thoughtful review. Your comments helped us improve the paper and guided the additional experiments we reported in our response.

As the discussion period closes on November 18 (AoE), we would appreciate it if you could take a look at our response and let us know whether it addresses your concerns. We would be happy to clarify any remaining points.

Comments on the paper

1 comment

Dear AC, SAC and PC,

Thank you for overseeing the review of our paper. We have responded to all three reviews and revised the manuscript accordingly. We have not yet heard back from the reviewers and would be happy to clarify any remaining questions before the discussion closes on November 18 (AoE).

Two questions recurred across the reviews.

How does batch feedback improve subsequent selections? We compared adaptive weighting with 11 fixed weights. On each of the three datasets, adaptive TDAL achieved a higher mean cumulative hit count than the best fixed weight. On IFN-γ and IL-2, more true hits entered the top 32 after feedback from high-weight batches than from low-weight batches, on average (Tables 3 and 4).

How do the budget and number of rounds affect discovery? With a budget of 480 measurements, small-budget TDAL found more hits on average than PoH on both datasets under all three schedules (Table 1). With a budget of 2,048 measurements and only 5–15 feedback rounds, PoH had the higher mean hit count across the full set of paired seeds.

Thank you for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.