- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies active learning in perturbation design with limited budgets. Specifically, this paper proposes a dynamic acquisition function TDAL that incorporates future actions to go beyond standard myopic acquisitions using temporal difference learning. Moreover, the authors propose to use leave-batch-out reconstruction to create augmented trajectories to learn TDAL from a single online perturbation screening trajectory. The proposed TDAL is evaluated on several single and multiple genetic perturbation screening datasets and it outperforms other classical baseline approaches, whose performance are often dataset-specific.
Critical strengths
The paper studies an important question within a growing field in drug discovery and the paper is well motivated by the fact that different perturbation datasets favour different acquisition functions in active learning. The proposed approach is novel and the paper is well-written.
Critical weaknesses
Some of the design choices of the TDAL are not well-motivated. The experimental setup can be improved to further show the potential impact of the proposed approach in real-world applications. Please see my questions below.
Questions for authors
-
The authors propose to use a discount factor w_j to down-weight the learned future values using Hellinger distance, which implies a utility with discounted number of hits where the discount factor is trajectory-depend. Why does this design choice make sense? The authors did ablation studies by setting w_j to 0 or 1 and both ablations can reduce the performance. Why does the model perform better with w_j = 0 (discount future hits to 0) than w_j = 1 (consider all hits equally)?
-
How well the proposed method perform within a smaller budget (e.g., <500)? The method is tested with 2k-52k perturbations which are not considered as budgeted in practice. Moreover, the proposed method does not performance well in the beginning of the experimental design loops (Figure 3), which are more budgeted regimes, and it has either similar or worse performance than PoH across all datasets.
3 comments
Q2. How large are the screening budgets relative to the candidate pools?
Across the eight datasets, the total budget covers 11.68%–29.14% of each candidate pool (revised Table C3). The numbers of possible gene pairs and triples grow quadratically and cubically, respectively, with the number of genes, so combinatorial screens have larger absolute budgets. Our budget of 2,048 measurements per single-gene dataset is comparable to GeneDisco's 2,560 measurements (64 per batch) [4] and PoH's 2,000 active queries per real dataset [5].
A dedicated evaluation with a budget of 480 measurements. In Figure 3, the IFN-γ and IL-2 screens run for 60 rounds. The model state explicitly includes the fraction of the budget remaining and the number of rounds remaining (Appendix A.6, Eq. 19). With many rounds left, the policy can query candidates with moderate immediate hit probabilities if their outcomes could improve later predictions. A shorter screen calls for earlier use of the predictions already available. To evaluate discovery when the budget is small from the outset, we set the total budget to 480 measurements on both datasets (revised Table 1). Both methods started with the same 128 observations and allocated the remaining 352 queries across 11, 22, or 44 rounds. Small-budget TDAL achieved a higher mean cumulative hit count than PoH in all six settings. Values are means ± sample SD across ten paired seeds.
| Rounds | Queries per round | IFN-γ TDAL | IFN-γ PoH | IL-2 TDAL | IL-2 PoH |
|---|---|---|---|---|---|
| 11 | 32 | 74.3 ± 21.3 | 60.4 ± 6.3 | 117.1 ± 32.1 | 99.0 ± 7.6 |
| 22 | 16 | 66.9 ± 21.9 | 61.6 ± 5.6 | 114.9 ± 32.3 | 98.2 ± 3.4 |
| 44 | 8 | 75.2 ± 22.3 | 60.3 ± 2.8 | 117.0 ± 32.9 | 100.7 ± 3.1 |
References
[1] Jiang et al. The Dependence of Effective Planning Horizon on Model Accuracy. AAMAS, 2015.
[2] Amit et al. Discount Factor as a Regularizer in Reinforcement Learning. ICML, 2020.
[3] White. Unifying Task Specification in Reinforcement Learning. ICML, 2017.
[4] Mehrjou et al. GeneDisco. ICLR, 2022, §5.
[5] Rubbi et al. Many Needles in a Haystack. ICML, 2026, §5.2.
Dear Reviewer,
Thank you again for your thoughtful review. Your comments helped us improve the paper and guided the additional experiments we reported in our response.
As the discussion period closes on November 18 (AoE), we would appreciate it if you could take a look at our response and let us know whether it addresses your concerns. We would be happy to clarify any remaining points.
Thank you for your questions about the role of discounting and how TDAL performs under a smaller screening budget.
Q1. What role does discounting play, and why does w=0 outperform w=1?
During training, TDAL evaluates a batch using the discounted return Rj+wjRj+1+wjwj+1Rj+2+⋯. We observe the current batch hit count Rj directly. We must bootstrap the successor value from a single screening trajectory that cannot be reset, so its estimate can have substantial error. As Proposition 3 shows, a smaller weight w introduces bias by discounting distant rewards, but it also limits how much value estimation error propagates into the current TD target. With w=1, future hits count in full, and errors in successor value estimates can propagate along the trajectory and influence batch selection when observations are limited. Setting w=0 removes the bootstrap error from the successor term. The acquisition network fθ still learns from the difference between observed and predicted hits in the current batch, Rt−Rt(At). Updating the selection rule after each batch preserves the central idea of a dynamic acquisition function. This can explain why w=0 outperforms w=1 in Table 2 of the revised manuscript [1][2].
Hellinger weighting and fixed discount factors. The best fixed weight varies across response landscapes and is difficult to determine before screening. We tested 11 fixed weights (revised Table 3). The best values were 0.1 on IFN-γ, 0.2 on IL-2, and 0.5 on NK. TDAL sets the current batch weight wj online using the Hellinger distance between predicted hit probabilities for the unmeasured candidates before and after feedback [3]. When feedback barely changes the predictions, wj is close to zero, and training relies mainly on the observed batch hit count Rj. A larger change may indicate that the batch revealed information unavailable from earlier observations. A larger wj assigns more successor value to that batch, allowing its contribution to subsequent exploration to enter the training target. On all three datasets, adaptive TDAL achieved a higher mean cumulative hit count than the best fixed weight selected in hindsight. Values are means ± sample SD across five seeds.