acceptodds
Under review as a conference paper at ICLR 2027

Few-Shot Bioactivity Prediction with Meta-Learning under Assay Heterogeneity

Abstract

Accurate bioactivity prediction is a central challenge in early-stage drug discovery, as individual assays often contain too few measurements to train reliable models independently. Meta-learning offers a principled approach to this few-shot setting, but assay heterogeneity may limit its effectiveness. Here, we test this hypothesis and show that meta-learning performance degrades as meta-training tasks become more heterogeneous. To address this, we introduce MetaHeta, a meta-learning framework that accounts for assay heterogeneity by conditioning predictions on auxiliary data from related assays, with relatedness defined flexibly from available assay information. The architecture of MetaHeta combines linear attention over large auxiliary datasets with exact attention over scarce task-specific context, enabling efficient scaling to the former without compromising exact attention over the latter. We demonstrate the benefits of our approach on assays from ChEMBL and BindingDB, improving few-shot bioactivity prediction and downstream compound prioritization in retrospective Bayesian optimization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
meta-learning, bioactivity prediction, few-shot learning, assay heterogeneity, neural processes

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_4we1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Given low data regime for biological assay prediction problem, meta-learning could be beneficial by training models using data from multiple assays even though only limited labeled data are available for each assay. The authors argue that the assay heterogeneity could have a negative impact for the meta-learning of bioactivity prediction. They define assay similarity based on target-and description-based relatedness and demonstrate the blindly adding irrelevant assay could have a negative impact to the prediction performance. They formulate a model architecture called MetaHeta that integrates the features embeddings from the target assay and auxiliary tasks of the most relevant assays with attention mechanism. They benchmark on simulated and real-world data for and demonstrate promising results. A retrospective Bayesian optimization experiment also suggests the model can improve compound prioritization.

Critical strengths
  • The problem formulation is reasonable and practical to existing challenges in bioactivity prediction. Leveraging information from other assays can potentially improve bioactivity prediction in low-data regime, but not all assays are equally relevant, therefore explicitly modeling for assay heterogeneity should be helpful.
  • The experimental evaluation is comprehensive including (1) a controlled synthetic kinase experiment to investigate the effect of increasing task heterogeneity, and (2) evaluation on real-world FS-Mol assays for few-shot bioactivity prediction. The retrospective Bayesian optimization for compound prioritization strengthens the claim as an orthogonal validation.
Critical weaknesses
  • There is limited biological validation of the assay relatedness. It’s critical because the assay relatedness is used for auxiliary task suggestion which is central for this paper. The relatedness based on target identity is intuitive, but the relatedness based on text-based assay description requires further investigation. Similarity derived based on the PubMedBERT embedding can be a surrogate and a good starting point for the biological relevance, while it could be affected by how the assays are documented. More analysis would be helpful based on target identify, assay types, experimental conditions, etc.
Questions for authors
  • How much of the performance gain comes from including the related auxiliary assays, and how much from the specific MetaHeta architecture?
  • To what extent, the degradation in performance comes from increasing more auxiliary data that dilutes the target-specific data rather than the assay heterogeneity? Would it still hurt the performance to include 20 auxiliary tasks even they are all biologically relevant to the assay of interest?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 4we1 (1/2)

Thank you for taking the time to review our manuscript. We respond in two parts, first examining the biological information captured by description-based assay relatedness, and then discussing how auxiliary assays and the MetaHeta architecture contribute to predictive performance.

W1: On the biological validation of description-based assay relatedness

There is limited biological validation of the assay relatedness. [...] More analysis would be helpful based on target identify, assay types, experimental conditions, etc.

Thank you for highlighting this important point, which deepens our understanding of the biological features captured by description similarity. In the newly added Appendix H (please refer to the revised PDF), we analyse what biological and experimental information is captured by the similarity between PubMedBERT embeddings of assay descriptions. Specifically, we use ChEMBL metadata to compare how often FS-Mol tasks and their retrieved auxiliary assays share the same biological target, assay type, assay format, and other experimental characteristics. We compare the agreement rates across all retrieved pairs with those obtained by randomly assigning auxiliary assays to FS-Mol assays 1,000 times. In the table below, we report enrichment as the ratio of the retrieved match rate to the median match rate across the 1,000 random assignments. For clarity, here we omit the 5th and 95th percentiles, which are reported in Table 6.

Overall, assays retrieved by description similarity were about 173 times as likely to share a biological target as randomly paired assays (32.83% vs. 0.19%). Assay type and BioAssay Ontology (BAO) format, which classifies assays by their experimental system (e.g., cell-based or biochemical), were also frequent matches. Both showed more modest enrichment because they also matched often among random pairs. Tissue showed 65-fold enrichment, although tissue matches were rare even among retrieved pairs (1.29% vs. 0.02%). Finally, retrieved pairs were about 21 times as likely to share a ChEMBL source document (28.05% vs. 1.36%), which may reflect shared protocols, documentation conventions, or how assays were grouped for deposition.

Shared annotationRetrieved (%)Random (%)Enrichment
Target identity32.830.19172.90
Assay type91.5069.511.32
BAO format74.6731.412.38
Cell-type name9.510.5417.54
Organism54.6232.341.69
In vitro/in vivo10.321.646.30
Tissue1.290.0265.00
ChEMBL document ID28.051.3620.68
AuthorsOpenReview ↗

Response to Reviewer 4we1 (2/2)

Q1: On the contributions of auxiliary assays and MetaHeta architecture

How much of the performance gain comes from including the related auxiliary assays, and how much from the specific MetaHeta architecture?

In Section 5.4 (Figures 5 and 7), we vary the number of auxiliary assays for both MetaHeta and ICICL-TNP. For both methods, adding a single auxiliary assay substantially improves Ros2R^2_{\mathrm{os}} and reduces NLL relative to using none. Adding more assays yields little further improvement in Ros2R^2_{\mathrm{os}}, while NLL continues to decrease.

The gains from MetaHeta's architecture can be observed in Table 2, by comparing performance of our method directly to that of ICICL-TNP. Given exactly the same auxiliary data, MetaHeta achieves 1.20–1.87 times higher Ros2R^2_{\mathrm{os}} than ICICL-TNP across the four FS-Mol context sizes. Both methods can be viewed as architectural modifications to vanilla TNP that allow them to incorporate auxiliary assays while preserving assay identity. In our FS-Mol experiments, MetaHeta adds only 1,536 parameters to vanilla TNP, all in the input embedder for the one-hot task indicators, while ICICL-TNP adds about 90,000. Preserving assay identity is critical to avoid conflating measurements from auxiliary assays with those from the target assay, as illustrated by the poor performance of RF-DauxD_{\mathrm{aux}} in Table 2.

Q2: On whether auxiliary tasks could dilute the target-specific data

To what extent, the degradation in performance comes from increasing more auxiliary data that dilutes the target-specific data rather than the assay heterogeneity?

As discussed in our response to Q1, the ablation in Section 5.4 shows that including auxiliary assays improves predictive performance relative to using none, with no systematic degradation as the number of auxiliary assays increases to five.

Would it still hurt the performance to include 20 auxiliary tasks even they are all biologically relevant to the assay of interest?

We investigate this in the newly added Appendix I (please see the revised PDF). We retrain MetaHeta on the core EPHA meta-dataset DmetacoreD_{\mathrm{meta}}^{\mathrm{core}}, raising the maximum number of auxiliary assays from 5 to 20. We then evaluate the model’s performance as we vary the number of auxiliary datasets included in DauxD_{\mathrm{aux}}. As shown in Figure 8, performance generally improves as more auxiliary assays are added, with diminishing gains and no systematic degradation up to 20 assays.

Thank you again for your thoughtful feedback. We hope our responses address your questions, and we are happy to clarify any remaining points. If we have resolved your concerns, we would be grateful if you would consider raising your score.

Thanks for providing the additional experiment that demonstrates the effectiveness on assay retrieval based on their similarity. This proves my assessment that it is easiest to identify relevant assays if there are target involved in the description. The retrieval performance on assays without specific target involvement can be modest. This is understandable, and I encourage the authors to include this in the revised manuscript.

Reviewer_mfYiOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a method to leverage several assays to learn bioactivity prediction models. Using the meta-learning framework, the authors design MetaHeta, a methodology enabling the selection of relevant auxiliary assays, using different approaches to measure the similarity between two assays. MetaHeta performs competitively across the different evaluations of the author, although the use of noisy auxiliary assays in the episodic training is studied and shows negative returns.

Critical strengths

Well-motivated problem and sound methodological approach. The paper addresses an important problem in early-stage drug discovery, where few-shot bioactivity prediction remains a significant challenge. The hypothesis that assay heterogeneity limits the effectiveness of meta-learning approaches is clearly motivated. The proposed use of similarity measures to select relevant auxiliary tasks is intuitive and well justified, and the experimental ablations convincingly demonstrate the importance of selecting relevant auxiliary data rather than simply increasing its quantity.

Comprehensive experimental evaluation and reproducibility. The experimental evaluation is thorough, covering both controlled semi-synthetic benchmarks and real-world bioactivity prediction tasks, as well as downstream compound prioritization. The semi-synthetic setting is particularly interesting, as it allows the authors to isolate the impact of task heterogeneity and compare performance against Bayes-optimal predictors. The results are especially convincing in the most data-scarce regimes, where few-shot learning is most relevant. Furthermore, the ablation studies provide useful insights into the method's behavior and sensitivity to hyperparameters.

Critical weaknesses

Question on the heterogeneous results at scale. In the controlled study, the meta-training distribution progressively shifts from exclusively in-domain tasks (EPHA) to a mixture containing increasingly unrelated kinase tasks, while keeping the training data scale fixed (5k + 5k tasks). I wonder whether the observed degradation would persist at larger training scales. In particular, if increasing the amount of heterogeneous training data even more might eventually improve generalization, rather than harm it.

Question on overlapping data / leakage ChemBL contains several duplicated assays, and I just want to make sure an effort was made to ensure the absence of leakage between the auxiliary set and the query set.

Questions for authors

see weaknesses

1 comment
AuthorsOpenReview ↗

Response to Reviewer mfYi

We sincerely thank you for taking the time to review our manuscript. We address your two questions below.

Question on the heterogeneous results at scale

[...] In particular, if increasing the amount of heterogeneous training data even more might eventually improve generalization, rather than harm it.

Thank you for this thought-provoking suggestion. In the newly added Appendix K (please refer to the revised PDF), we extend the original analysis by keeping the 5,000 EPHA meta-training tasks fixed and increasing the number of added non-TK tasks from 5,000 to 15,000 and 20,000. We evaluate both the original model (MetaHeta-S) and its larger version (MetaHeta-L) on held-out EPHA tasks. We include MetaHeta-L because the capacity of the original model could limit the potential gains from additional non-TK meta-training tasks.

The results are shown in Figure 9 in Appendix K. We observe that the effect of adding more non-TK tasks does not follow a general pattern and varies with context size. However, at ∣DC∣=32|\mathcal{D}^C|=32, adding 20,000 non-TK tasks reverses the degradation observed with smaller augmentations, and this effect occurs also at ∣DC∣=16|\mathcal{D}^C|=16 for the larger model. These results suggest that increasing heterogeneous training data could eventually improve generalization, depending on context size and model capacity.

Question on overlapping data / leakage

ChemBL contains several duplicated assays, and I just want to make sure an effort was made to ensure the absence of leakage between the auxiliary set and the query set.

For our few-shot evaluation on ChEMBL assays, we used the published FS-Mol split and selected auxiliary assays exclusively from its training partition. FS-Mol constructs the split so that no test assay shares an annotated protein target with a training assay, reducing the risk of leakage through duplicated assays. To reduce the risk of overlap with the ChEMBL-derived FS-Mol training data in our retrospective Bayesian optimization experiments, we selected assays that BindingDB annotated as patent-derived and that carried no ChEMBL source annotation. In summary, we put effort into minimizing the risk of leakage between auxiliary and evaluation data.

Thank you again for your thoughtful comments. We would be happy to answer any follow-up questions. If you found our response helpful, we would be grateful if you would consider raising your rating of our submission.

Reviewer_8T7ROpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a meta-learning approach for few-shot bioactivity prediction, where they condition preditions on auxiliary data from related assays. To include these auxliary data, they propose to combine exact attention and linear attention for computational efficiency. They also perform experiments to show that task heterogenity degrades performance of meta-learning methods.

Critical strengths
  • The authors propose a computationally efficient architecture for handling auxiliary data
  • The authors include an experiment to prove one of their main claims, i.e.: heterogeneity of tasks result in degraded performance of meta-learnign methods.
  • Extensive evaluation and comparison with different types of baselines.
  • The autors evaluate this baselines and their method in a well-established benchmark dataset, FS-Mol.
  • The code used for this project is provided so the results might be reproducible.
Critical weaknesses
  • Currently, for ICICL-TNP, one of the strongest baselines and the one that serves as base to contruct the proposed method, the authors set a values of pseudo-tokens per auxiliary task of 64. Varying the number of auxiliary pseudo-tokens in ICICL-TNP would help establish whether the proposed improvements actually arise from removing the pseudo-token bottleneck or only because of a larger number of parameters.
  • The authors define two different ways in which assay relatedness can be regarded: either by target similarity or by textual assay description similarity. They motivate that assay-based relatedness should be used when targets are not known, or not sufficient information is availabe. However the authors provide no information about how both settings could influence the outcome to inform readers about which modality would be more useful in case both kinds of data are available.
Questions for authors
  1. In this work, ICICL-TNP is set to use 64 pseudo-tokens per auxiliary dataset, and the number of context pseudo-tokens equals the context size (Appendix A.4.2). How was the value of 64 chosen for this hyperparameter, was it tuned? Could you report ICICL-TNP on FS-Mol with more pseudo-tokens?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 8T7R (1/2)

Thank you for your thoughtful review of our submission. We respond in two parts, first discussing the number of auxiliary pseudo-tokens in ICICL-TNP, and then comparing target-based and description-based measures of assay relatedness.

W1 and Q1: Pseudo-token count in ICICL-TNP

Varying the number of auxiliary pseudo-tokens in ICICL-TNP would help establish whether the proposed improvements actually arise from removing the pseudo-token bottleneck or only because of a larger number of parameters.

As for the parameter count, we matched the model sizes closely and used the same parameter-sharing scheme for the context and auxiliary branches in both models. On FS-Mol, ICICL-TNP has 6,706,818 parameters, about 1.3% more than MetaHeta’s 6,618,242. As such, gains of our method cannot be attributed to a larger parameter count. Additional experiment with more pseudo-tokens is reported below.

How was the value of 64 chosen for this hyperparameter, was it tuned?

We did not tune this hyperparameter. We chose 64 pseudo-tokens (PTs) per auxiliary dataset to keep the compression ratio (the number of data points per PT) close to those used by Ashman et al. With 64 PTs, 95% of FS-Mol auxiliary datasets have a compression ratio of at most 3.14:1, comparable to the average ratios in the experiments of Ashman et al. We report the number of PTs and the average compression ratio for each of their experiments in the newly added Appendix J (please refer to the revised PDF).

Could you report ICICL-TNP on FS-Mol with more pseudo-tokens?

We document this additional result in Appendix J of the revised manuscript. Specifically, we reran ICICL-TNP on FS-Mol with 128 PTs per auxiliary dataset, keeping the number of context PTs equal to the context size. As shown in the table below, doubling the auxiliary PT count had mixed effects on ICICL-TNP performance. At context size 4, both metrics were slightly worse. At size 8, NLL was slightly higher and Ros2R^2_{\mathrm{os}} was nearly unchanged. Both metrics worsened at size 16 and improved at size 32. Increasing the auxiliary PT count from 64 to 128 did not close the performance gap on FS-Mol: MetaHeta achieved lower NLL and higher Ros2R^2_{\mathrm{os}} than both ICICL-TNP variants at all four context sizes.

Method4 · NLL ↓4 · Ros2R^2_{\mathrm{os}} ↑8 · NLL ↓8 · Ros2R^2_{\mathrm{os}} ↑16 · NLL ↓16 · Ros2R^2_{\mathrm{os}} ↑32 · NLL ↓32 · Ros2R^2_{\mathrm{os}} ↑
ICICL-TNP (64 PTs)2.291 ± 0.1120.069 ± 0.0381.786 ± 0.0220.117 ± 0.0201.485 ± 0.0070.155 ± 0.0201.362 ± 0.0440.131 ± 0.026
ICICL-TNP (128 PTs)2.312 ± 0.1280.061 ± 0.0211.800 ± 0.0190.118 ± 0.0361.546 ± 0.0060.135 ± 0.0341.339 ± 0.0170.155 ± 0.013
MetaHeta2.216 ± 0.0500.129 ± 0.0281.765 ± 0.0370.140 ± 0.0291.466 ± 0.0200.187 ± 0.0181.277 ± 0.0100.228 ± 0.011
AuthorsOpenReview ↗

Response to Reviewer 8T7R (2/2)

W2: Comparison of target-based and description-based assay relatedness

The authors define two different ways in which assay relatedness can be regarded: either by target similarity or by textual assay description similarity. [...] However the authors provide no information about how both settings could influence the outcome to inform readers about which modality would be more useful in case both kinds of data are available.

Thank you for raising this point. Knowing how to select auxiliary assays when both target identities and assay descriptions are available is important for practitioners. We have added a direct comparison of the two selection strategies in the Section 5.5 of the revised manuscript.

In particular, we evaluated the two strategies on 100 ChEMBL assays whose targets also occur in the FS-Mol training set. For description-based selection, we applied our standard procedure after excluding training assays that shared the evaluation assay’s biological target. For target-based selection, we randomly sampled training assays that shared that target. We matched the sizes of the two auxiliary sets using the smaller of the two available counts. Results are shown in the table below. Same-target selection yielded slightly higher Ros2R^2_{\mathrm{os}} at context sizes 4 and 8, but the gains were small relative to variation across seeds. At context sizes 16 and 32, the differences in Ros2R^2_{\mathrm{os}} were negligible. NLL was similar across all four context sizes. Overall, these results suggest that description-based selection is a reasonable default, with the benefit of target-based selection diminishing as context size increases.

Auxiliary assay selection4 · NLL ↓4 · Ros2R^2_{\mathrm{os}} ↑8 · NLL ↓8 · Ros2R^2_{\mathrm{os}} ↑16 · NLL ↓16 · Ros2R^2_{\mathrm{os}} ↑32 · NLL ↓32 · Ros2R^2_{\mathrm{os}} ↑
Description-based2.569 ± 0.4670.090 ± 0.0231.756 ± 0.0240.125 ± 0.0081.495 ± 0.0300.175 ± 0.0151.374 ± 0.0150.210 ± 0.017
Target-based2.588 ± 0.4790.102 ± 0.0121.757 ± 0.0300.130 ± 0.0121.499 ± 0.0320.176 ± 0.0101.381 ± 0.0140.210 ± 0.017

We hope the additional analyses address your questions. We would be happy to clarify anything that remains open. If you have no remaining concerns, we would appreciate it if you would consider raising the score for our submission.

References

M. Ashman et al., "In-Context In-Context Learning with Transformer Neural Processes", AABI 2024

Thank you for your detailed answer and for the additional experiments to address my concerns. I will raise my score to a 4.

Thank you again for your constructive feedback, which helped us improve the manuscript. We are glad to have addressed your concerns and sincerely appreciate your support of our work.

Comments on the paper

2 comments
Area_Chair_WLCaOpenReview ↗

Please check the authors' rebuttal ASAP.

Dear reviewers,

As the discussion period will end very soon, please check the authors' rebuttal and see if your concerns have been addressed. Thank you.

-AC

AuthorsOpenReview ↗

Rebuttal Summary

We thank the Area Chair for overseeing the review process and the reviewers for their constructive feedback.

Summary of rebuttal contributions

  • Compared target-based and description-based auxiliary assay selection, providing guidelines for practitioners when both sources of information are available
  • Analyzed what biological and experimental information is captured by description-based assay relatedness
  • Demonstrated that adding up to 20 related auxiliary assays does not dilute task-specific data
  • Clarified why we used 64 pseudo-tokens per auxiliary assay for ICICL-TNP and extended its comparison with MetaHeta (our method) on FS-Mol, showing that even with 128 pseudo-tokens per auxiliary assay, ICICL-TNP is still outperformed by MetaHeta
  • Scaled up the controlled heterogeneity experiment by adding up to 20,000 non-TK tasks to the core EPHA meta-dataset and evaluated the effect across two model sizes

Reviewer evaluations

All three reviewers appreciated the breadth of our evaluation, highlighting the controlled heterogeneity experiment as a strength of our work. All three recommend acceptance.

  • Reviewer 8T7R appreciated the efficiency of MetaHeta’s architecture and raised their score to 4/4 (clear acceptance) following the rebuttal
  • Reviewer mfYi highlighted the soundness of our methodological approach and currently recommends weak acceptance (3/4). We await confirmation that our response answered their questions in full
  • Reviewer 4we1 recognized the practical relevance of addressing assay heterogeneity in bioactivity prediction. The reviewer confirmed that our response addressed their concerns, described the manuscript as "solid", and maintained their recommendation of weak acceptance (3/4)

Significance of this work

  • We demonstrate in a controlled experiment that assay heterogeneity can hinder meta-learning performance in few-shot bioactivity prediction
  • We introduce MetaHeta, a meta-learning framework that addresses this challenge by conditioning predictions on data from related assays
  • MetaHeta is designed for practical use. Its architecture combines exact and linear attention to process auxiliary measurements efficiently, and practitioners can define assay relatedness flexibly based on the information available, such as target identity or assay descriptions

Taken together, these contributions give us confidence that this work will be of interest to the ICLR and AI for Science communities.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.