acceptodds
Under review as a conference paper at ICLR 2027

MGDEval: A Curated Multi-Endpoint Benchmark for Generalization in Molecular Glue Degrader Activity Prediction

Abstract

Molecular glue degraders (MGDs) are small molecules that induce or stabilize interactions between target proteins and E3 ubiquitin ligases, leading to target ubiquitination and subsequent proteasomal degradation. Their activity depends on the specific molecular and protein context, yet experimentally reported bioactivity remains fragmented across public resources, limiting systematic computational evaluation. We introduce MGDEval, an integrated data resource and benchmark dedicated to MGD activity prediction. Through manual curation, MGDEval integrates 19,815 records, including 9,893 with quantitative activity information, while preserving biological context and source evidence. To assess generalization to new scaffolds and target proteins, we benchmark 45 representation–model configurations for classification and regression across DC50, Dmax, EC50, and IC50. Classification performance declined with increasing distribution shift: mean AUROC fell from 0.895 under random splitting to 0.839 under scaffold splitting and 0.561 under target-cluster splitting. Analyses of this gap showed reduced chemical and protein support under target-cluster splitting and limited transfer gains from current protein representations, motivating models that learn how molecules, targets, and effectors jointly determine activity. Exact-label activity measurements and cell-line information improved generalization, highlighting the importance of data quality and biological context for prediction on unseen targets. MGDEval provides a reusable data foundation and reproducible evaluation framework for advancing activity prediction and guiding molecular glue discovery across new chemical and target spaces.

Primary area:
datasets and benchmarks
Keywords:
molecular glue degraders, machine learning, activity prediction, benchmark, generalization, drug discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uDzfOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper curates a dataset and benchmark for molecular glue degrader activity prediction. The authors trace records from four databases back to primary sources, producing 19,815 records (5,170 exact-label, 4,723 interval, 9,922 unlabeled) across four endpoints, then evaluate 45 representation×model configurations under random, scaffold, and target-cluster splits. Mean AUROC drops 0.895 → 0.839 → 0.561; mean R² 0.646 → 0.499 → −0.190. I review as an ML researcher without domain expertise, the value of this paper lies in a set of diagnostic analyses of the generalization gap.

Critical strengths
  • The diagnostic analyses go well beyond typical benchmark papers: a 1-NN retrieval control, modality-randomization controls, label-precision and cell-line ablations, and a separation of calibration from selective-prediction utility.
  • The support-stratified AUROC decomposition — attributing 74.4% of EC50’s net discrimination to pairs where both samples have high chemical support — is an original and reusable protocol for any work claiming OOD generalization.
Critical weaknesses
  • No variance is reported anywhere, and the target-cluster setting has too few independent entities to support its conclusions.

  • Curation quality is asserted but never quantified. The central claim is that manual curation improves on four existing databases; the evidence is four case studies, with no inter-annotator agreement, error rate from independent re-annotation, or systematic diff against sources. For a dataset paper this is the most important missing experiment.

  • The benchmark discards its most distinctive property in the first preprocessing step: §4.1 collapses interval and censored measurements to midpoints or reported bounds (79% of IC50 records). Intervals should not be collapsed to point values — ML already has the right tools for this and here the censoring is informative rather than incidental: a compound is written as “> c” precisely because it is too weak to quantify. Substituting the bound therefore piles the negative class onto reporting conventions like 1000 and 10000 nM, which amounts to manufacturing labels rather than recording them.

  • The benchmark has no headroom gradient, and the paper is not positioned in the ML literature. All 45 configurations fall in ~[0.5, 0.7] AUROC on target-cluster with uniformly negative R². Trivial baseline ranks first even cannot measure progress. Separately, there is no engagement with DrugOOD, TDC/WILDS, or cold-start splits in drug-synergy prediction, which is often used in ML community.

Questions for authors
  • Please report cross-fold variance for all main tables and per-fold target-cluster results; and for that split, report Spearman/MAE plus a predict-the-training-mean baseline.

  • Could you quantify curation quality?

5 comments
AuthorsOpenReview ↗

Comment 1/5

Q1: Could you quantify curation quality?

Response:

We thank the reviewer for this question. While existing databases collect broad MG records, MGDEval focuses on MG degraders with activity labels about DC50/Dmax/IC50/EC50. Through manual verification, completion and standardization, we convert raw entries into an AI-ready dataset with unified prediction tasks, data splits and baselines. Our curation contributions fall into five areas.

(1) Organizing curated records by supervision signals. Raw MG records from MGDB, MGTbind, MolGlueDB and TPDdb are mixed with inconsistent activity labels and cannot be directly used for AI. We conducted extensive curation to stratify them into 3 tiers (19,815 records). Type 1 has exact activity labels for supervised regression and classification (5170 records). Type 2 includes interval/censored values for weak supervision (4723 records), and offers clear classification labels when activity class is conclusive (≥100nM or ≤100nM). Type 3 covers unlabeled target–molecule–effector associations for unsupervised pretraining (9922 records). This multi-supervision hierarchy enables AI modeling across diverse supervision regimes.

(2) Recovering patent activity labels and adding quantitative activity information. In MGTbind and TPDdb, 318 DC50/Dmax records reported activity only as B/C/D grades. We revisited the experimental results and grade definitions in the original patents to recover numerical or interval labels suitable for supervised learning, including 138 records from WO2024092039A1. Another 206 records used “+” symbols whose numerical ranges could not be verified; these were included in the unsupervised data. Overall, by reviewing papers, supplementary materials, and patents, we added exact-value activity data absent from the 4 source databases for 817 distinct records, e.g., we retrieved 7 IC50 measurements from WBC100 across varied cellular backgrounds from a single patent.

(3) Separating mixed records, completing protein sequences, and rectifying molecular structures. We split 74 multi-effector entries in MGDB and 64 multi-target entries in MolGlueDB by tracing original papers to match each measurement with its target and effector. We filled missing target/effector sequences for 1,461 supervised samples from MolGlueDB and TPDdb. Among all sources, 1,481 records had invalid SMILES incompatible with feature extraction. Via RDKit validation, desalting, standardization and literature cross-check, we rectified and standardized 1,038 molecular structures. For unsupervised data, we further supplemented 2,872 target sequences, 2,352 effector sequences and rectified the structures of 661 molecules.

(4) Collecting experimental conditions and building standardized context annotations. The 4 source databases lacked experimental-condition descriptions linked to individual activity measurements. We manually reviewed the source papers and patents, matching conditions scattered across the main text, methods, figure captions, and supplementary materials to the corresponding measurements. This completed 1,739 cell-line fields and 941 treatment-duration fields in the labeled data. Based on the retrieved information, we organized 32 categories of keyword fields, including detection methods, assay readouts, treatment duration, dosing, and experimental constructs. These fields standardized fragmented and inconsistently worded descriptions into a uniform context format, producing experimental-context annotations for 13,696 records.

(5) Correcting activity values, and standardizing/completing units. We checked the experimental descriptions and results in the source papers and patents to correct 36 records from IC50 to EC50, perform 439 concentration-unit conversions, and supply standardized unit annotations for 1,581 records without explicit units. We also corrected activity values verified against the original sources, e.g., JWJ-01–306’s reported degradation was corrected from 0.6% (MGDB) to 60%.

Quantified improvements in field coverage compared with the four source databases.

  • Target protein sequences: 78.11% → 100%.
  • Effector protein sequences:80.73% → 100%.
  • Cell lines: 27.95% → 77.13%, standardized to Cellosaurus IDs.
  • Treatment time: 42.7% → 70.87%.
  • Experimental context: 0% → 69.12%.

For items not reaching 100%, the required information is absent from original papers and patents, and we have made our best attempt at data recovery.

Annotation error rate in the random audit. During curation, all records underwent multiple rounds of checks. In the final random audit, we checked 6,848 of the 13,696 records with experimental-context annotations against the original papers or patents and identified annotation errors in nine records (0.13%). We corrected the flagged records, screened the full dataset, and incorporated the resulting corrections into the final version.

Our responses to the remaining questions continue in Comments 2/5–5/5.

AuthorsOpenReview ↗

Comment 2/5

Q2: The target-cluster setting has too few independent entities to support its conclusions.

Response:

Thank you for highlighting target coverage. In public data, relatively few molecular glue degrader targets have both endpoint-specific quantitative measurements and complete molecular and protein information. DC50/Dmax/EC50/IC50 cover 37/22/28/47 targets and 1,626/1,077/1,177/2,641 distinct molecules, respectively. Thus, the supervised data contain many molecules across relatively few targets. These data reflect our best efforts to curate available public records; we will continue adding measurements for new targets.

Records without quantitative activity labels are retained as Type 3 data and used for pretraining. Type 3 covers 168 targets, including 112 absent from all four supervised endpoints.

Target-cluster splitting, a scaffold split at the target-protein level, represents our hardest task with strong translational implications. To strengthen our conclusions, we added a less stringent three-fold Target-ID split (corresponds to “random” splitting of target proteins, where the training and test sets contain distinct target IDs without enforcing cluster-based grouping) and a Leave-one-target-out evaluations (all compounds of a single target are reserved for testing in turn). Target-ID keeps all samples for each target in one fold while allowing similar targets across folds. Mean AUROCs for Scaffold/Target-ID/Target-cluster were 0.839/0.622/0.561. Leave-one-target-out evaluation, training on all remaining targets, yielded 0.633. Both evaluations support our main finding that generalizing to new targets is more challenging than to new scaffolds. Complete fold-level results have been added to the anonymous repository.

Q3: Please report cross-fold variance for all main tables and per-fold target-cluster results; and for that split, report Spearman/MAE plus a predict-the-training-mean baseline.

Response:

For space reasons, the original manuscript reported only fold-averaged results. At submission, the anonymous repository already included cross-fold SDs, Spearman correlation, RMSE, and MAE. We added per-fold results for all splits, cross-fold variances and SDs, and all regression metrics to the repository. We also added SDs to the revised manuscript tables.

(1) Cross-fold variance. To illustrate cross-fold variance for the reviewer, we report example AUROC results (mean ± SD). For each endpoint, we select the configuration with the highest mean AUROC on the Random split. Our goal is to show how its performance drops across generalization settings of varying difficulty.

Endpoint Random ↑  Scaffold ↑  Target-ID ↑  Target-cluster ↑ 
DC50 .886 ± .007  .816 ± .041  .623 ± .097  .567 ± .171 
Dmax .899 ± .019  .816 ± .047  .604 ± .042  .570 ± .195 
EC50 .954 ± .009  .929 ± .043  .809 ± .131  .585 ± .266 
IC50 .936 ± .011  .849 ± .048  .631 ± .083  .564 ± .115 

(2) Per-fold Target-cluster results: Spearman ρ, MAE, and R². We show examples of per-fold regression results. For each endpoint, we select the configuration with the lowest mean MAE on the Random split. Values are listed in fold order 1/2/3.

Endpoint   Spearman ρ ↑          MAE ↓           R² ↑ 
DC50 .277 / .419 / −.354  .591 / .694 / 1.083  .088 / .075 / −.256 
Dmax −.016 / .432 / .083  36.954 / 21.901 / 21.944  −2.676 / −.084 / −.088 
EC50 .694 / .304 / −.028  .653 / 1.065 / .953  .287 / .085 / −.079 
IC50 .140 / .112 / .324  .925 / .907 / .890  .016 / −.376 / −.062 

(3) Predict-the-training-mean baseline. For each Target-cluster fold, the baseline predicts the mean of the training labels.

Metric    DC50      Dmax     EC50     IC50 
R² ↑ (mean ± SD) −.041 ± .030  −.353 ± .377  −.811 ± .624  −.290 ± .088 
MAE ↓ (mean ± SD) .790 ± .171  24.040 ± 1.970  1.540 ± .260  .954 ± .075 

MAE is in pActivity units for concentration endpoints and percentage points for Dmax.

Our responses continue in Comment 3/5.

AuthorsOpenReview ↗

Comment 3/5

Q4: The benchmark has no headroom gradient. All 45 configurations fall in ~[0.5, 0.7] AUROC on target-cluster with uniformly negative R². Trivial baseline ranks first even cannot measure progress.

Response:

Thank you for highlighting the importance of measuring progress on this benchmark.

In our initial design, we implemented 3 evaluation schemes: molecular Random/Scaffold split and Target-cluster split. The scaffold split simulates the discovery of novel high-activity molecular glues against known disease targets for which some labeled molecular glue compounds are already available. By contrast, the Target-cluster, can be viewed as the “scaffold” split at the target-protein level. Target proteins are grouped by sequence and structural similarity and partitioned by clusters, which mimics the discovery of molecular glue candidates for novel disease targets with no prior known molecular glues. This setting resembles a zero-shot learning setup for new targets, represents our most challenging evaluation task and carries substantial practical implications for advancing healthcare. The weak predictive performance observed under this cross-target setting computationally corroborates the inherent difficulty of this task. Our MGDEval establishes cross-target activity prediction as a challenging benchmark task for the ML community, and highlights that the molecular glue paradigm opens up a vast technical frontier for addressing novel, traditionally undruggable targets, with ample room for future methodological advances. We believe this is the core significance and value of our benchmarking work.

Although quantitative prediction under Target-cluster remains challenging, the evaluated models outperform the training-mean baseline on average, with mean R² of −0.190 versus −0.374. This shows that the models retain predictive information even in the most demanding setting, and may inspire future researchers to develop more sophisticated AI models for cross-target molecular glue prediction.

Furthermore, inspired by your comment, to provide a clearer difficulty gradient, we added the milder Target-ID (corresponds to “random” splitting of target proteins, where the training and test sets contain distinct target IDs without enforcing cluster-based grouping) and Leave-one-target-out evaluations (all compounds of a single target are reserved for testing in turn) described in Q2. Their mean AUROCs of 0.622 and 0.633, compared with 0.561 under Target-cluster, provide complementary evaluation settings with different levels of difficulty. These settings allow future methods to be assessed across both related-target transfer and more demanding target-cluster generalization.

In brief, our MGDEval provides curated data, a common evaluation protocol, and baselines for measuring progress. Covering five distinct evaluation scenarios ranging from molecular-level Random split and Scaffold split to target-centric Leave-one-target-out, random Target-ID split and Target-cluster split, this benchmark — refined based on your professional comment — establishes a spectrum of difficulty gradients for molecular glue discovery. The biomedical challenges and practical relevance increase progressively across these settings, which naturally leads to a gradual decline in model performance. This trend clearly reveals the substantial room and strong necessity for further research in these cross-target tasks. We believe this multi-granularity evaluation framework can serve as a solid benchmark and reference for future AI-driven molecular glue discovery.

In addition, our failure analyses and ablations examine how protein representations, label quality, and cellular information affect performance. We also test pretraining on unlabeled molecule–protein associations. These results point to learning transferable interactions among molecules, targets, and effectors as a direction for improving cross-target generalization (§5.1–5.3).

Our responses continue in Comment 4/5.

Reviewer_1rWcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MGDEval, a manually curated resource and benchmark for molecular glue degrader activity prediction across DC50, Dmax, EC50, and IC50. It evaluates 45 representation–model combinations under random, scaffold, and target-cluster splits, finding substantially weaker generalization to held-out targets.

Critical strengths
  1. The manual reconstruction of measurements from papers and patents, including molecular identities, protein roles, and experimental context, addresses a real obstacle to reliable molecular glue modeling.
  2. The multi-endpoint evaluation and target-cluster splits provide a useful perspective on generalization beyond familiar molecules and targets. The similarity and nearest-neighbor analyses help interpret the reported performance gaps.
Critical weaknesses
  1. Target-cluster splitting can induce a substantial shift in molecular chemistry, while the three target folds have uneven sample and class distributions. Consequently, the performance drop cannot be attributed specifically to unseen protein targets, and the stability of the estimates is unclear.
  2. EC50 and IC50 may measure downstream cellular responses or biochemical inhibition rather than target degradation. Their inclusion under molecular glue activity prediction without a clearer mechanistic distinction weakens the interpretation of the four-endpoint benchmark.
  3. The main contribution is data consolidation and standardized evaluation rather than a new modeling method. Existing curated molecular glue resources, such as MGDB and MGTbind (are also the source of MGDEval), and the related TACK benchmark for PROTACs limit the methodological novelty. The added benefit of MGDEval's curation over the original resources is not clearly established.
Questions for authors
  1. How might interval or censored measurements affect the reported benchmark results?
  2. How many observations from different assay contexts were merged during sample construction, and how robust are the target-cluster findings to alternative fold assignments?
5 comments
AuthorsOpenReview ↗

Comment 1/3

Q1: What added benefit does MGDEval’s curation provide over the original resources?

Response:

We thank the reviewer for asking us to clarify the added benefits of our curation over the source resources. Existing databases broadly collect molecular glue records, whereas MGDEval focuses on molecular glue degraders with activity labels. Through manual verification, completion, and standardization, we convert these records into an AI-ready dataset and provide unified prediction tasks, data splits, and baseline evaluations. Our main curation efforts and quantitative contributions cover four areas.

(1) Recovering patent activity labels and adding quantitative activity information. In MGTbind and TPDdb, 318 DC50/Dmax records reported activity only as B/C/D grades. We revisited the experimental results and grade definitions in the original patents to recover numerical or interval labels suitable for supervised learning, including 138 records from WO2024092039A1. Another 206 records used “+” symbols whose numerical ranges could not be verified; these were included in the unsupervised data. By reviewing papers, supplementary materials, and patents, we also added quantitative measurements absent from the source databases to 817 distinct records, including seven IC50 measurements for WBC100 across different cellular backgrounds.

(2) Separating mixed records, completing protein sequences, and recovering usable molecular structures. We separated 74 records listing multiple effectors in MGDB and 64 listing multiple targets in MolGlueDB, consulting the source papers to identify the target and effector corresponding to each measurement. We also completed target and effector sequences for 1,461 supervised samples from MolGlueDB and TPDdb. Across the source databases, 1,481 records contained invalid SMILES that prevented direct molecular feature extraction. Using RDKit checks, salt removal, and standardization, together with verification against the source literature, we recovered usable structural representations for 1,038 molecular records. In the unsupervised data, we additionally completed 2,872 target sequences and 2,352 effector sequences and recovered usable structural representations for 661 molecular records.

(3) Collecting experimental conditions and building standardized context annotations. The source databases lacked experimental-condition descriptions linked to individual activity measurements. We manually reviewed the source papers and patents, matching conditions scattered across the main text, methods, figure captions, and supplementary materials to the corresponding measurements. This completed 1,739 cell-line fields and 941 treatment-duration fields in the labeled data. Based on the retrieved information, we organized 32 categories of keyword fields, including detection methods, assay readouts, treatment duration, dosing, and experimental constructs. These fields standardized fragmented and inconsistently worded descriptions into a uniform context format, producing experimental-context annotations for 13,696 records.

(4) Correcting activity endpoints and values, and standardizing and completing units. We checked the experimental descriptions and results in the source papers and patents to correct 36 records from IC50 to EC50, perform 439 concentration-unit conversions, and supply standardized unit annotations for 1,581 records without explicit units. We also corrected activity values verified against the original sources; for example, the reported degradation for JWJ-01–306 was corrected from 0.6% to 60%.

Our responses to the remaining questions continue in Comments 2/3 and 3/3.

AuthorsOpenReview ↗

Comment 2/3

Q2: Given shifts in molecular chemistry and uneven sample and class distributions, can the performance drop be attributed specifically to unseen protein targets? How robust are the target-cluster findings to alternative fold assignments?

Response:

Thank you for your careful and insightful comments. Your comments helped us examine how shifts in chemical space and differences in fold composition relate to the performance drop under the Target-cluster split. The original Target-cluster split improves balance by minimizing weighted deviations across folds in sample counts, positive and negative counts, activity distributions, and cluster counts, with greater weight on sample and class counts. Because the data are concentrated on a few targets, holding out entire clusters still leaves differences between folds.

To address this concern, we added a milder three-fold Target-ID split: all samples for the same target remain in one fold, and similar proteins can occur in different folds. The new split matched or improved the original split on all six measures of fold balance. The average cross-fold AUROC SD fell by about one-third. Mean AUROC for DC50/Dmax/EC50/IC50 increased from 0.551/0.524/0.654/0.514 to 0.586/0.531/0.798/0.574. Even when the training set contained compounds similar to those in the test set, mean AUROC was lower when we held out entire targets than under the Scaffold split (0.622 vs. 0.839). Per-target analyses show that targets with greater support from similar molecules or related proteins generally perform better: chemical support correlates with AUROC for EC50 (Spearman’s ρ=0.939), and chemical and protein support correlate with AUROC for DC50 (ρ=0.525 and 0.462, respectively). These results show that, even when all test targets are unseen, predictive performance can differ substantially with the support available from similar molecules and related proteins in training.

To examine sensitivity to the splitting scheme, we also performed leave-one-target-out evaluation, holding out one target at a time for testing (mean AUROC: 0.633). Both this evaluation and the milder three-fold split preserve the main conclusion that cross-target generalization is more challenging. We will add per-fold results and revise the manuscript to jointly consider protein support, chemical support, and data composition when explaining and analyzing the performance drop.

Q3: EC50 and IC50 may measure downstream cellular responses or biochemical inhibition rather than target degradation. Their inclusion under molecular glue activity prediction without a clearer mechanistic distinction weakens the interpretation of the four-endpoint benchmark.

Response:

A molecular glue degrader brings a target protein to the cell’s degradation machinery. When cells depend on that protein for growth, losing it slows their growth. A recruitment EC50 shows how much compound is needed to bring the proteins together, the step that enables degradation [1]. In cells that depend on the target, a growth IC50 shows how much compound is needed to produce the downstream effect on cell growth [2]. A biochemical IC50 describes the compound’s effect on the activity tested in that assay. MGDEval includes these readouts alongside direct measurements of target degradation to capture both protein recruitment and downstream functional effects.

References: [1] Kozicka et al., Nat. Chem. Biol. (2024); [2] Krönke et al., Science (2014).

Our responses on the comparison with TACK, interval and censored measurements, and merging across assay contexts continue in Comment 3/3.

AuthorsOpenReview ↗

Comment 3/3

Q4: What does MGDEval contribute beyond existing curated molecular glue resources and the related TACK benchmark for PROTACs?

Response:

Thank you for the suggestion. MGDEval differs from TACK and makes additional contributions in five areas:

(1) Distinct data and research focus. TACK studies PROTACs, while MGDEval focuses on molecular glue degraders. PROTACs link target and E3 through two separate binding ends; glues alter protein interfaces, enabling two otherwise weakly interacting proteins to recognize each other. Lower molecular weight gives glues better drug-like properties. Molecular glue activity modeling remains an emerging field with scarce data and few dedicated benchmarks.

(2) Broader curation and activity coverage. TACK evaluates DC50/Dmax; MGDEval additionally includes complementary EC50/IC50 endpoints. We also recover experimental-context descriptions linked to individual measurements from original sources.

(3) Broader comparisons and stricter generalization tests. We evaluate more representations and classifiers and introduce target-based splits beyond TACK’s main scaffold setting to assess generalization to new drug targets.

(4) Tiered data use and unsupervised pretraining. We organize exact, interval, and activity-unlabeled records separately. Molecule–protein association pretraining enables unlabeled records to contribute to representation learning.

(5) Deeper failure analysis and ablations. We investigate cross-target performance drops through controls and ablations examining protein representations, label quality, and cellular information, providing directions for model improvement.

Together, these contributions extend data curation, activity coverage, and model evaluation beyond TACK.

Q5: How might interval or censored measurements affect the reported benchmark results?

Response:

Thank you for this helpful comment. Section 5.2 compares training on all measurements (A) with training on exact values only (B). Exact-only training improved average performance, but reduced training coverage. For IC50, most measurements were intervals or censored. Using only exact values removed most training examples and lowered AUROC on unseen targets. Following your suggestion, we added two experiments to examine this trade-off.

Mean AUROC (Random/Scaffold/Target-cluster):

  • A. All measurements (exact, interval, and censored): 0.856/0.774/0.561
  • B. Exact values only: 0.863/0.781/0.585
  • C. Exact values plus interval or censored measurements with a clear class (new): 0.859/0.777/0.566
  • D. Exact values and intervals, with censored measurements removed (new): 0.860/0.777/0.576

In C, a class is clear when the reported bounds determine the label. If ≤100 nM is positive, <50 nM is positive, while >50 nM could be either class. Censoring leaves 8.7% of samples with ambiguous class labels.

Mean AUROC changed little across the four settings, and removing censored measurements improved mean R². The benchmark’s main finding held across all settings, with performance declining from random to scaffold to target-cluster splits.

Q6: How many observations from different assay contexts were merged during sample construction?

Response:

Merging measurements across assay contexts reduced the dataset by 313 rows (3.30%). We reran the benchmark with these measurements kept separate, and the main finding held.

Your question also prompted us to test whether the details in assay descriptions help prediction. We encoded each description with Qwen-0.6B and appended the embedding to the original features. Using the same records and folds, mean AUROC across the four endpoints went from 0.904/0.833/0.594 to 0.903/0.840/0.616 (Random/Scaffold/Target), with the largest gain on Target. We will add these results to the revised manuscript.

Reviewer_c5fEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MGDEval, a curated dataset and benchmark for molecular glue degrader (MGD) activity prediction. The authors integrate existing databases and further curate experimental records, obtaining 19,815 records, including 9,893 with quantitative activity information. They evaluate 45 representation–model combinations across four endpoints (DC50, Dmax, EC50, and IC50) using random, scaffold, and target-cluster splits. The results highlight poor generalization to unseen targets and examine the effects of data quality, cellular context, and association pretraining.

Critical strengths
  • Predicting MGD activity is an important yet underexplored problem. Collecting and standardizing experimental activity data across multiple sources could provide a useful resource for the community, particularly given the limited availability of public MGD datasets.
Critical weaknesses
  1. Unclear contribution of the dataset. Several curated MGD databases already exist, including MGDB, MGTbind, MolGlueDB, and TPDdb. The main contribution of this work appears to be integrating and further curating these existing resources. Although the authors describe additional manual extraction and correction efforts, it remains unclear how much genuinely new information is contributed beyond the original databases.
  2. Despite the large number of records, the supervised benchmarks contain only 19–43 unique targets per endpoint, with just 19 targets for Dmax. With three-fold target-cluster cross-validation and highly imbalanced target coverage, the effective number of independent evaluation units is very small. This raises concerns about the statistical reliability of the reported results and whether meaningful conclusions about generalization can be drawn.
Questions for authors
  1. Dataset novelty and availability: How many records contain genuinely new measurements or annotations not available in the original databases? Can the authors release the curated dataset, including source-level provenance and mappings to the original records, to substantiate the contribution and enable independent evaluation?
  2. Clarification on molecular structure recovery: In Appendix A.1, the authors report recovering complete molecular structures for 1,038 of 1,481 records. However, the original databases already provide molecular structures, including SMILES. Could the authors clarify what was missing or incomplete in these records and why structure recovery was necessary? Specifically, how many structures were genuinely absent from the original databases, versus those requiring correction or standardization? A breakdown by source database would help clarify the actual contribution of this curation effort.
  3. Statistical reliability: How many distinct targets and target clusters are included in the training and test sets of each fold? Given the very limited number of targets and their highly uneven representation, can the authors provide uncertainty estimates or demonstrate that the main conclusions are robust to different target-cluster partitions?
  4. Assay comparability: How do the authors account for differences in experimental protocols when aggregating activity measurements and computing evaluation metrics? Can they report within-assay performance, aggregated across sufficiently populated assays, to assess whether the conclusions remain valid after controlling for assay-dependent variability?
4 comments
AuthorsOpenReview ↗

Comment 1/3

Q1: About the dataset novelty, contribution or enhancements of our MGDEval compared to the 4 original databases. (Reviewer’s Weakness 1 and Question 1)

Response:

We appreciate your inquiry about MGDEval’s enhancements over the 4 source databases. Our curation adds quantitative activity labels and experimental-condition text, fills missing fields, and corrects existing records.

Curation gains over the 4 source databases (19,815 records).

FieldResult (before → after)Records
SMILESStandardization via RDKit16,465
SMILESRectified via manual retrieval of papers and patents+1,699
Exact-value activity measurements4353 → 5170+817
Structured experimental context0% → 69.12%+13,696
Cell line27.95% → 77.13%+9,745
Treatment time42.70% → 70.87%+5,582
Available target protein UniProtID or Sequence78.11% → 100%+4,338
Available effector protein UniProtID or Sequence80.73% → 100%+3,818

For incomplete entries (<100%), required data were unavailable in original papers and patents despite our best recovery attempts. Standardized prediction tasks, data splits and baseline evaluations enable our curated dataset directly usable for AI modeling. Our main contributions are outlined below.

(1) Organizing curated records by supervision signals. Raw MG records from MGDB, MGTbind, MolGlueDB and TPDdb are mixed with inconsistent activity labels and cannot be directly used for AI. We conducted extensive curation to stratify them into 3 tiers (totally 19,815 records). Type 1 has exact activity labels for supervised regression and classification (5170 records). Type 2 includes interval/censored values for weak supervision (4723 records), and offers clear classification labels when activity class is conclusive (≥100nM or ≤100nM). Type 3 covers unlabeled target–molecule–effector associations for unsupervised pretraining (9922 records). This multi-supervision hierarchy enables AI modeling across diverse supervision regimes.

(2) Recovering patent activity labels and adding quantitative activity information. In MGTbind and TPDdb, 318 DC50/Dmax records reported activity only as B/C/D grades. We revisited the experimental results and grade definitions in the original patents to recover numerical or interval labels suitable for supervised learning, including 138 records from WO2024092039A1. Another 206 records used “+” symbols whose numerical ranges could not be verified; these were included in the unsupervised data. Overall, by reviewing papers, supplementary materials, and patents, we added exact-value activity data absent from the 4 source databases for 817 distinct records, e.g., we retrieved seven IC50 measurements from WBC100 patent across varied cellular backgrounds from a single patent.

(3) Separating mixed records, completing protein sequences, and rectifying molecular structures. We split 74 multi-effector entries in MGDB and 64 multi-target entries in MolGlueDB by tracing original papers to match each measurement with its target and effector. We filled missing target/effector sequences for 1,461 supervised samples from MolGlueDB and TPDdb. Among all sources, 1,481 records had invalid SMILES incompatible with feature extraction. Via RDKit validation, desalting, standardization and literature cross-check, we rectified and standardized 1,038 molecular structures. For unsupervised data, we further supplemented 2,872 target sequences, 2,352 effector sequences and rectified the structures of 661 molecules.

(4) Collecting experimental conditions and building standardized context annotations. The 4 source databases lacked experimental-condition descriptions linked to individual activity measurements. We manually reviewed the source papers and patents, matching conditions scattered across the main text, methods, figure captions, and supplementary materials to the corresponding measurements. This completed 1,739 cell-line fields and 941 treatment-duration fields in the labeled data. Based on the retrieved information, we organized 32 categories of keyword fields, including detection methods, assay readouts, treatment duration, dosing, and experimental constructs. These fields standardized fragmented and inconsistently worded descriptions into a uniform context format, producing experimental-context annotations for 13,696 records.

(5) Correcting activity values, and standardizing/completing units. We checked the experimental descriptions and results in the source papers and patents to correct 36 records from IC50 to EC50, perform 439 concentration-unit conversions, and supply standardized unit annotations for 1,581 records without explicit units. We also corrected activity values verified against the original sources, e.g., JWJ-01–306’s reported degradation was corrected from 0.6% (MGDB) to 60%.

We clarified and quantified our curation contributions in the revised manuscript. Our responses continue in Comments 2/3 and 3/3.

AuthorsOpenReview ↗

Comment 2/3

Q2: Could the authors clarify what was missing or incomplete in these records and why structure recovery was necessary? (Reviewer’s Question 2)

Response:

Invalid SMILES prevented direct molecular feature extraction in these records. Of the 1,481 affected records, we rectified invalid SMILES in 1,038 molecular records through RDKit checks, salt removal, standardization, and verification against the source literature.

Q3: About limited and uneven target coverage and the robustness of the conclusions across target splits (Reviewer’s Weakness 2 and Question 3)

Response:

Thank you for highlighting target coverage. Public data of the 4 source databases with endpoint-specific quantitative measurements and complete molecular and protein information cover relatively few targets. Dmax has the lowest coverage, with 22 targets, 19 sequence clusters, and 1,077 small molecules. The five most represented targets account for 71.0% of its samples. We include records without quantitative activity labels in the Type 3 unlabeled dataset and use them for pretraining. Type 3 covers 168 targets, including 112 that are absent from all four supervised endpoints. In fact, the Target-cluster, can be viewed as the “scaffold” split at the target-protein level. Target proteins are grouped by sequence and structural similarity and partitioned by clusters, which mimics the discovery of molecular glue candidates for novel disease targets with no prior known molecular glues. This setting resembles a zero-shot learning setup for new targets, represents our most challenging evaluation task and carries substantial practical implications for advancing healthcare.

The scarcity of annotated targets represents an inherent data constraint that the community, including our work, has to confront. This situation arises from the early-stage nature of molecular glue research and also underscores the substantial untapped potential for discovering potent molecular glue degraders against a broad range of disease-relevant targets.

To enhance the robustness of our conclusions, we added the milder Target-ID (corresponds to “random” splitting of target proteins, where the training and test sets contain distinct target IDs without enforcing cluster-based grouping) and Leave-one-target-out evaluations (all compounds of a single target are reserved for testing in turn). Target-ID keeps all samples for each target in one fold while allowing related targets in different folds. This split retains close structural matches between training and test compounds. Mean AUROC for Scaffold / Target-ID / Target-cluster was 0.839 / 0.622 / 0.561. Leave-one-target-out evaluation yielded a mean AUROC of 0.633 from pooled out-of-fold predictions.

Both evaluations we added support our main finding that generalizing to new targets is more challenging than generalizing to new scaffolds. Until now, our MGDEval covers five distinct evaluation scenarios ranging from molecular-level Random split and Scaffold split to target-centric Leave-one-target-out, random Target-ID split and Target-cluster split. This benchmark establishes a spectrum of difficulty gradients for molecular glue discovery, and reflects two discovery goals: finding new molecular glue compounds for known target proteins and developing molecular glue compounds for new disease target proteins. MGDEval establishes cross-target prediction as a challenging benchmark task, provides baselines, and analyzes where current models struggle. We will continue adding quantitative measurements to expand supervised target coverage. Detailed results are available in the anonymous repository.

Q4: Can the authors release the curated dataset, including source-level provenance and mappings to the original records? (Reviewer’s Question 1)

Response:

We plan to release the full dataset upon acceptance. The curated dataset demo in our anonymous repository shows how we organize molecular and assay information and how readers can trace each curated record back to the original evidence. One patent example illustrates both the mapping and the correction.

Curated record: Compound 1, Dmax >80% | Source record: PROTACDB4.0_Sheet1_R016558 (reported 80%) | Patent evidence: WO2024092039A1, Table 2, compound 1 (grade D, >80%).

Our responses continue in Comment 3/3.

AuthorsOpenReview ↗

Comment 3/3

Q5: How many distinct targets and target clusters are included in the training and test sets of each fold? (Reviewer’s Question 3)

Response:

Numbers correspond to folds 1/2/3.

EndpointTraining targetsTest targetsTraining clustersTest clusters
DC5024/27/2313/10/1421/22/2111/10/11
Dmax16/14/146/8/813/13/126/6/7
EC5019/19/189/9/1017/17/168/8/9
IC5033/30/3114/17/1629/29/2814/14/15

Q3 reports robustness checks under alternative target splits. We also analyzed predictive uncertainty in §5.3 and Appendix Table D6C. Temperature scaling reduced highly confident errors from 24.9% to 9.0% of all MLP predictions under the target-cluster split.

Q6: How are differences in experimental protocols handled when aggregating activity measurements and computing evaluation metrics? Do the conclusions hold when performance is evaluated within sufficiently populated assays? (Reviewer’s Question 4)

Response:

Thank you for pointing out that differences in assay protocols may affect both activity aggregation and performance evaluation, and for suggesting a within-assay analysis to test the robustness of our conclusions.

Adding assay information to the benchmark. The original benchmark aggregated activity measurements by molecule, target protein, effector protein, and cell line. This could combine measurements from different assay protocols used in the same cell line. Following your suggestion, we added two additional experiments. In the first experiment, we retained measurements with different assay descriptions as distinct records during activity aggregation. We then trained the models using the original features. Mean AUROC increased by about 0.012. In the second experiment, we used the same records and splits, encoded the assay descriptions with an LLM (Qwen 0.6B), and concatenated the LLM embeddings with the original input features. Across the four endpoints, mean AUROC for Random / Scaffold / Target increased from 0.895 / 0.839 / 0.561 to 0.903 / 0.840 / 0.616. These results show that retaining and using assay information improved mean AUROC, especially for the important cross-target tasks (from 0.561 to 0.616).

Within-assay performance. Thank you for this suggestion. Following your suggestion, we evaluated rankings within each assay. In each test fold, we calculated Spearman’s ρ for groups with ≥5 distinct compounds and varying activities, then averaged groups equally. The three splits shared 1,471 test samples from 31 assay groups. Spearman’s ρ was 0.589/0.169/0.213 for Random/Scaffold/Target. The random split still yielded better within-assay rankings than either cold split, and the two cold splits gave similar results.

These results show that the paper’s main conclusions still hold.

AuthorsOpenReview ↗

Robustness and practical meaning of the target splits

Thank you for raising the question of what conclusions we can draw given the limited and uneven target coverage. We would like to clarify how the target splits reflect different drug discovery goals and what we learned by varying them.

The scaffold split reflects the search for more potent glues against an established target. The Target-ID and leave-one-target-out evaluations ask whether knowledge from studied proteins can guide the search for glues against a new disease target. The Target-cluster split goes one step further by holding out an entire group of related proteins. This setting reflects the challenge of extending molecular glue discovery to a less-studied group of disease targets.

The main conclusion held across all three target evaluations. Prediction on new targets remained more difficult than prediction on new scaffolds, both when we held out one target at a time and when we reassigned targets to folds without sequence clustering. Performance improved when related proteins could remain available and was lowest when entire target clusters were held out. This shows why the distinction matters in practice: extending molecular glue therapies to less-studied targets requires models that can transfer what they learn across increasingly dissimilar proteins.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.