acceptodds
Withdrawn from ICLR 2027

When Biological Evidence Misleads: Reliability-Aware Transfer for Cross-Scale Drug Response Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Knowledge-guided drug-response models often inject external biological evidence as though its availability implied its usefulness. This assumption is fragile when evidence is sparse, unevenly covered across compounds, or shifted between training and test drugs. We study cross-scale drug-response prediction with a frozen BindingDB drug–protein encoder, pathway-level evidence construction, and a cell-line response model in which the external channel can be suppressed independently of the response prediction. The evaluation is built around matched no-evidence counterfactuals and two distribution-shift protocols: held-out-drug and Bemis–Murcko scaffold-disjoint cross-validation. Across these settings, unconditional fusion can create large drug-specific regret, whereas a training-support density rule reduces the most severe observed harm without using test-time response labels. The rule improves supported-drug macro-AUC in the held-out-drug experiment and preserves a safer worst-drug profile under scaffold-disjoint replication; a provenance-weighted variant is slightly better on the latter mean and regret, so neither mechanism is a uniform winner. Fixed-checkpoint corruption tests show that the density rule is sensitive to out-of-support evidence but remains vulnerable when partial corruption leaves coverage nominally in range. The learned gate confidence is not a calibrated utility score in this data regime, and a SelectiveNet-style coverage constraint reintroduces harmful transfer on an out-of-support drug. These results support a bounded mechanism-level conclusion: evidence availability, support under the training distribution, and realized predictive utility are distinct quantities that should be evaluated separately under drug-level shift.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Negative transfer, Reliability-aware machine learning, Knowledge-guided learning, Distribution shift, Selective prediction, Drug response prediction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CkkiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies negative transfer from external drug–protein binding evidence to cell-line drug-response prediction. The authors construct a frozen evidence branch from BindingDB-supervised drug–protein representations, aggregate the evidence into 13 pathway slots, and introduce a selective fusion mechanism. Its principal reliability rule permits evidence for a test drug only if its number of covered pathways is positive and does not exceed the maximum coverage observed among supported training drugs. The paper further distinguishes (i) performance relative to an independently trained no-evidence model from (ii) the inference-time effect of disabling evidence within a fixed checkpoint. Experiments on 22 drugs, eight of which have mapped evidence, include held-out-drug and scaffold-disjoint splits, per-drug harm statistics, seed sensitivity, and inference-time corruptions.

The central question is important, and the paper is unusually transparent about negative and inconclusive findings. Nevertheless, I am not yet convinced that the proposed mechanism constitutes a sufficiently novel or validated reliability-aware transfer method for ICLR. The main gate is a simple one-sided pathway-count threshold; the apparent benefit may be largely attributable to suppressing evidence for a single outlying drug; and the larger-seed experiments do not establish consistent performance gains.

Critical strengths
  1. The paper appropriately challenges the assumption that available external biological evidence necessarily improves downstream predictions. Reporting per-drug negative transfer is more informative than reporting aggregate AUC alone.

  2. Matched seeds, several harm thresholds, per-drug results, conditional worst-drug sensitivity, and fixed-checkpoint corruptions are valuable. The Sorafenib partial-removal case is particularly informative: reducing pathway coverage can turn a rejected evidence source into an admitted, harmful one.

Critical weaknesses
  1. The proposed admission rule simply rejects evidence when its pathway count exceeds the maximum observed in training. While interpretable, this heuristic does not estimate evidence quality or predictive utility. Related work on quality-aware fusion and negative-transfer prevention already exists, including Provable Dynamic Fusion (ICML 2023), Residual Feature Integration (ICLR 2026), and confidence-aware multimodal fusion for drug–target prediction (AAAI 2026). The authors should clarify what is methodologically new beyond these approaches, while acknowledging their different tasks and guarantees.

  2. All supported training drugs satisfy the maximum-coverage rule by construction, so the hard rejection mechanism appears to matter mainly at test time. Table 7 shows a zero applied gate only for Sorafenib among the eight supported drugs, raising the concern that the improvement over the ordinary learned gate is driven largely by one compound. Please report fold- and seed-level thresholds, test-drug coverage, suppression decisions, and per-drug AUC differences against the learned-gate control. Also clarify whether the two methods produce identical predictions on non-suppressed drugs, and compare with matched-rate random rejection or alternative simple thresholds.

  3. Only eight of the 22 drugs have mapped positive evidence, despite the large number of cell-line response rows. This is insufficient to support broad claims about drug-level reliability, gate–utility relationships, or worst-drug robustness. More random seeds and overlapping cross-validation folds cannot replace evaluation on a larger, independently held-out set of drugs.

  4. Pathway count measures evidence coverage, not biological correctness or usefulness for a particular cell line. Sparse but incorrect evidence may pass the rule, while valid high-coverage evidence may be rejected. Although the paper includes pathway- and drug-shuffling tests, stronger controls should preserve coverage while introducing incorrect drug–target links or biologically mismatched pathway evidence. Comparing the count threshold with one selected using training-side validation utility would also help determine whether coverage actually predicts negative-transfer risk.

  5. The main evaluation transfers binding-derived evidence into cell-line drug-response prediction, rather than demonstrating reliable transfer to patient outcomes. As acknowledged by the authors, only 186 of 338 patient rows can be linked to the stated TCGA cohort, leaving patient-level conclusions unsupported. The paper should narrow its cross-scale and practical-reliability claims or provide a substantially stronger patient-level evaluation.

Questions for authors
  1. How many unique supported test drugs are ever rejected by the coverage rule?

  2. Are OOD abstention and the ordinary learned-gate baseline training-equivalent?

  3. How does the proposed rule compare against stronger, label-blind reliability estimators?

  4. What part of the auxiliary evidence is biologically informative?

  5. How sensitive is the conclusion to dataset construction and external evidence provenance?

Reviewer_BVyWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks when an auxiliary biological-evidence channel should be allowed to influence drug-response predictions for drugs not seen in training. A frozen drug–protein encoder, pretrained on BindingDB, produces an evidence vector for each of 13 pathway slots. These vectors are added to a response model through a gated residual, one gate per pathway. A simple rule, called “OOD abstention,” turns the channel off when a test drug covers more pathways than any supported training drug.

The main conceptual contribution is to separate two quantities that ablations usually treat as one. UtrainU_{\text{train}} compares a model trained with evidence against a separately trained model without it. UoffU_{\text{off}} zeroes the channel inside the same trained checkpoint. The evaluation uses 22 drugs, 8 of which have mapped evidence, under held-out-drug and scaffold-disjoint CV. It includes seed-matched comparisons, harm counts at four thresholds, LODO ranges, corruption tests on a fixed checkpoint, and a control that isolates the residual.

Critical strengths

Separating UtrainU_{\text{train}} from UoffU_{\text{off}} is a real contribution, and the paper measures both carefully. Most work on gating auxiliary modalities reports a single number and does not say which of the two it means.

Critical weaknesses

1.The abstract looks like it was completely AI-generated. Abstract is too abstract to read and hard to follow, and it lacks a clear logical flow, which is packed with so many hyphens and adjectives. This also appears in the writing style of main article.

2.The rule only checks whether a drug’s pathway count is above the training maximum. As the authors acknowledge, it does not catch counts below the training minimum or gaps within the range. The name “OOD abstention” therefore overstates the method twice: the rule is not a real OOD test, and the model never abstains, since it just switches the channel off. The paper also cites Mahalanobis-based detection (Lee et al., 2018) and quality-aware dynamic fusion (Zhang et al., 2023) as motivation but compares against neither.

3.Table 7 reports that sorafenib’s mean applied gate is exactly 0.0000 and its UoffU_{\text{off}} is exactly +0.0000, because its density always exceeds τ\tau. So within the checkpoint, the evidence channel contributes nothing to sorafenib.

Questions for authors

I took a look at the AI Use Statement, and pretty much everything was done by AI. Can author illustrates what the human actually did in this paper?

While AI-assisted research is becoming increasingly common, authors remain responsible for ensuring the scientific rigor and quality of their submissions. I encourage the authors to conduct a more thorough evaluation and carefully validate their claims before submission to a venue such as ICLR.

Reviewer_3WGiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies selective use of binding-derived evidence for drug-response prediction. Frozen evidence representations are aggregated into 13 pathway slots and fused through a learned gate, with a hard rule rejecting coverage above the training maximum. Experiments on 22 drugs, eight with mapped evidence, distinguish independently trained no-evidence baselines from fixed-checkpoint ablations and examine missingness-induced reactivation. Mean improvements are small and uncertain in the eight-seed extension, while an isolated-residual control illustrates exact fallback to a frozen base.

Critical strengths
  1. Section 3.4 and Table 7 clearly distinguish the effect of evidence-aware training from the inference-time contribution of the evidence channel. The isolated-residual control also clarifies which baseline evidence suppression preserves.
  2. The paper transparently reports seed sensitivity, unfavorable full-cohort results, and reactivation under partial evidence removal. Drug-level harm summaries and corruption controls reveal limitations that average performance alone would obscure.
Critical weaknesses
  1. The diagnostic contribution is insufficiently established relative to prior negative-transfer and quality-aware fusion work, including TUGDA and QMF. Although the count rule is presented as a simple reference mechanism, showing that partial removal reverses its rejection decision does not establish a conclusion beyond this particular heuristic and cohort.
  2. Validation of the selection mechanism is narrow: only eight drugs have mapped evidence, and Sorafenib is the only supported drug with a zero applied gate in held-out Table 7. The eight-seed extension reports mean utility +0.0009 with a descriptive interval [−0.0011, +0.0036] and a full-cohort macro-AUC delta around −0.0023, without the learned-gate comparator. These results do not establish robust selection gains.
  3. Essential operational definitions remain missing, including the response-data release and binary labels, similarity metric/threshold, evidence weights, pathway mappings, cohort/fold assignments, and KL term. These omissions prevent independent reconstruction; Section 8 states that code and checkpoints are not released, leaving no alternative way to resolve them.
Questions for authors
  1. What conclusion about evidence reliability extends beyond the maximum-coverage rule and prior reliability-aware fusion work?
  2. Please provide per-drug coverage, thresholds, admission decisions, and paired learned-gate/OOD results for both protocols. Can the hard mask be toggled on identical checkpoints with full evidence unchanged, including the eight-seed extension?
  3. Please provide the missing experimental definitions and validation/model-selection procedure, plus non-identifying artifacts sufficient to reconstruct the principal tables.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.