- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies negative transfer from external drug–protein binding evidence to cell-line drug-response prediction. The authors construct a frozen evidence branch from BindingDB-supervised drug–protein representations, aggregate the evidence into 13 pathway slots, and introduce a selective fusion mechanism. Its principal reliability rule permits evidence for a test drug only if its number of covered pathways is positive and does not exceed the maximum coverage observed among supported training drugs. The paper further distinguishes (i) performance relative to an independently trained no-evidence model from (ii) the inference-time effect of disabling evidence within a fixed checkpoint. Experiments on 22 drugs, eight of which have mapped evidence, include held-out-drug and scaffold-disjoint splits, per-drug harm statistics, seed sensitivity, and inference-time corruptions.
The central question is important, and the paper is unusually transparent about negative and inconclusive findings. Nevertheless, I am not yet convinced that the proposed mechanism constitutes a sufficiently novel or validated reliability-aware transfer method for ICLR. The main gate is a simple one-sided pathway-count threshold; the apparent benefit may be largely attributable to suppressing evidence for a single outlying drug; and the larger-seed experiments do not establish consistent performance gains.
Critical strengths
-
The paper appropriately challenges the assumption that available external biological evidence necessarily improves downstream predictions. Reporting per-drug negative transfer is more informative than reporting aggregate AUC alone.
-
Matched seeds, several harm thresholds, per-drug results, conditional worst-drug sensitivity, and fixed-checkpoint corruptions are valuable. The Sorafenib partial-removal case is particularly informative: reducing pathway coverage can turn a rejected evidence source into an admitted, harmful one.
Critical weaknesses
-
The proposed admission rule simply rejects evidence when its pathway count exceeds the maximum observed in training. While interpretable, this heuristic does not estimate evidence quality or predictive utility. Related work on quality-aware fusion and negative-transfer prevention already exists, including Provable Dynamic Fusion (ICML 2023), Residual Feature Integration (ICLR 2026), and confidence-aware multimodal fusion for drug–target prediction (AAAI 2026). The authors should clarify what is methodologically new beyond these approaches, while acknowledging their different tasks and guarantees.
-
All supported training drugs satisfy the maximum-coverage rule by construction, so the hard rejection mechanism appears to matter mainly at test time. Table 7 shows a zero applied gate only for Sorafenib among the eight supported drugs, raising the concern that the improvement over the ordinary learned gate is driven largely by one compound. Please report fold- and seed-level thresholds, test-drug coverage, suppression decisions, and per-drug AUC differences against the learned-gate control. Also clarify whether the two methods produce identical predictions on non-suppressed drugs, and compare with matched-rate random rejection or alternative simple thresholds.
-
Only eight of the 22 drugs have mapped positive evidence, despite the large number of cell-line response rows. This is insufficient to support broad claims about drug-level reliability, gate–utility relationships, or worst-drug robustness. More random seeds and overlapping cross-validation folds cannot replace evaluation on a larger, independently held-out set of drugs.
-
Pathway count measures evidence coverage, not biological correctness or usefulness for a particular cell line. Sparse but incorrect evidence may pass the rule, while valid high-coverage evidence may be rejected. Although the paper includes pathway- and drug-shuffling tests, stronger controls should preserve coverage while introducing incorrect drug–target links or biologically mismatched pathway evidence. Comparing the count threshold with one selected using training-side validation utility would also help determine whether coverage actually predicts negative-transfer risk.
-
The main evaluation transfers binding-derived evidence into cell-line drug-response prediction, rather than demonstrating reliable transfer to patient outcomes. As acknowledged by the authors, only 186 of 338 patient rows can be linked to the stated TCGA cohort, leaving patient-level conclusions unsupported. The paper should narrow its cross-scale and practical-reliability claims or provide a substantially stronger patient-level evaluation.
Questions for authors
-
How many unique supported test drugs are ever rejected by the coverage rule?
-
Are OOD abstention and the ordinary learned-gate baseline training-equivalent?
-
How does the proposed rule compare against stronger, label-blind reliability estimators?
-
What part of the auxiliary evidence is biologically informative?
-
How sensitive is the conclusion to dataset construction and external evidence provenance?