- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper investigates the necessity of non-additive modeling in multi-input prediction tasks. The authors define the interaction terms of the objective function via a higher-order function decomposition and attempt to prove that, when the objective contains non-zero interaction terms, any functionally additive model has a strictly positive lower bound on the approximation error; under a bounded softmax model, this result further translates to a lower bound on the KL divergence, while under fixed-variance Gaussian regression, it corresponds to a lower bound on the mean squared error. In drug synergy prediction, the authors propose a low-rank bilinear interaction operator , which combines main effects and pairwise interactions from single-drug and cell line representations. Experiments include synthetic data, DrugComb, OncopolyPharmacology, multiple evaluation metrics, parameter budget comparisons, and ablation studies. On the combination-reserved partition of DrugComb, the proposed model achieved an AUROC of 0.792 with approximately 1.87 million parameters, outperforming the 0.770 AUROC of FiLM, which used 7.20 million parameters; however, on the unseen drug partition, its classification advantage largely disappeared.
Critical strengths
1.The results on parameter efficiency are of practical significance. Tables 1 and 3 show that the proposed model outperforms the full FiLM on DrugComb with approximately one-quarter of the parameters and maintains its advantage under a comparable parameter budget. Each model underwent a separate learning rate search; the main results report five seeds along with AUROC and AUPRC, and these design choices enhance the comparative value of the analysis.
2.The study includes experiments that reveal the method’s scope of applicability. Synthetic data with known interaction strengths serve as suitable test cases; the absence of single-drug and dual-drug classifications directly highlights the method’s generalization limitations. The authors report near-random classification performance on these challenging classifications, which helps readers assess the method’s practical scope of application.
Critical weaknesses
-
The core proof treats the hierarchical orthogonal decomposition under relevant inputs as a fully orthogonal decomposition; an explicit lower bound has not yet been fully proven.
-
The empirical results do not yet constitute a direct verification of this theorem. Section 4.1 claims that setting all bilinear terms to zero restores the functional additivity model, but Appendix D.4 explicitly retains the summed GELU and linear prediction heads. In general, is already capable of generating cross-input interactions and therefore cannot be directly regarded as . This also affects the interpretation of as an interaction component in the mathematical sense.
Questions for authors
1.Please provide the complete derivation or precise citation for the lower bound in Appendix B.2 and the exponent , and explain how to modify the variance identity used in B.4 for the relevant inputs. After the modification, do the form and constant of the lower bound in Theorem 1 remain unchanged?
- Does the Additive baseline still use GELU or other nonlinearities after cross-conditional summation? Please provide the complete computational formulas for both this baseline and the version with the bilinear terms removed. If the latter does not belong to , how should the relevant theoretical verification and interpretive conclusions be adjusted?
3.Is there a clear derivation linking the KL/MSE lower bounds in the theorem to the AUROC gap under varying thresholds, or to the disappearance of advantages for unseen drugs? If not, do the authors agree to frame these results as empirical observations and supplement them with MSE comparisons across the main difficulty classifications?
4.Please provide the versions, sources, and drug-cell line intersections for the two datasets, as well as details on how replicate experiments were handled. Additionally, please clarify whether the labels for Bliss, ZIP, HSA, and CSS in Table 5 are defined according to their respective scoring thresholds, and whether the bootstrap analysis was grouped by shared drug pairs or repeated measurements.
3 comments
Thank you for your rebuttal. I think it definitely strengthensthe paper. However, the paper concernstill remains (e.g. lack the main figure). I had to keep my original score.
Addition of the Main Figure to the Revised Manuscript
Thank you for your feedback.
We have added a new main figure to the revised manuscript to provide a clearer overview of the proposed method and its key components. Consequently, the previous Figure 1 has been renumbered as Figure 2, and the subsequent figures have been renumbered accordingly. We have updated the corresponding references throughout the manuscript. We hope this addition addresses your concern and improves the clarity of the paper.
We thank Reviewer BMnb for the technically precise critique. Proof corrected (Appendix A). Strict additive baseline added. Dataset details clarified.
W1. We concede this error. The step ∑kΩk(f∗)=Var[f∗] invokes mutual orthogonality, invalid under dependent-input HOFD (Chastaing et al. 2012, §2.4). Removed. The corrected proof (Appendix A) uses two steps.
Step 1 (L2 gap). Apply Lemma 1 to T=P([n]) (hierarchical, ∣T∣=2n). For any h∈Fstd, gu=fu∗ for all ∣u∣≥2. Lemma 1 gives ∥f∗−h∥2≥δ2n−1S. Exponent corrected from 2n−n−2 to 2n−1 (full derivation in Q1).
Step 2 (L2 to KL). Pinsker: DKL(p∗∥ph)≥21∥f∗−h∥2≥21δ2n−1S (general K). For K=2, Bernoulli-Pinsker (Cover and Thomas 2006, Thm. 11.6.1): DKL(p∗∥ph)≥2∥f∗−h∥2≥2δ2n−1S, so Δ=2δ2n−1S. Condition (C.3) removed.
W2. We concede both points.
GELU issue. Setting Bij=0 leaves ρ(∑iWiϕi(ci)), a GELU on the aggregated sum, not strictly in Fstd. Corrected in the revised paper.
Δint issue. Δint measures the empirical contribution of the bilinear branch through an ablation-based comparison of model predictions. It is distinct from the theoretical KL gap Δ established in Theorem 1 and should not be interpreted as a mathematical interaction component in the HOFD sense. Reframed as "empirical interaction contribution."
Strict Fstd baseline (new). y^=∑ihi(ci): independent MLPs per condition, no post-summation nonlinearity. By construction all HOFD components of order ≥2 are zero. Synthetic validation (y=x1x2): strict MSE stays at 0.83≈Var(x1x2); Iψ reaches MSE <0.01.
DrugComb, MSE-trained, combo split (5 seeds, Tab. 10):
Strict baseline added to all main tables (Tab. 2, 4, 5, 7).
Q1. Origin of 2n−n−2. The original proof applied Lemma 1 to T={u:∣u∣≥2}, ∣T∣=2n−n−1, giving exponent ∣T∣−1=2n−n−2. This is invalid: Lemma 1 requires T to be hierarchical (every subset of every element in T must also be in T). The family {∣u∣≥2} omits the singletons {i}, which are subsets of every element of size 2; it is not hierarchical.
Correction. Use T=P([n]), trivially hierarchical, ∣T∣=2n. Exponent 2n−1 (weaker, rigorous). Revised Theorem 1: infh∈FstdDKL(p∗∥ph)≥2δ2n−1S=:Δ>0 (K=2). Constants changed; qualitative message unchanged.
Q2. Neither Additive$^\star$ nor the bilinear ablation (Bij=0) is strictly in Fstd: both apply a post-aggregation GELU that introduces cross-input terms. The strict baseline y^=∑ihi(ci) uses independent per-condition MLPs with no post-summation step; fu∗≡0 for ∣u∣≥2 by construction. This baseline, not the ablation or Additive$^\star$, is the correct experimental verification of the theorem.
Q3. No general KL-to-AUROC guarantee follows from Theorem 1 without additional assumptions on score distributions and ranking margins. (i) AUROC is invariant to strictly increasing transformations of prediction scores, whereas KL divergence measures differences between full predictive distributions. (ii) Pinsker provides a lower bound on KL in terms of squared probability differences; this direction does not yield a general bound on ranking error (AUROC).
The empirical analogue of the theorem's split-level prediction (Tab. 13, Iψ minus strict; combo: 5 seeds, drug-single: 3 seeds):
The gap is smaller on drug-single, consistent with reduced interaction signal; S not directly measured, not a direct consequence of Theorem 1. Pearson on combo: +0.360 (Tab. 10).
Q4. Versions. DrugComb: Zagidullin et al. (2019), TDC
DrugComb, PyTDC 1.1.15. OncopolyPharmacology: O'Neil et al. (2016), TDCONEIL, same version.Intersection. The 37 drugs and 39 cell lines of OncopolyPharmacology do not overlap with DrugComb. No cross-dataset evaluation.
Replicates. DrugComb aggregates replicates to one score per (Drug1, Drug2, CellLine) tuple; we load TDC pickles directly.
Labels (Tab. 5). Each score type (Loewe, Bliss, ZIP, HSA, CSS) uses threshold >10 for its own positive class, not tuned. For the main results (Tab. 2 and 4), positives are defined by Loewe~>10.
Bootstrap. Resamples individual (drug, drug, cell line) triplet rows with replacement. 2000 iterations; the same sampled row indices are applied to both compared models at each iteration. App. D.