acceptodds
Under review as a conference paper at ICLR 2027

On the Necessity of Non-Additive Modeling for Interaction Prediction: Theory and Application to Drug Synergy

Abstract

Many prediction tasks require combining separately observed inputs, such as drugmolecules and a cell line, imaging modalities, or sensor channels, whose jointeffect cannot be captured by summing independent contributions. Whether inter-action modeling is theoretically necessary in such settings, or merely empiricallyuseful, has remained an open question. We prove a structural necessity theoremfor interaction prediction: any model in the functionally additive class, regard-less of the expressiveness of its per-condition components, suffers an irreducibleprediction gap when the target contains genuine cross-condition interaction terms.We derive an explicit, strictly positive lower bound for this gap that depends onlyon the magnitude of the target’s interaction component, thereby characterizing alimitation of the function class rather than of any particular architecture. We in-stantiate this result in drug synergy prediction, where the standard null modelsof pharmacological interaction are additive by construction, making the neces-sity gap concrete and measurable. We introduce Iψ, a structured bilinear operatorthat computes pairwise drug interactions as a low-rank product of individual com-pound embeddings, instantiating the non-additive class the theorem calls for. Onthe Therapeutics Data Commons (TDC) DrugComb benchmark, Iψcloses thepredicted gap across five synergy scores under a common training and evaluationprotocol, with each model receiving its own hyperparameter sweep. Iψuses abouta quarter of the parameter budget of FiLM, adapted here for drug synergy, andstill exceeds it at its full published size on the combo split. On an independentbenchmark (OncopolyPharmacology) the ordering holds. An ablation confirmsthe gain comes from the bilinear term itself. Hard generalization splits confirmthe theorem’s boundary condition.

Primary area:
learning theory
Keywords:
structural necessity theorem, non-additive models, interaction prediction, drug synergy prediction, bilinear operators

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BMnbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper investigates the necessity of non-additive modeling in multi-input prediction tasks. The authors define the interaction terms of the objective function via a higher-order function decomposition and attempt to prove that, when the objective contains non-zero interaction terms, any functionally additive model has a strictly positive lower bound on the approximation error; under a bounded softmax model, this result further translates to a lower bound on the KL divergence, while under fixed-variance Gaussian regression, it corresponds to a lower bound on the mean squared error. In drug synergy prediction, the authors propose a low-rank bilinear interaction operator IψI_\psi, which combines main effects and pairwise interactions from single-drug and cell line representations. Experiments include synthetic data, DrugComb, OncopolyPharmacology, multiple evaluation metrics, parameter budget comparisons, and ablation studies. On the combination-reserved partition of DrugComb, the proposed model achieved an AUROC of 0.792 with approximately 1.87 million parameters, outperforming the 0.770 AUROC of FiLM, which used 7.20 million parameters; however, on the unseen drug partition, its classification advantage largely disappeared.

Critical strengths

1.The results on parameter efficiency are of practical significance. Tables 1 and 3 show that the proposed model outperforms the full FiLM on DrugComb with approximately one-quarter of the parameters and maintains its advantage under a comparable parameter budget. Each model underwent a separate learning rate search; the main results report five seeds along with AUROC and AUPRC, and these design choices enhance the comparative value of the analysis.

2.The study includes experiments that reveal the method’s scope of applicability. Synthetic data with known interaction strengths serve as suitable test cases; the absence of single-drug and dual-drug classifications directly highlights the method’s generalization limitations. The authors report near-random classification performance on these challenging classifications, which helps readers assess the method’s practical scope of application.

Critical weaknesses
  1. The core proof treats the hierarchical orthogonal decomposition under relevant inputs as a fully orthogonal decomposition; an explicit lower bound has not yet been fully proven.

  2. The empirical results do not yet constitute a direct verification of this theorem. Section 4.1 claims that setting all bilinear terms to zero restores the functional additivity model, but Appendix D.4 explicitly retains the summed GELU and linear prediction heads. In general, head(GELU(∑igi(ci)))\mathrm{head}(\mathrm{GELU}(\sum_i g_i(c_i))) is already capable of generating cross-input interactions and therefore cannot be directly regarded as Fstd\mathcal F^{std}. This also affects the interpretation of Δint\Delta_{\mathrm{int}} as an interaction component in the mathematical sense.

Questions for authors

1.Please provide the complete derivation or precise citation for the lower bound in Appendix B.2 and the exponent 2n−n−22^n-n-2, and explain how to modify the variance identity used in B.4 for the relevant inputs. After the modification, do the form and constant of the lower bound in Theorem 1 remain unchanged?

  1. Does the Additive baseline still use GELU or other nonlinearities after cross-conditional summation? Please provide the complete computational formulas for both this baseline and the version with the bilinear terms removed. If the latter does not belong to Fstd\mathcal F^{std}, how should the relevant theoretical verification and interpretive conclusions be adjusted?

3.Is there a clear derivation linking the KL/MSE lower bounds in the theorem to the AUROC gap under varying thresholds, or to the disappearance of advantages for unseen drugs? If not, do the authors agree to frame these results as empirical observations and supplement them with MSE comparisons across the main difficulty classifications?

4.Please provide the versions, sources, and drug-cell line intersections for the two datasets, as well as details on how replicate experiments were handled. Additionally, please clarify whether the labels for Bliss, ZIP, HSA, and CSS in Table 5 are defined according to their respective scoring thresholds, and whether the bootstrap analysis was grouped by shared drug pairs or repeated measurements.

3 comments
AuthorsOpenReview ↗

We thank Reviewer BMnb for the technically precise critique. Proof corrected (Appendix A). Strict additive baseline added. Dataset details clarified.

W1. We concede this error. The step ∑kΩk(f∗)=Var[f∗]\sum_k \Omega_k(f^{\ast})=\mathrm{Var}[f^{\ast}] invokes mutual orthogonality, invalid under dependent-input HOFD (Chastaing et al. 2012, §2.4). Removed. The corrected proof (Appendix A) uses two steps.

Step 1 (L2L^2 gap). Apply Lemma 1 to T=P([n])T=\mathcal{P}([n]) (hierarchical, ∣T∣=2n|T|=2^n). For any h∈Fstdh\in\mathcal{F}^\mathrm{std}, gu=fu∗g_u=f^{\ast}_u for all ∣u∣≥2|u|\geq 2. Lemma 1 gives ∥f∗−h∥2≥δ2n−1S\|f^{\ast}-h\|^2\geq\delta^{2^n-1}S. Exponent corrected from 2n−n−22^n-n-2 to 2n−1\mathbf{2^n-1} (full derivation in Q1).

Step 2 (L2L^2 to KL). Pinsker: DKL(p∗∥ph)≥12∥f∗−h∥2≥12δ2n−1SD_\mathrm{KL}(p^{\ast}\|p_h)\geq\tfrac{1}{2}\|f^{\ast}-h\|^2\geq\tfrac{1}{2}\delta^{2^n-1}S (general KK). For K=2K=2, Bernoulli-Pinsker (Cover and Thomas 2006, Thm. 11.6.1): DKL(p∗∥ph)≥2∥f∗−h∥2≥2δ2n−1SD_\mathrm{KL}(p^{\ast}\|p_h)\geq 2\|f^{\ast}-h\|^2\geq 2\delta^{2^n-1}S, so Δ=2δ2n−1S\Delta=2\delta^{2^n-1}S. Condition (C.3) removed.

W2. We concede both points.

GELU issue. Setting Bij=0B_{ij}=0 leaves ρ(∑iWiϕi(ci))\rho(\sum_i W_i\phi_i(c_i)), a GELU on the aggregated sum, not strictly in Fstd\mathcal{F}^\mathrm{std}. Corrected in the revised paper.

Δint\Delta_\mathrm{int} issue. Δint\Delta_\mathrm{int} measures the empirical contribution of the bilinear branch through an ablation-based comparison of model predictions. It is distinct from the theoretical KL gap Δ\Delta established in Theorem 1 and should not be interpreted as a mathematical interaction component in the HOFD sense. Reframed as "empirical interaction contribution."

Strict Fstd\mathcal{F}^\mathrm{std} baseline (new). y^=∑ihi(ci)\hat{y}=\sum_i h_i(c_i): independent MLPs per condition, no post-summation nonlinearity. By construction all HOFD components of order ≥2\geq 2 are zero. Synthetic validation (y=x1x2y=x_1 x_2): strict MSE stays at 0.83≈Var(x1x2)0.83\approx\mathrm{Var}(x_1 x_2); IψI_\psi reaches MSE <0.01<0.01.

DrugComb, MSE-trained, combo split (5 seeds, Tab. 10):

ModelAUROCPearson
Strict Fstd\mathcal{F}^\mathrm{std} baseline0.6440.348
IψI_\psi (ours)0.7920.707
Gain+0.148+0.360

Strict baseline added to all main tables (Tab. 2, 4, 5, 7).

Q1. Origin of 2n−n−22^n-n-2. The original proof applied Lemma 1 to T={u:∣u∣≥2}T=\{u:|u|\geq 2\}, ∣T∣=2n−n−1|T|=2^n-n-1, giving exponent ∣T∣−1=2n−n−2|T|-1=2^n-n-2. This is invalid: Lemma 1 requires TT to be hierarchical (every subset of every element in TT must also be in TT). The family {∣u∣≥2}\{|u|\geq 2\} omits the singletons {i}\{i\}, which are subsets of every element of size 2; it is not hierarchical.

Correction. Use T=P([n])T=\mathcal{P}([n]), trivially hierarchical, ∣T∣=2n|T|=2^n. Exponent 2n−12^n-1 (weaker, rigorous). Revised Theorem 1: inf⁡h∈FstdDKL(p∗∥ph)≥2δ2n−1S=:Δ>0\inf_{h\in\mathcal{F}^\mathrm{std}} D_\mathrm{KL}(p^{\ast}\|p_h) \geq 2\delta^{2^n-1}S =: \Delta > 0 (K=2K=2). Constants changed; qualitative message unchanged.

Q2. Neither Additive$^\star$ nor the bilinear ablation (Bij=0B_{ij}=0) is strictly in Fstd\mathcal{F}^\mathrm{std}: both apply a post-aggregation GELU that introduces cross-input terms. The strict baseline y^=∑ihi(ci)\hat{y}=\sum_i h_i(c_i) uses independent per-condition MLPs with no post-summation step; fu∗≡0f^{\ast}_u\equiv 0 for ∣u∣≥2|u|\geq 2 by construction. This baseline, not the ablation or Additive$^\star$, is the correct experimental verification of the theorem.

Q3. No general KL-to-AUROC guarantee follows from Theorem 1 without additional assumptions on score distributions and ranking margins. (i) AUROC is invariant to strictly increasing transformations of prediction scores, whereas KL divergence measures differences between full predictive distributions. (ii) Pinsker provides a lower bound on KL in terms of squared probability differences; this direction does not yield a general bound on ranking error (AUROC).

The empirical analogue of the theorem's split-level prediction (Tab. 13, IψI_\psi minus strict; combo: 5 seeds, drug-single: 3 seeds):

SplitMSEBCE τ=0\tau=0BCE τ=10\tau=10
combo+0.148+0.113+0.051
drug-single+0.045+0.028-0.008

The gap is smaller on drug-single, consistent with reduced interaction signal; SS not directly measured, not a direct consequence of Theorem 1. Pearson on combo: +0.360 (Tab. 10).

Q4. Versions. DrugComb: Zagidullin et al. (2019), TDC DrugComb, PyTDC 1.1.15. OncopolyPharmacology: O'Neil et al. (2016), TDC ONEIL, same version.

Intersection. The 37 drugs and 39 cell lines of OncopolyPharmacology do not overlap with DrugComb. No cross-dataset evaluation.

Replicates. DrugComb aggregates replicates to one score per (Drug1, Drug2, CellLine) tuple; we load TDC pickles directly.

Labels (Tab. 5). Each score type (Loewe, Bliss, ZIP, HSA, CSS) uses threshold >10>10 for its own positive class, not tuned. For the main results (Tab. 2 and 4), positives are defined by Loewe~>10>10.

Bootstrap. Resamples individual (drug, drug, cell line) triplet rows with replacement. 2000 iterations; the same sampled row indices are applied to both compared models at each iteration. App. D.

Thank you for your rebuttal. I think it definitely strengthensthe paper. However, the paper concernstill remains (e.g. lack the main figure). I had to keep my original score.

AuthorsOpenReview ↗

Addition of the Main Figure to the Revised Manuscript

Thank you for your feedback.

We have added a new main figure to the revised manuscript to provide a clearer overview of the proposed method and its key components. Consequently, the previous Figure 1 has been renumbered as Figure 2, and the subsequent figures have been renumbered accordingly. We have updated the corresponding references throughout the manuscript. We hope this addition addresses your concern and improves the clarity of the paper.

Reviewer_zVXqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that additive predictors incur an irreducible approximation error when the target contains genuine interactions, and proposes a low-rank bilinear model for drug synergy prediction. Experiments show improved performance and parameter efficiency over several baselines. However, the core theoretical insight is largely a consequence of classical projection results, the dependent-input proof raises correctness concerns, and the theory provides limited justification for the specific architecture, weakening the claimed novelty.

Critical strengths
  1. The paper addresses a meaningful question about the limitations of additive models, motivated by drug synergy prediction.

  2. The proposed bilinear model improves over the evaluated baselines on DrugComb while using substantially fewer parameters than the full-size FiLM baseline.

Critical weaknesses

1.The paper states that the input variables are correlated. However, in practical applications, the HOFD components are treated as fully orthogonal vectors.

2.Practical generalization and methodological novelty are limited, and the proposed method fails to show advantages on unseen drugs.

3.Theory provides weak guidance for methods. At best, theorems only tell us "do not restrict the model to be purely additive", but cannot answer why bilinear fusion is chosen instead of other fusion approaches.

Questions for authors
  1. Theoretical correctness. The dependent-input proof appears to treat hierarchically orthogonal HOFD components as mutually orthogonal, particularly when equating the sum of component variances with the total variance. Can the authors establish the claimed lower bound while accounting for covariance terms?
  2. Theoretical novelty. For independent inputs, the approximation gap follows directly from classical orthogonal projection arguments. What substantive new result does the theorem provide beyond existing functional ANOVA theory?
  3. Consistency of the additive restriction. Removing the bilinear terms leaves a nonlinear transformation of summed representations, which is generally non-additive. How do the authors reconcile this architecture with their claim that removing those terms recovers the additive function class?
3 comments
AuthorsOpenReview ↗

We thank Reviewer zVXq for the precise identification of the proof issues. Proof corrected (Appendix A). Strict additive baseline added to all main tables.

W1. We concede. Two errors corrected in Appendix A. Under dependent-input HOFD (Chastaing et al. 2012), components are hierarchically but not mutually orthogonal.

Error 1. ∑kΩk(f∗)=Var[f∗]\sum_k \Omega_k(f^{\ast}) = \mathrm{Var}[f^{\ast}] invokes mutual orthogonality, invalid under dependent HOFD. Removed.

Error 2. Lemma 1 was applied to T={u:∣u∣≥2}T = \{u : |u| \geq 2\}, not hierarchical (omits singletons {i}\{i\}). Corrected to T=P([n])T = \mathcal{P}([n]).

Corrected proof. Apply Lemma 1 to T=P([n])T = \mathcal{P}([n]) (∣T∣=2n|T| = 2^n). For any h∈Fstdh \in \mathcal{F}^\mathrm{std}, hu=0h_u = 0 for ∣u∣≥2|u| \geq 2, so gu=fu∗g_u = f_u^{\ast}. Lemma 1 gives ∥f∗−h∥2≥δ2n−1S\|f^{\ast} - h\|^2 \geq \delta^{2^n-1} S; no covariance terms appear. Exponent corrected to 2n−1\mathbf{2^n-1}. Bernoulli-Pinsker (Cover and Thomas 2006, Thm. 11.6.1): inf⁡h∈FstdDKL(p∗∥ph)≥2δ2n−1S=:Δ>0\inf_{h\in\mathcal{F}^\mathrm{std}} D_\mathrm{KL}(p^{\ast}\|p_h) \geq 2\delta^{2^n-1}S =: \Delta > 0. Condition (C.3) removed. Δ>0\Delta > 0 whenever S>0S > 0 fully preserved. Full proof in Appendix A.

W2. We concede the drug-out gap. Scope: The combo split (new combinations of known drugs) and drug-out split (entirely unseen drugs) test different tasks. Theorem 1 concerns the combo regime; it makes no claim about extrapolation to unseen entity representations.

Controlled synthetic evaluation (Table 6, Appendix C.6). Target yAB=xA⊤B∗xBy_{AB} = x_A^\top B^{\ast} x_B (rank-4), features always in-distribution, 10 populations ×\times 3 seeds:

Modelpair_heldone_entityboth_entityfresh_entity
StrictAdditive−0.09±0.13-0.09 \pm 0.13+0.08±0.16+0.08 \pm 0.16+0.04±0.26+0.04 \pm 0.26−0.09±0.04-0.09 \pm 0.04
ConcatMLP+0.99±0.00+0.99 \pm 0.00+0.59±0.22+0.59 \pm 0.22+0.15±0.40+0.15 \pm 0.40+0.14±0.21+0.14 \pm 0.21
IψI_\psi+0.99±0.02+0.99 \pm 0.02+0.90±0.06+0.90 \pm 0.06+0.80±0.09\mathbf{+0.80 \pm 0.09}+0.79±0.04\mathbf{+0.79 \pm 0.04}

Both IψI_\psi and ConcatMLP fit pair_held (R2≈0.99R^2 \approx 0.99), confirming ConcatMLP can learn the interaction. IψI_\psi obtains R2=0.79±0.04R^2 = 0.79 \pm 0.04 on fresh_entity (parameters frozen after training): transferable interaction rule, not memorization. ConcatMLP-256 (4×4\times wider) reaches +0.45±0.07+0.45 \pm 0.07 (Table 15), still +0.34+0.34 below IψI_\psi. ConcatMLP-64 R2R^2 ranges from −0.89-0.89 to +0.57+0.57 across populations (Table 16); IψI_\psi stays 0.79±0.040.79 \pm 0.04. Real drug-out gaps may additionally reflect feature-space shift from scaffold diversity not reproduced synthetically.

W3. The theorem identifies Ω≥2(f∗)\Omega_{\geq 2}(f^{\ast}) as the irreducible source of error; escaping Fstd\mathcal{F}^\mathrm{std} is necessary but not sufficient. ConcatMLP escapes Fstd\mathcal{F}^\mathrm{std} yet gains only +0.006+0.006 AUROC over the strict baseline (0.650 vs 0.644, Table 2), whereas IψI_\psi gains +0.148+0.148 (0.792 vs 0.644). The bilinear term ϕA⊤B ϕB\phi_A^\top B\,\phi_B provides an explicit, compact parameterization through reusable per-entity representations, an inductive bias absent from generic joint fusion; the gap is structural, not a capacity artefact (capacity sweep: Table 15; entity generalization: Table 6). Concretely, each Bij(ϕi,ϕj)B_{ij}(\phi_i, \phi_j) explicitly targets the second-order HOFD cross-term f{i,j}∗f_{\{i,j\}}^{\ast}, the terms SS quantifies and Fstd\mathcal{F}^\mathrm{std} cannot represent by construction (Sec. 3).

Q1. Yes; see W1 and Appendix A. Proof uses only Lemma 1 of Chastaing et al. (hierarchical orthogonality) and Bernoulli-Pinsker; no covariance terms E[fu∗fv∗]\mathbb{E}[f^{\ast}_u f^{\ast}_v] appear. Δ=2δ2n−1S\Delta = 2\delta^{2^n-1}S (K=2K=2). Δ>0\Delta > 0 whenever S>0S > 0 fully preserved.

Q2. Theorem 1 extends the classical L2L^2 gap in four ways. (i) KL gap: establishes an explicit KL lower bound on the full predictive distribution, not just the conditional mean (App. A, Step 2). (ii) Dependent inputs: uses Chastaing et al. (2012) HOFD under (C.1)--(C.2); classical ANOVA assumes independence. Drug co-administration induces correlated molecular representations; this extension is not merely theoretical (Sec. 2). (iii) Quantitative bound: inf⁡DKL≥2δ2n−1S\inf D_\mathrm{KL} \geq 2\delta^{2^n-1}S (K=2K=2); empirically, the AUROC gain collapses on drug-single splits (Table 1). (iv) Architectural implication: Ω≥2(f∗)\Omega_{\geq 2}(f^{\ast}) identifies which terms must be modeled; does not prescribe a unique architecture (see W3).

Q3. We concede. Setting Bij=0B_{ij}=0 leaves ρ(∑iWiϕi(ci))\rho(\sum_i W_i\phi_i(c_i)), non-additive due to the post-aggregation GELU; our original claim was incorrect. Strict Fstd\mathcal{F}^\mathrm{std} baseline added: y^=∑ihi(ci)\hat{y}=\sum_i h_i(c_i) (independent per-condition MLPs, no post-summation nonlinearity), included in Tables 2, 4, 5, 7, 10, 11. By construction its HOFD components of order ≥2\geq 2 vanish; on y=x1x2y=x_1 x_2 its MSE stays at 0.83≈Var(x1x2)0.83 \approx \mathrm{Var}(x_1 x_2). IψI_\psi improves AUROC by +0.148+0.148 over this baseline (Table 10). Setting Bij=0B_{ij}=0 is now "bilinear ablation," explicitly not in Fstd\mathcal{F}^\mathrm{std}. Construction in Appendix C.

AuthorsOpenReview ↗

Addition of the Main Figure to the Revised Manuscript

We have added a new main figure to the revised manuscript to provide a clearer overview of the proposed method and its key components. Consequently, the previous Figure 1 has been renumbered as Figure 2, and the subsequent figures have been renumbered accordingly. We have updated the corresponding references throughout the manuscript.

Thank you for the rebuttal. The revised proof and the new strictly additive baseline address my original technical concerns. I appreciate the synthetic generalization results, but the gains on unseen drugs remain limited. I also find the theoretical advance modest and the case for bilinear fusion over other interaction-aware designs limited. I therefore continue to lean toward rejection.

Reviewer_Ub9UOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies the limitations of strictly additive predictors when the target contains interactions. It proposes an error lower bound and a low-rank bilinear model for drug synergy, evaluated on synthetic data and two benchmarks, with additional drug holdout experiments on DrugComb.

Critical strengths
  1. Simple and parameter-efficient design. The low-rank bilinear operator is straightforward. On DrugComb combo, the reported AUROC improves from 0.770 for adapted FiLM to 0.792, using 1.87M rather than 7.20M parameters (Table 1). The matched-budget comparison (Table 3) further supports parameter efficiency within the tested models.

  2. Useful experimental controls and scope assessment. The study includes per-model learning-rate tuning, five seeds for the main experiments, a synthetic test with known interactions, and a second dataset. I also appreciate the reporting of difficult unseen-drug splits, including the weak results, which helps readers assess the method’s applicability.

Critical weaknesses
  1. The central claim provides little new insight for this task. The paper’s main message is that a model restricted to adding separate per-input contributions cannot represent a genuinely non-additive target. Existing Concat–MLP and FiLM predictors already allow these dependencies, as the authors acknowledge. The theorem therefore formalizes a limitation that these models have already avoided. The reported gains demonstrate the utility of a particular fusion design, but do not establish a substantial new understanding of interaction prediction.

  2. An unjustified step in the explicit bound. Appendix B.4 (lines 676–679) uses a variance-additivity identity that the dependent-input HOFD does not generally satisfy: hierarchical orthogonality does not eliminate all covariance terms (Chastaing et al., 2012, §2.4). This leaves the final normalization step unjustified. A corrected bound may preserve the qualitative necessity conclusion. The regression corollary relies on the earlier L2 approximation bound, so this objection specifically concerns the normalization step in B.4.

  3. The claimed theoretical validation is unsupported. In §4.1 and Appendix D.5, removing the bilinear terms still leaves GELU after summation. This model can retain interactions, so calling it the model’s “additive restriction” is not justified. In §5.4, the disappearance of an AUROC advantage on unseen drugs is called validation of the theorem’s boundary. The theorem sets a lower bound on an additive model’s error relative to the true target. The experiment measures the AUROC difference between two trained models. A smaller difference between these models does not show that the theorem’s error bound has shrunk.

Questions for authors
  1. What is the main new theoretical consequence beyond the cited HOFD approximation result, and what does it add to our understanding of the drug-synergy task?
  2. Can Appendix B.4 be corrected to account for dependence, and would this change the stated bound or its claimed implications?
  3. Does “additive restriction” refer to the latent sum before GELU or to the final prediction? How does this relate to the separately trained Additive baseline?
3 comments
AuthorsOpenReview ↗

We thank Reviewer Ub9U for the careful reading. Proof corrected (Appendix A). Strict additive baseline and BCE experiments added.

W1. Concat-MLP and FiLM escape Fstd\mathcal{F}^\mathrm{std}, but escaping is necessary and not sufficient: the theorem proves why.

Escaping Fstd\mathcal{F}^\mathrm{std} is not enough. Concat-MLP escapes Fstd\mathcal{F}^\mathrm{std} by construction, yet gains only +0.006+0.006 AUROC over the strict additive baseline (0.650 vs 0.644, Table 2), while IψI_\psi gains +0.148+0.148 (0.792 vs 0.644). The gap is not about expressiveness but about which terms are missing: Ω≥2(f∗)\Omega_{\geq 2}(f^*) is the irreducible structure. Concat expands capacity but introduces no inductive bias toward Ω≥2\Omega_{\geq 2}; the capacity is absorbed by individual-drug signals. Bilinear coupling computes cA⊤B cBc_A^\top B\, c_B, targeting Ω≥2\Omega_{\geq 2} directly. Whether Concat can represent these terms in principle is not in question; whether it learns them efficiently under standard training is a separate matter the theorem does not decide.

Design criterion. Gap >0>0 when S>0S>0; collapse when S→0S\to 0. Consistent with observations in W3 (MSE, BCE τ=0\tau=0, τ=10\tau=10).

Numerical validation. We verify K(ρ,α)≥Δ(ρ,α)=2δ3SK(\rho,\alpha) \geq \Delta(\rho,\alpha) = 2\delta^3 S by Gauss-Hermite quadrature over α∈[0.2,4]\alpha\in[0.2,4], ρ∈{0,0.3,0.7,0.9}\rho\in\{0,0.3,0.7,0.9\}. The bound holds in all 4 panels (minimum ratio K/Δ=1.013K/\Delta=1.013). For ρ=0\rho=0 it is tight: K/Δ∈[1.013,1.25]K/\Delta \in [1.013, 1.25]; for ρ>0\rho>0 it grows conservative (ratios 11 to 9500). New figure (Figure 1) in revised paper.

W2. Two proof errors corrected. The corrected Theorem 1 and full proof are in Appendix A.

Issue: The step ∑kΩk(f∗)=Var[f∗]\sum_k \Omega_k(f^{\ast}) = \mathrm{Var}[f^{\ast}] uses mutual orthogonality, invalid under dependent-input HOFD (Chastaing et al. 2012, §2.4). Removed.

Corrected proof (2 steps).

Step 1 (L2L^2 gap): Apply Lemma 1 to T=P([n])T = \mathcal{P}([n]) (hierarchical, ∣T∣=2n|T| = 2^n). For any h∈Fstdh \in \mathcal{F}^\mathrm{std}, gu=fu∗g_u = f_u^{\ast} for all ∣u∣≥2|u| \geq 2. Lemma 1 gives ∥f∗−h∥2≥δ2n−1S\Vert f^{\ast} - h \Vert^2 \geq \delta^{2^n-1} S. Exponent corrected from 2n−n−22^n - n - 2 to 2n−1\mathbf{2^n - 1} (weaker but rigorous).

Step 2 (L2L^2 to KL): Pinsker gives DKL(p∗ ∥ ph)≥12δ2n−1SD_\mathrm{KL}(p^{\ast} \,\|\, p_h) \geq \tfrac{1}{2}\delta^{2^n-1}S (general KK). For K=2K=2, Bernoulli-Pinsker (Cover & Thomas 2006, Thm. 11.6.1) gives Δ=2δ2n−1S\Delta = 2\delta^{2^n-1}S. Condition (C.3) removed.

W3. Additive restriction: We concede. Setting Bij=0B_{ij}=0 leaves a GELU after summation; this model is not in Fstd\mathcal{F}^\mathrm{std}. Corrected in the revised paper.

Strict Fstd\mathcal{F}^\mathrm{std} baseline: y^=∑ihi(ci)\hat{y} = \sum_i h_i(c_i), independent MLPs, no post-summation nonlinearity. Trained on y=x1x2y = x_1 x_2: MSE stays at 0.83≈Var(x1x2)0.83 \approx \mathrm{Var}(x_1 x_2); IψI_\psi reaches MSE <0.01<0.01.

DrugComb, MSE-trained, combo split (5 seeds, Table 10):

ModelAUROCPearson
Strict Fstd\mathcal{F}^\mathrm{std} baseline0.6440.348
IψI_\psi (ours)0.7920.707
Gain+0.148+0.360

BCE classification. Minimizing BCE ≡\equiv minimizing DKL(p∗∥ph)D_\mathrm{KL}(p^{\ast}\|p_h) (they differ by H(p∗)H(p^{\ast}), constant in hh): same probability space as Theorem 1. Labels at τ=0\tau=0 and τ=10\tau=10 (field-standard threshold). Same protocol, same seeds.

DrugComb, BCE, combo split (5 seeds, Table 11):

ModelMSE-AUROCBCE τ=0\tau=0BCE τ=10\tau=10
Strict Fstd\mathcal{F}^\mathrm{std} baseline0.6440.6500.826
IψI_\psi (ours)0.7920.7630.877
Gain+0.148+0.113+0.051

Drug-single (Table 13): The gain collapses across all objectives under drug-single splits, consistent with reduced exploitable interaction signal:

SettingGain (combo)Gain (drug-single)
MSE+0.148+0.045
BCE τ=0\tau=0+0.113+0.028
BCE τ=10\tau=10+0.051-0.008

Q1. HOFD (Chastaing et al.) gives an L2L^2 gap: inf⁡h∈Fstd∥f∗−h∥2≥δ2n−1S\inf_{h\in\mathcal{F}^\mathrm{std}}\|f^*-h\|^2 \geq \delta^{2^n-1}S. Theorem 1 adds a KL gap via Pinsker: no additive model assigns correct synergy probabilities, not merely a wrong point prediction, regardless of capacity (doubling width: +0.001+0.001 AUROC, Table 17). The bilinear cA⊤B cBc_A^\top B\,c_B in IψI_\psi explicitly parameterizes cross-entity interactions, the terms the theorem identifies as the irreducible source of the gap.

Q2. Yes: both issues corrected (see W2). Revised: Δ=12δ2n−1S\Delta = \tfrac{1}{2}\delta^{2^n-1}S (general KK), Δ=2δ2n−1S\Delta = 2\delta^{2^n-1}S (K=2K=2). Condition (C.3) removed. Qualitative implication Δ>0\Delta > 0 whenever S>0S > 0 fully preserved.

Q3. "Additive restriction" = Bij=0B_{ij}=0 in IψI_\psi: remaining model ρ(∑iWiϕi)\rho(\sum_i W_i\phi_i), additive in latent sum before ρ\rho, not in the output. Additive$^\star$ has the same structure (nonlinear head on aggregated sum); neither is strictly in Fstd\mathcal{F}^\mathrm{std}. We added a strict Fstd\mathcal{F}^\mathrm{std} baseline (y^=∑ihi(ci)\hat{y}=\sum_i h_i(c_i)) in all main tables (Tables 2, 4, 5, 7); construction in W3.

AuthorsOpenReview ↗

Addition of the Main Figure to the Revised Manuscript

We have added a new main figure to the revised manuscript to provide a clearer overview of the proposed method and its key components. Consequently, the previous Figure 1 has been renumbered as Figure 2, and the subsequent figures have been renumbered accordingly. We have updated the corresponding references throughout the manuscript.

Thank you for your rebuttal. I appreciate the clarifications and the additional experiments. However, my main concern about the novelty of the contribution has not been resolved, and the new analyses do not change my overall assessment. I have to keep my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.