acceptodds
Under review as a conference paper at ICLR 2027

CRIQ: Feature Interaction Quantification in Deep Neural Networks with Coordinate-Ray Integrals

Abstract

Higher-order game-theoretic interactions reveal critical neural network behaviors. However, existing black-box approximations suffer from combinatorial subset sampling bottleneck in high dimensions. To address this limitation, we introduce Coordinate-Ray Interaction Quantification (CRIQ), a scalable framework for interaction estimation in deep neural networks that maps discrete derivatives to coordinate-ray integrals. To resolve non-differentiability in continuous piecewise-linear (CPWL) models and bypass high-order tensor differentiation in feature embeddings, we also introduce Iterated Dimensional Reduction (IDR), which decomposes discrete derivatives into an alternating sum of 1D first-order derivative integrals. We derive CRIQ's sample complexity, propose an estimation algorithm and measure its convergence stability empirically, showing that CRIQ makes feature-level interaction quantification in high-dimensional neural networks computationally feasible.

Primary area:
interpretability and explainable AI
Keywords:
Feature Attribution, Feature Interaction, Shapley Interaction, Explainable AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_e5F2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to estimate feature interactions without enumerating all feature subsets. It introduces Coordinate-Ray Interaction Quantification (CRIQ), which uses the Multivariate Fundamental Theorem of Calculus to express discrete derivatives as integrals of mixed partial derivatives under suitable smoothness assumptions. The authors also propose Iterated Dimensionality Reduction (IDR), which rewrites these expressions as alternating sums of one-dimensional integrals of first-order derivatives. This allows the method to handle continuous piecewise-linear networks and multidimensional embeddings without computing higher-order derivative tensors. The paper derives sample-complexity bounds and evaluates estimation accuracy and convergence on a ReLU MLP, DistilBERT, and synthetic unanimity networks.

Critical strengths
  1. Efficient interaction estimation is an important problem. The number of feature subsets grows quickly, making exact computation difficult when many features are involved.

  2. The paper develops the connection between discrete interactions and derivative integrals systematically, with clear definitions, explicit assumptions, and supporting proofs.

Critical weaknesses
  1. Average MSE may hide errors on important interactions in the unanimity dimension-scaling experiment, where many interactions are zero. A normalized error and an all-zero baseline would make the results easier to interpret. Global Spearman correlation also does not directly show whether the strongest interactions are recovered.

  2. A baseline forward pass and a CRIQ backward pass are both counted as one query, but they have different costs. Gradient evaluation generally needs more computation and activation storage. Without runtime and memory results, it is hard to assess CRIQ's practical efficiency.

  3. MLP and DistilBERT are tested with at most 12 input features, while the 30-feature experiment uses a simple synthetic network. These results provide limited evidence for how the method scales to more input features in more complex networks.

Questions for authors
  1. It is helpful to report normalized error and an all-zero baseline for the unanimity dimension-scaling experiment. Top-n recovery by absolute interaction magnitude would also show whether the strongest interactions are identified.

  2. Please compare runtime and peak memory under the same hardware and comparable batching settings. Specify whether one CRIQ query includes the forward computation required to obtain the gradient.

2 comments

We would like to thank the reviewer for their positive assessment and constructive feedback. In our revised manuscript, we upgraded our core estimator from the original 2-point estimator to an amortized 1-point quadrature estimator to enable single-backward-pass gradient reuse across all features, and added the requested metrics, runtime/memory profiles, and higher-dimensional benchmarks.

W1: Average MSE may hide errors on important interactions in the unanimity dimension-scaling experiment, where many interactions are zero. A normalized error and an all-zero baseline would make the results easier to interpret. Global Spearman correlation also does not directly show whether the strongest interactions are recovered.

Q1: It is helpful to report normalized error and an all-zero baseline for the unanimity dimension-scaling experiment. Top-n recovery by absolute interaction magnitude would also show whether the strongest interactions are identified.

Thank you for these helpful suggestions. Following your recommendations, we updated all evaluation plots (Figures 2–3 and Appendix G) as follows:

  • Normalized MSE (NMSE) & Top-10 Recovery: We added NMSE=∑∣T∣=k(I^(T)−I(T))2∑∣T∣=kI(T)2\text{NMSE} = \frac{\sum_{|T|=k} (\hat{I}(T) - I(T))^2}{\sum_{|T|=k} I(T)^2} to normalize error by total ground-truth interaction energy, along with Precision@10 and Jaccard@10 ranked by absolute interaction magnitude ∣I(T)∣|I(T)| (Appendix A.5). On the n=20n=20 Unanimity benchmark (Figure 2) , CRIQ rapidly recovers top interactions alongside SPEX while generally outperforming SHAP-IQ and SVARM-IQ. Top-10 recovery remains challenging across most methods at low budgets due to the (252)\binom{25}{2} candidate pairs, consistent with the rank-correlation trends, but generally consistently improves with increasing budget.
  • All-Zero Baseline: We plotted the all-zero baseline as a dashed horizontal reference line in all MSE panels ((a) in all figures), clearly showing that CRIQ remains at or below the all-zero baseline from the earliest runtime budgets while rank-based metrics improve.

W2: A baseline forward pass and a CRIQ backward pass are both counted as one query, but they have different costs. Gradient evaluation generally needs more computation and activation storage. Without runtime and memory results, it is hard to assess CRIQ's practical efficiency.

Q2: Please compare runtime and peak memory under the same hardware and comparable batching settings. Specify whether one CRIQ query includes the forward computation required to obtain the gradient.

Thank you for highlighting the asymmetry between forward and backward passes and the need to evaluate memory overhead. We confirm that one CRIQ sample/query includes both the forward pass and the backward pass, and we updated our benchmarks (Appendix A.6) to reflect this:

  • Wall-Clock Runtime & Peak Memory Footprint Benchmarks: Rather than comparing methods by raw query count QQ, we now plot all metrics directly against wall-clock evaluation time (in seconds) on the x-axis and include dedicated Peak Memory Footprint (MB) panels (f in all figures), measuring host-side algorithm memory. We also report peak allocated GPU VRAM and per-batch GPU runtimes for forward vs. backward passes across all architectures in Appendix A.6.
  • Even after accounting for the per-step cost of backpropagation, our 1-point amortized estimator achieves faster convergence per second in higher dimensions. Moreover, while GPU VRAM and runtime scales predictably with batch size and remains well within standard limits (∼1 GB\sim 1\text{ GB} even on DistilBERT at n=25n=25), host-side algorithm memory cannot be reduced via batching—where SPEX spikes up to 120 MB120\text{ MB} in k=3k=3 for long sentence benchmark ($n=25) while CRIQ stays compact near the other methods, demonstrating CRIQ's practical effectiveness.

W3: MLP and DistilBERT are tested with at most 12 input features, while the 30-feature experiment uses a simple synthetic network. These results provide limited evidence for how the method scales to more input features in more complex networks.

To provide stronger evidence of quantitative scalability on a non-synthetic deep network with larger feature counts, we added a Long-Sentence NLP Benchmark on DistilBERT evaluated on 20 real SST-2 sentences of length n=25n=25 (Figures 13~15 in Appendix G for SII). CRIQ avoids the severe initial variance of black-box methods and delivers stable MSE/NMSE and steadily increasing rank-based metrics across runtime.

Thank the authors for the detailed replies and additional experiments. They largely address my concerns, so I will retain my positive rating. Good luck.

Reviewer_f9qzOpenReview ↗
Rating
1
Confidence
3(+1)
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The papers consider the problem of estimating the interaction index in deep learning. The authors argue that prior methods often treat DL models as black boxes when estimating the interaction index, leading to sample variance scaling with feature dimension. To address this issue, practitioners often use other heuristic methods, which again introduce inductive biases.

Instead, the authors propose leveraging DL model gradients to estimate the interaction index, introducing Coordinate-Ray Interaction Quantification (CRIQ). Essentially, the methods describe the discrete feature interactions as integrals of partial derivatives. For piecewise linear NNs and vector-valued embeddings, it introduces dimensional reduction (IDR), rewriting these quantities using first-order derivative integrals. The authors then provide some theoretical guarantees for the proposed methods as well as some experiments.

Critical strengths

The paper’s motivation is sound.

Critical weaknesses

I have many concerns with this draft.

  1. I think that the presentation of this paper is not good. I have a hard time to understand what is happening in this paper though. I think the paper presentation must be substantially improved to be publishable. Here are some suggestions: a. In the introduction's first paragraph, the authors jumped into feature attribution in interpreting model predictions too quickly. At least elaborate on what you are talking about, maybe giving an example to give the audience a taste.

    b. In the third paragraph in the intro, the authors suddenly jump into introducing CRIQ, mixed partial derivatives, etc without sufficient connection to the second paragraph. This makes things very hard to understand. I think the authors should elaborate first:

     i. What additional information can we exploit, instead of considering NNs as black boxes? Mention the grad here.
    
     ii. Why should grads tell us about interactions?
    
     iii. How does this recover the desired discrete quantity?
    

    After that, we will try to go to the key idea of this paper. There are many other issues across other sections of this paper, which I cannot list them all here.

  2. There are many gaps between the theoretical statements and the methodology proposed by this paper. For example:

    a. Props 1, 2, though they establish exact identities expressing discrete interactions through derivative integrals, do NOT show that replacing these integrals with endpoint evaluations is accurate.

    b. Prop 3 provides a guarantee for the unbiased estimator, but does not actually provide the guarantee for the implemented estimator.

  3. Experiments do not support the key claims of this paper: for example, the learned-model dim-scaling experiments only use 8-12 features, the NLP eval uses five specified sentences, and there are no experiments with the interaction order greater than 3. This is not sufficient for me, as the main claim of this paper is to deal with the high feature dimension.

  4. There is an inconsistency between the pseudo-code and the amortized estimator

  5. I do not see the code for this paper provided in the supplementary material (or a link to an anonymous repo). Since there are many things about this paper that concern me, I need to verify the real implementation of this paper before evaluating it further.

Given the above issues, I unfortunately have to recommend a rejection for this draft. I will wait till the rebuttal before assessing it further.

Questions for authors

See Weaknesses

4 comments

We would like to thank the reviewer for the detailed critique, which helped us identify critical gaps in our presentation, theory, and experiments. In our revised manuscript, we upgraded our core estimator from the original 2-point estimator to an amortized 1-point quadrature estimator with formal error bounds, expanded our experimental scale, and aligned our pseudocode with our implementation.

W1: I think that the presentation of this paper is not good. I have a hard time to understand what is happening in this paper though. I think the paper presentation must be substantially improved to be publishable.

Following your suggestions, we substantially rewrote Section 1 (Introduction) and updated the illustrative figure:

  • Gradual Introduction & Concrete Example: In the first paragraph, we introduce cooperative game-theoretic feature attribution step-by-step, first explaining individual feature contributions (e.g., an individual word in sentiment classification) and then motivating pairwise and higher-order interactions using a concrete NLP example: "not" and "bad" carry negative sentiment individually, but jointly evaluate to neutral-to-positive sentiment.
  • Connecting Neural Network Gradients to Discrete Interactions: In the third paragraph, we explicitly bridge black-box methods and our gradient-based formulation in three steps:
    1. Exploiting Differentiability: Unlike black-box approaches that only query scalar model outputs f(ZS)f(Z_S), deep neural networks can provide continuous input gradients ∇xf(x)\nabla_x f(x) through automatic differentiation.
    2. Why Gradients Capture Interactions: Mixed partial derivatives (∂kf∏j∈T∂zj\frac{\partial^k f}{\prod_{j \in T} \partial z_j}) measure how one feature's sensitivity changes when other features vary, directly capturing synergies and redundancies across feature contexts.
    3. Recovering Exact Discrete Quantities: By the Multivariate Fundamental Theorem of Calculus (MFTC), integrating these mixed partial derivatives over an axis-aligned hyper-rectangle recovers the exact discrete derivative ΔTf(ZS)\Delta_T f(Z_S) used in game-theoretic interaction indices.

We also updated Figure 1 (illustrative figure) to visually illustrate how a network's input space partitions into polytopes, how the exact interaction equals the integral of gradients along a coordinate ray, and how our 1-point estimator approximates this integral with bounded Taylor bias.

W2: There are many gaps between the theoretical statements and the methodology proposed by this paper.

We agree that exact integral identities alone do not justify replacing integrals with endpoint evaluations, and that theoretical guarantees must match the actual implemented algorithm. We closed both gaps in Section 4.3 and Appendix F:

  • 2a (Explicit Bias Bounds for Endpoint Evaluation):
    • Proposition 3 (proof in Appendix F.1) derives an explicit upper bound on the approximation bias introduced when replacing the 1D coordinate-ray integral with its base endpoint evaluation (ξ=0\xi = 0). Using a first-order Taylor expansion along the coordinate ray, we prove that for C2C^2 networks, the bias of the 1-point estimator I1P(T)I_{1P}(T) relative to the true Probabilistic Interaction Index IPII(T)I_{PII}(T) is bounded by 2k−2Dmax⁡2M22^{k-2} D_{\max}^2 M_2, where Dmax⁡D_{\max} is the maximum feature displacement and M2M_2 is the maximum second-order curvature.
    • Remark 1 (proof in Appendix F.2) extends this analysis to Continuous Piecewise-Linear (CPWL) networks, proving that the bias is identically zero within any single affine polytope (M2=0M_2 = 0) and is bounded across activation boundaries by ∣I1P(T)−IPII(T)∣≤2k−2Dmax⁡Δgmax⁡Pboundary|I_{1P}(T) - I_{PII}(T)| \le 2^{k-2} D_{\max} \Delta g_{\max} P_{\text{boundary}}, where Δgmax⁡\Delta g_{\max} is the maximum gradient jump and PboundaryP_{\text{boundary}} is the boundary-crossing probability.
  • 2b (Guarantees for the Actual Implemented Estimator):
    • Proposition 4 (proof in Appendix F.3) derives the finite-sample convergence guarantee directly for the implemented biased 1-point estimator IM(T)I_M(T) around its expectation I1P(T)I_{1P}(T), establishing sample complexity M≥2C1P2ϵ2(ln⁡(2/δ)+kln⁡n)M \ge \frac{2C_{1P}^2}{\epsilon^2}(\ln(2/\delta) + k \ln n).
    • Appendix F.3.1 further analyzes the exact MquadM_{\text{quad}}-point Gauss-Legendre quadrature implementation (Iquad(T)I_{\text{quad}}(T)) executed in Algorithm 1, decomposing total estimation error into: (i) the 1-point bias bounded by Proposition 3 / Remark 1, (ii) a quadrature discretization bias that decays as O(sup⁡t∈[0,1]∣gT(2Mquad)(t)∣/(2Mquad)!)\mathcal{O}(\sup_{t \in [0,1]} |g_T^{(2M_{\text{quad}})}(t)| / (2M_{\text{quad}})!), and (iii) a finite-sample concentration bound governed by the discrete interior margin ηquad>0\eta_{\text{quad}} > 0 of the interior quadrature nodes.

(We continue the rebuttal in the next comment)

W3: Experiments do not support the key claims of this paper: for example, the learned-model dim-scaling experiments only use 8-12 features, the NLP eval uses five specified sentences, and there are no experiments with the interaction order greater than 3. This is not sufficient for me, as the main claim of this paper is to deal with the high feature dimension.

We substantially expanded our quantitative experiments across higher feature dimensions, larger evaluation sets, additional architectures, and multiple interaction orders and indices (Section 5, Appendix G, and Supplementary Materials):

  • Higher Feature Dimensions & Larger Evaluation Sets:
    • Long-Sentence NLP Benchmark (DistilBERT, n=25n=25): Evaluated across 20 SST-2 sentences of length n=25n=25 (Figure 13~15, Appendix G for SII), where exact subset enumeration (2252^{25}) is infeasible. CRIQ generally achieves lower MSE and NMSE than all black-box baselines across practical runtime budgets, as well as more stable convergence. While SPEX does show similar performance at higher speed, it is also extremely unstable in performance.
    • Expanded Fixed-Dimension Benchmarks: Expanded to 50 individual instances each for DistilBERT (n=11n=11), RoBERTa-base (n=11n=11), and CPWL MLP (n=11n=11), plus 30 instances for the Unanimity network (n=20n=20, Figure 2).
  • Interaction Orders (k∈{1,2,3}k \in \{1, 2, 3\}) and Multiple Indices: In addition to k=2k=2 Shapley Interaction Index in the main text, Appendix G and Supplementary Material reports full evaluations across orders k=1,2,3k=1, 2, 3 as well as Banzhaf, Shapley-Taylor, and FaithSHAP interaction indices. We focus empirically on k≤3k \le 3 to be consistent with prior interaction literature as well as because storing and evaluating all (nk)\binom{n}{k} interactions across all baselines becomes memory- and runtime-prohibitive for all methods.

W4: There is an inconsistency between the pseudo-code and the amortized estimator.

Thank you for highlighting this inconsistency. We revised Algorithm 1 (Appendix C) so that the pseudocode matches our 1-point amortized estimator:

  1. Precomputing the MquadM_{\text{quad}}-point Gauss-Legendre quadrature nodes and weights (tm,wm)(t_m, w_m) for dμ(t)d\mu(t);
  2. Computing a single gradient vector g←∇zf(ZS)∈Rng \leftarrow \nabla_z f(Z_S) \in \mathbb{R}^n with one backward pass and forming the elementwise product v←(x−R)⊙g∈Rnv \leftarrow (x - R) \odot g \in \mathbb{R}^n; and
  3. Amortizing across all target coalitions TT by reusing vv and the weights ωl=Sl−ul(m)ul(m)(1−ul(m))\omega_l = \frac{S_l - u_l^{(m)}}{u_l^{(m)}(1 - u_l^{(m)})} to evaluate YT=1k∑p∈Tvp∏l∈T∖{p}ωlY_T = \frac{1}{k}\sum_{p \in T} v_p \prod_{l \in T \setminus \{p\}} \omega_l.

W5: I do not see the code for this paper provided in the supplementary material (or a link to an anonymous repo). Since there are many things about this paper that concern me, I need to verify the real implementation of this paper before evaluating it further.

Thank you for raising this concern. We will release the complete codebase as supplementary material upon acceptance.

Reviewer_f9qzOpenReview ↗

Official response by the reviewer f9qz

I thank the authors for their response. I noticed that there are many changes between the original version and the revised version, so it took me a while to go back and forth between the two versions to see what has been added, what has been removed, and what has changed. I also checked the reviews of other reviewers to have a better view of this draft, as well as the rebuttals. However, after the rebuttals, many new concerns also arose. Here are some that I caught:

  1. Their own figure and comments seem to contradict what is claimed in the rebuttal and the main body, for example: a. In the rebuttal above, Q: Higher Feature Dimensions & Larger Evaluation Sets, the authors claimed "CRIQ generally achieves lower MSE and NMSE than all black-box baselines across practical runtime budgets". But Figure 13 (Appendix, p. 30) seems to show the opposite to me. Worse, CRIQ seems not to converge for me: the error curve stays flat in Fig. 13, and I think it is because of the bias floor, not stability.

b. In the rebuttal to the reviewer CW9j above, the author claimed "low MSE/NMSE immediately… steadily increasing rank-based performance, demonstrating clear speed and stability advantages": Fig. 14 and 15 do not tell that story to me.

c. In the rebuttal to reviewer e5F2, they say "clearly showing that CRIQ remains at or below the all-zero baseline from the earliest runtime budgets while rank-based metrics improve". This is also factually wrong if we check the figures, say Figs. 12, 18.

The above is just some of my figure fact-checking. I am not sure if there are more or not.

  1. Some claims contradict what their algorithm really does: In the abstract and main body, they say: "CRIQ avoids subset sampling". But it seems to me that CRIQ does do that (see e.g. Algorithm 1, Apx C, line 8).

At this point, it is very hard for me to believe the contents of the paper and the claims and premises of the authors in the rebuttals. After very careful consideration, I have no choice but to maintain my rejection and have to increase my confidence score to 3.

Thank you for your detailed review of our revised manuscript, as well as highlighting where our prior rebuttal summaries were overly broad. We would like to clarify both points as follows:

2. Some claims contradict what their algorithm really does: In the abstract and main body, they say: "CRIQ avoids subset sampling". But it seems to me that CRIQ does do that (see e.g. Algorithm 1, Apx C, line 8).

Thank you for highlighting this point. Our use of "combinatorial subset sampling" refers to sampling over the discrete power set 2N2^N, which differs fundamentally from CRIQ's product-Bernoulli Monte Carlo integration over the continuous multilinear extension. When calculating CIIs, most existing black-box estimators use cardinality-stratified sampling, with each strata s=∣S∣s=|S| having size that grows combinatorially as (ns)\binom{n}{s}. Even with it, they tne dto have quickly increasing variance as feature dimension nn increases. In contrast, CRIQ parameterizes the continuous MLE along a 1D trajectory u(t)∈(0,1)nu(t) \in (0,1)^nand draws Sj∼Bernoulli(uj(m))S_j \sim \text{Bernoulli}(u_j^{(m)}) at Gauss-Legendre quadrature nodes tmt_m (Algorithm 1, Line 8), which does not follow the same combinatorial process. Consequently, our method has dimension-free single sample bound and logarithmic sample complexity, and does not require combinatorially growing number of samples to achieve low variance.

To make this distinction clear, we will revise our phrasing to explicitly specify "combinatorial subset sampling required to achieve sufficient estimate variance".

1. Their own figure and comments seem to contradict what is claimed in the rebuttal and the main body.

Thank you for catching where our previous summaries were overly broad. We meant to summarize common empirical patterns, but it came out as over-generalized. We would like to clarify as follows:

  • Figure 13 (k=1k=1) and the bias floor: Including Figure 13 in "Figures 13-15" was an overgeneralization. At order k=1k=1, the overall variance is sufficiently low that the black-box estimators can converge more rapidly compared to the bias floor of 1-point CRIQ estimator. For the bias floor, as we have stated in our paper, CRIQ has a 1-point curvature bias error as we discussed theoretically in Section 4 and empirically in Section 5 (in nn RoBERTa test). Consequently, rather than failing to converge, it is a sign of having converged to that bias limit. Our claim of lower MSE/NMSE still applies at higher-order interactions (Figures 14-15).
  • Figures 14 & 15 (n=25n=25 DistilBERT, k=2,3k=2,3): In Figure 14 (k=2k=2), CRIQ (red) starts at MSE\text{MSE} near the zero baseline (dotted line), compared to SHAP-IQ (paired and unpaired) and/or SVARM-IQ which start far above the baseline. It stays lower than the those methods as runtime increases. Concurrently, the three rank-based metrics (Rank correlation, Precision@10, Jaccard@10) all improve steadily as runtime increases. In Figure 15 (k=3k=3), CRIQ also still has low MSE compared to SHAP-IQ. For rank-based metrics, we still see correlation increasing. For Precision and Jaccard@10, while the median is zero for both SHAP-IQ and CRIQ, CRIQ still shows upper quartile (red-shaded area) that does not occur for SHAP-IQ.
  • Figures 12 (unanimity k=3k=3) & 18 (MLP k=3k=3) vs. the all-zero baseline: Stating that CRIQ starts at or below the all-zero baseline across all figures was too broad. In the manuscript, we will explicitly qualify this behavior as holding primarily for k≤2k \le 2 rather than universally across all orders.

We will ensure that our discussion frames the advantages of CRIQ (low initial error and rapid stabilization) as common empirical patterns in high-dimensional (n≫Qn \gg Q) and order (k≥2k \ge 2) regimes in high-capacity models, rather than universal claims across every interaction order/budget/model combination.

Reviewer_CW9jOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Higher-order game-theoretic interactions in deep neural networks are crucial in interpreting what features are important for model predictions, but their computational cost is very high. To solve the high computational cost problem for, this work introduces CRIQ that evaluates atomic game interactions by mapping discrete derivatives to coordinate-ray integrals. To deal with the non-differentiability problem in CPWL, the work also uses IDR that decompose discrete derivatives into the sum of first-order derivative integrals to avoid it.

Critical strengths

S1: The idea to utilizing the derivation property in deep neural networks to decompose discrete derivatives into the sum of first-order derivative integrals is great. The mathematical process makes the calculation can be done in neural networks.

Critical weaknesses

W1: It seems that most non-differentiability functions used in the experiments are ReLU, and most deep neural networks are simple. Conducting experiments on more complex models will make the results more convincing.

W2: In the fast few years, there are some research works that already explored the way to decomposing discrete derivatives into the sum of first-order derivative integrals when applying Shapley value to deep neural networks and even LLMs. It is better to cite these related works properly.

Questions for authors

Q1: Will it cost a lot of time to use Monte Carlo sampling to estimate cardinal interaction indices CII?

Q2: Can you show us some case studies on the real applications, which can demonstrate that CRIQ can give better explanation for model prediction?

2 comments

We would like to thank the reviewer for their positive assessment and constructive feedback. In our revised manuscript, we upgraded our core estimator from the original 2-point estimator to an amortized 1-point quadrature estimator to enable gradient reuse across all features, and added experiments and references to address your comments.

W1: It seems that most non-differentiability functions used in the experiments are ReLU, and most deep neural networks are simple. Conducting experiments on more complex models will make the results more convincing.

We agree that evaluating CRIQ beyond ReLU MLPs is essential to demonstrate practical generality. In the revised manuscript, we expanded our quantitative benchmarks to include RoBERTa-base (which uses smooth GELU activations and multi-head self-attention) and an expanded DistilBERT test with 25-word sentences (n=25n=25).

W2: In the fast few years, there are some research works that already explored the way to decomposing discrete derivatives into the sum of first-order derivative integrals when applying Shapley value to deep neural networks and even LLMs. It is better to cite these related works properly.

Thank you for pointing out these related studies. We expanded Related Works under the dedicated paragraph "Gradient-based Estimate of Game-Theoretic Values" to cite prior works connecting discrete marginal differences to continuous gradient-based formulations, including Ancona et al. (2018), Fumagalli et al. (2025), Xu et al. (2026), and Deng et al. (2026). We also clarified how CRIQ builds upon these works by establishing an exact integral identity for arbitrary higher-order atomic game-theoretic interactions (k≥1k \ge 1) and resolving vanishing higher-order derivatives in CPWL networks with IDR.

Q1: Will it cost a lot of time to use Monte Carlo sampling to estimate cardinal interaction indices CII?

To address this question, we updated all benchmark plots to show the evaluation metrics directly against wall-clock runtime (in seconds, including both forward and backward passes) rather than query count. While black-box subset sampling suffers from high variance in high dimensions, our estimator maintains low variance even at manageable level of runtime:

  • Dimension-Independent Sample Complexity via Amortization: As shown in Propositions 3–4 and Appendix F.3.1, our 1-point estimator evaluates all nn feature gradients in a single backward pass and updates all (nk)\binom{n}{k} target interactions simultaneously, achieving sample complexity M=O(ϵ−2(ln⁡(2/δ)+kln⁡n))M = \mathcal{O}(\epsilon^{-2}(\ln(2/\delta) + k \ln n)).
  • Wall-Clock Runtime (Figures 2–3 & Appendix G):
    • On the n=20n=20 Unanimity benchmark (Figure 2), CRIQ achieves low MSE/NMSE and high correlation, Precision@10, and Jaccard@10 at low runtime).
    • On the n=25n=25 Long-Sentence DistilBERT benchmark (Appendix G, Figures 14~15 for higher-order SII, evaluated against high-budget SHAP-IQ pseudo-ground truth), CRIQ achieves low MSE/NMSE immediately and avoids the severe initial variance of black-box sampling while still achieving steadily increasing rank-based performance, demonstrating clear speed and stability advantages in high-dimensional regimes.

Q2: Can you show us some case studies on the real applications, which can demonstrate that CRIQ can give better explanation for model prediction?

To illustrate practical utility, we added Section 6 (Applications) presenting a qualitative case study on a cost-constrained clinical diagnostics scenario with n=100n=100 clinical variables divided into 5 acquisition cost tiers (1 to 10, total cost 520). Under a fixed query budget of Q=1024Q=1024, we evaluate the area under the AUROC-cost curve up to a 50 dollar cap (AUCPC50AUCPC_{50}) and the minimum expenditure required to reach 95%95\% of full-model performance (COST95COST_{95}).

Table 1 highlights two practical advantages of CRIQ in this setting:

  • Stability in High Dimensions: Due to combinatorial sampling noise at n=100n=100, SHAP-IQ's 2nd-order Shapley interaction estimates degrade sharply compared to 1st-order estimates, worsening COST95COST_{95} from 34.00 to 121.00. In contrast, CRIQ remains stable and improves when incorporating 2nd-order interactions (COST95COST_{95} improves from 43.00 to 36.00 for Shapley).
  • Native Support for Asymmetric Cost-Weighted Indices: Unlike symmetric-only interaction estimators, CRIQ parameterizes arbitrary path trajectories u(t)∈[0,1]nu(t) \in [0,1]^n by design. Assigning inverse-cost weights wi∝1/ciw_i \propto 1/c_i via ui(t)=twiu_i(t) = t^{w_i} (Appendix C) enables CRIQ to compute Weighted SII to prioritize high-value, low-cost diagnostic indicators, achieving the highest AUCPC50AUCPC_{50} (0.87950.8795) and lowest COST95COST_{95} (13.00).

Thank you for the comprehensive rebuttal and explanation. This effectively addresses almost all of my initial concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.