acceptodds
Under review as a conference paper at ICLR 2027

Auditing Higher-Order Feature Interactions via Interventional Decomposition

Abstract

Signed pairwise interaction scores are widely used in explainable AI to classify feature cooperation as synergy (if positive) or redundancy (if negative). However, when interactions involve more than two features, pairwise scores fail: they either vanish entirely or flip sign across inputs, mislabeling synergy as redundancy. A single scalar summary is fundamentally insufficient to disentangle uniqueness (U), redundancy (R), and synergy (S). To overcome this limitation, we introduce InDe-X: Interventional Decomposition for eXplainability. While classical predictability decompositions study theoretical data capacity by retraining models on feature subsets, our method audits the internal causal pathways of a fixed, deployed network via post-hoc interventional masked inference, decomposing marginal predictive gains into per-feature U/R/S profiles over the Boolean lattice. We provide finite-sample concentration bounds, strict variance reduction via coupled diamond sampling, and uniform convergence guarantees over finite coalition vocabularies. InDe-X recovers ground-truth roles on tabular SCMs (overcoming up to 411x baseline conflation), surfaces synergistic and backup circuits in GPT-2 missed by activation patching, and identifies multi-region anatomical co-dependencies on ChestX-ray14 with superior deletion faithfulness.

Primary area:
interpretability and explainable AI
Keywords:
Explainable AI, Higher-Order Feature Interactions

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BtP9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to simplify the U/R/S decomposition into something more achievable in practice. This is maybe done by the marginal/interventional value function and this is maybe to connect to the mechanistic interpretability literature.

Critical strengths

Critical Strengths

Understanding U/R/S decompositions into unique contributions, redundant contributions, and synergistic contributions is an important problems.

It is very possible there are good ideas hidden behind the current writing.

Critical weaknesses

Critical Weaknesses

  1. This paper manages to conflate all four commonly used value functions from the XAI literature: the baseline value function, the marginal value function, the conditional value function, and the causal value function.

  2. The word "causal" is occasionally used when the "interventional" meaning is implied (i.e. true to the model). It is unclear whether this is an error due to the above conflation. If not, the wording interventional should probably be used throughout, or the authors should clarify they mean causal in the sense of mechanistic interpretability [A]. If that is true, the SCM wording should also probably be removed because it is confusing.

  3. This is of particular concern because the typical definitions of U/R/S are directly tied to the conditional/informational interpretation of interpretability. In particular, there is no existing definition of "interventionally redundant information" and the paper seemingly makes no attempt to define it for their work, despite its seeming importance for their claims and their experiments. It is very plausible to me that this would be an interesting research direction introduced by this paper; however, the current version makes no attempt to make this clear.

  4. The Monte Carlo bound and related variance reduction seem completely unrelated to the practical problem at hand, which is the exploration of all the subsets to properly define the minimum and maximum (later called the "budgeted regime"). It seems that assumptions on the functional form would be more appropriate to show the search gives a good approximation (e.g. submodularity like properties). The current theory does not seem relevant to the practical research.

  5. The reasoning behind the reduction should be discussed more explicitly in the paper. Otherwise, the importance of the work in connection with existing work on "diamond sampling" from the feature interaction literature and marginal contributions will not be fully understood.

[A] "Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability" Atticus Geiger et al. JMLR 2025.

Questions for authors

See the weaknesses above. If necessary, please ask your friend to convert the above weaknesses into a list of corresponding questions.

Reviewer_qt7aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper argues that current interaction-based explanation methods (e.g. Shapley-Taylor, k-SII) are inedequate to explain real-world models because, to remain human interpretable, they must often project high-order interactions into order-2 indices. This projection induces some counterintuitive interpretations whenever a positive interaction is meant to be interpreted as a "synergy" and a negative one as a "redundancy". The authors instead argue that a human interpretable explanations should report the "synergy" and "redundancy" of a feature using all combinations of the remaining features. The resulting algorithm called InDeX is applied on several explanability use-cases.

Critical strengths
  • The experiments are extensive and cover a range of application of XAI (tabular, vision, text). The wide applicability of the approach stems from its model-agnostic nature, making it relevant to the XAI community at large.
  • This paper highlights the literature on Partial Information Decomposition, which seems like a relevant alternative to cooperative game theory for computing rigorous notions of synergy and redundancy between input features. Although it is not clear how equation (4) relates to this existing literature, the definitions of RiR_i and SiS_i are interesting contributions. Indeed, they highlight the existence of contexts where adding a given feature has less/more impact on the model loss than adding this feature to an empty context. This conveys the notion of 'redundancy' and 'synergy' without needing to compute high-order interactions.
Critical weaknesses
  • I think that the article is dense and hard to read. Sometimes different terminologies are used to mean the same thing, unnecessary terms are used, or mathematical symbols are written without an introduction . For example

    • The first sentence quickly introduces the terminologies of 'unique', 'redundant', and 'synergistic' features (without an explanation of their meaning or a citation). The paper then reintroduces these terms at line 40 using upper case letters : '[...] how much a feature's role is unique (U), redundant (R), or synergistic (S)'. These three letters are then systematically used toghether in the triad 'U/S/R' but with slight differences. Line 49 writes 'U/R/S codomain', Table 1 uses 'U/R/S profile' while line 84 uses 'U/R/S-disentangled'. The rest of the paper uses 'U/R/S decomposition'. I would recommend taking the time to introduce and motivate the notions of unique, redundant, and synergistic features (citing prior work on PID), and then systematically use the same terminology (e.g. 'U/R/S decomposition') when discussing it.
    • line 124 writes 'atomic marginal intervention (baseline imputation) (Janzing et al , 2020)' but there is noting 'atomic' about Equation 1 since several features can be removed at the same time. The word 'atomic' is unnecessary.
    • The word Structural Causal Model (SCM and ⨁SCM3\bigoplus \text{SCM}_3) is used to represent models used in toy experiments but there are no details in appendix H about the causal graphs behind these experiments. A SCM supposes a graph of causation between the input features and the target (including edges between input features to signify feature dependence). Appendix H should clarify the dependence introduced between the input features. If features are independent, then why use the terminology SCM at all since the causal graph is so simple (e.g. x1 -> y <- x2)? Why not simply say 'Toy tabular models' instead of SCM?
    • Line 178 write Mobius coefficients without a proper definition or introduction. They are defined at line 994 of the appendix as part of the proof of Proposition 1, but not in a dedicated equation and not with a citation. It would also help to have a reference to Appendix A in the main text.
    • The way section 3.4 is currently written can be confusing because it introduces the pair-level synergy and redundancy indices (equation 9) after a Theorem that highlights how to compute them. I would be more clear if theoretical quantities were systematically introduced and motivated before presenting guarantees on their estimates. Sections 3.1 and 3.2 are a good template to follow : introduce a new theoretical quantity, motivate it with a failing scenario of existing work (Proposition 1), end with statistical estimates.
  • I don't think that Proposition 1 truly highlights a failure of existing interaction-based explanability methods. It instead demonstrate that the interpretation of the sign of feature interactions depends on the coallitional game used. k-SII and Shapley Taylor use the coallitional game ν(S)=EZ[f(xS,Z−S)]\nu(S) = \mathbb{E}_Z[f(x_S, Z_{-S})] instead of equation 1). Assuming that a full-order method is used, the triplet x1,x2,x3x_1, x_2, x_3 will be attributed the three-way interaction ⨁(x1,x2,x3)−0.5\bigoplus(x_1, x_2, x_3) - 0.5 and all other interactions will be null. This faithful interaction changes sign based on the parity of the three features, which does not coincide with the intuitive notions of synergy or redundancy. However, a negative interaction still means that a specific feature combination pushes the model output bellow its average, which is as useful but different interpretation. If one applied k-SII and Shapley Taylor at full order to the minus-loss-game (-1 times equation 1), they would get a three-way interaction of between x1,x2,x3x_1, x_2, x_3 of 1/21/2, the correct interpretation that these three features synergistically work together toward the model performance. The Proposition would be more fair if it accentuated that the main culprit is the coallitional game (more than the redistribution of Mobius coefficients), and that the choice of game is crucial to extract the desired interpretations.

  • The paper would gain a lot from discussing the main algorithm (and its complexity) inside the main text. The current section 3.4 is not sufficient to understand the complexity of the approach. The parameter TT does not appear to be introduced in the main manuscript. It would be possible to make room in the paper (for a pseudo-code for instance) by moving some of the theoretical results in the appendix. Theorem 1 does not need to be in the main paper because it is such a straighforward result : apply Hoeffding inequality on the random variable L^(S)\widehat{L}(S). Also, the statements from Theorem 3 are simply proven with a union bound over V\mathcal{V}, so this Theorem could perhaps be combined with Proposition 2 to reduce space. In appendix E, it appears that Theorem 3 is also supposed to state guarantees about the optimal subsets S⋆S^\star from optimizing L(S)L(S) but the results from step 2 of the proof do not appear in the theorem statement at line 295.

Questions for authors

I don't have any specific questions beyond the points presented above.

Reviewer_Ebx9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The prevailing paradigm in feature interaction analysis fundamentally fails when cooperation extends beyond two variables: projecting higher-order dynamics down to signed pairwise scalars arbitrarily conflates synergy with redundancy, erasing genuine cooperative mechanisms or inverting their signs across instances. This paper overcomes this representational dilemma by operationalizing the Partial Information Decomposition triad—Uniqueness, Redundancy, and Synergy—for post-hoc auditing of deployed neural networks without combinatorial retraining.

Critical strengths

1.The paper addresses a useful limitation of model explanations. A component can have little effect in isolation while becoming important in the presence of other components. This paper makes this dependence on context explicit through its minimum, standalone, and maximum marginal gains. Auditing a fixed predictor without retraining is particularly appropriate for studying the behavior of an existing model. The shared coalition evaluations and coupled sampling also provide a practical way to obtain these profiles under a specified inference budget.

Critical weaknesses

1.On Synth3, Table 10 reports strong recovery of uniqueness and synergy, but much weaker redundancy recovery: the Pearson correlations are (0.185\pm0.464) for per-feature redundancy and (0.261\pm0.109) for pairwise redundancy. Since identifying redundancy is a central objective, these results need more explanation. In addition, Figure 1 appears to show negative redundancy values and a substantial nonzero contribution for the spectator feature, despite the nonnegativity of the defined redundancy scores and the text’s claim that the spectator is near zero. It is unclear whether this reflects plotting, normalization, labeling, or computation. Together, these issues limit confidence in the broader claim that the method consistently recovers the intended U/R/S roles.

Questions for authors

What is the scope of the reported confidence guarantees? Do they apply to the adaptive vocabulary and repeatedly checked Student-(t) stopping rule used in the implementation, or only to fixed vocabularies and sample sizes? If they apply to the implemented procedure, how is simultaneous coverage controlled across the selected coalitions and stopping times?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.