- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies a phenomenon, I am now naming here "interaction collapse", in attribution scores. Meaning, that an attribution score (i.e. Shapley value) is just an average of higher-order interactions and hence collapses these interaction onto a scalar value. The paper then expresses the Shapley value (and Shapley Taylor interactions) as an average of context-dependent interaction effects and express the variance of the effects over multiple contexts as a measure of how representative the scalar summary is.
Critical strengths
- This paper studies an important area of interpretability and particularly Shapley-based explanations. Having a measure of telling practitioners when our explanations are good or good enough is valuable work. This also brings explanations more together with research fields such as uncertainty quantification (and representation), which I think is quite nice (though not at all connected, see weakness section).
- Therein, the paper has also, while underdeveloped, a nice solution that more or less comes for free if you do a permutation-based estimation of the Shapley values. I like having uncertainty values of this sort, but I do not think this paper does anything meaningful with it to make it worthwhile.
- Overall the writing of the paper is clear.
Critical weaknesses
While I think this work honestly touches on interesting, albeit well understood, concepts I see many weaknesses with it:
- Purely descriptive Results: This work reads very much like an essay on interpretability and not like a technical paper with a well-defined methodological contribution. The descriptive findings are very much well understood. A scalar average, is averaging away effects. Same for higher-order interactions. This is the main motiviation for studying higher-order interactions and/or regional explanations [1]. While this work here does mention these streams of research as related work. It does nothing to actually connect the results here to anything concretely. For me what would make this more interesting is not just a descriptive study of the "phenomenon", but a technical contribution of an adaptive algorithm that uses knowledge about uncertainties of some players and a) either samples more carefully around these coalitions or b) adequately expands the representation space of the explanation from attribution to higher-order interactions. This however, would require lots of theoretical work on describing what the new explanation is. I could see this turning in a pruned version of FaithSHAP or Shapley Taylor, but have not spent too much time thinking on the implications. This is not done in the work.
- Weak Empirical Evaluation: The empirical evaluation is very rudimentary and not at all adequate for this level of venue. The language example is a BERT sentiment classification model which seems to only be applied on four hand-picked movie reviews. The Tabular example is only california housing (8 features) and the synthetic experiment in section 5.1 instantiates the intro illustration with four hand-crafted coalition values.
- No Code: The submission does not contain any source code or anything for validating the results. I wanted to checkout if there is more than the mere 4 imdb sentence, but couldn't. While it might be "easy" nowadays to code-up things with agents, I think it's unacceptable to submit a research paper and not submit the means to reproduce its content.
References
- [1] GRANITE https://arxiv.org/abs/2601.22771
Questions for authors
Q1: Can the uncertainty measure be used to prune what kind of coalitions one should need to evaluate to build better explanations?
2 comments
I thank the authors for their response. I already like the direction where the evaluation is going. I still think that this paper needs a stronger emphasize on building on your tool, as you already say "[...] These results give the diagnostic a structural justification for prioritizing coalitions whose scalar summaries warrant closer inspection." Your added Algorithm 1 is one way of doing something with this information, but I still think more needs to be done for making this an impactful explanation method.
I will increase my score to weak reject to reflect that this an interesting line of research, which just needs more to make it a full-fledge paper.
Notes:
- It is very hard to take a look at the updates/changes in the paper as it is not highlighted what changed.
- Code should really be submitted at submission time, sucht that one can actually review the work carefully.
We thank the reviewer for recognizing the value of assessing when Shapley explanations are representative and for suggesting a concrete connection to adaptive interaction analysis. We agree that this practical connection deserves greater emphasis. Accordingly, we have revised the manuscript to clarify our theoretical contributions and substantially expanded Section 5 with a variance-guided inspection algorithm and quantitative studies on vision, language, and tabular models.
Scientific contribution. Prior work has documented that Shapley averaging can conceal opposing effects. Our contribution is to characterize the higher-order interaction structure behind this loss of information and establish when and why it warrants further analysis. Section 3 derives an order-wise representation showing how context-dependent higher-order effects are aggregated into ordinary Shapley values. Section 4 then establishes that large Shapley context variance implies substantial aggregate interaction energy among immediate higher-order extensions (Theorems 4.2–4.3). These results give the diagnostic a structural justification for prioritizing coalitions whose scalar summaries warrant closer inspection.
Variance-guided inspection (Q1). To address the reviewer's question of whether variance can guide coalition selection, we added Algorithm 1 in Section 5.4. The procedure starts with individual features, selects the top-K candidate coalitions by context variance, evaluates their one-feature extensions, and repeats this greedy selection at successive interaction orders. In this way, the variance–energy relationship guides inspection toward promising branches of a search space that grows combinatorially with order.
Section 5.4 evaluates this procedure (K=3, up to order four) on 30 ImageNet images and 30 IMDB reviews. On ImageNet, it inspects only 0.51% of possible fourth-order coalitions per input on average. The energy-ranked Top-10 interactions it discovers have 6.34× (ImageNet) and 2.83× (IMDB) higher interaction energy than a seeded random reference inspecting twice as many final-order coalitions. Their absolute Shapley–Taylor interaction scores are also 3.40× and 2.57× higher, respectively (see Section 5.4 and Appendix E.1 for experimental details).
Additional empirical evidence. We also extended the exact tabular evaluation (Section 5.2, Appendix E.2) from California Housing to Bike Sharing and Adult Census. Variance-based selection captures substantially more next-order interaction energy than ranking by scalar mean effects on California Housing (56.7% versus 34.5%) and Bike Sharing (68.7% versus 55.7%). The improvement is smaller on Adult Census (72.4% versus 69.4%), where large scalar effects already track context variance more closely. This pattern shows that variance-guided inspection is especially informative when scalar effect magnitudes do not reflect the underlying context dependence.
Connection to GRANITE. We appreciate the reference to GRANITE and share its motivation that feature interactions can complicate the interpretation of feature-level explanations. We have added it in the revised Introduction. GRANITE seeks regions where different influence operators agree, whereas our analysis examines the context dependence hidden within a fixed Shapley attribution. These questions are complementary. For example, consider v(S)=a1[1,2⊆S]−a1[1,3⊆S]. The pure, partial, and full influence scores for feature 1 all equal zero, yet its Shapley context variance is a2/3. Thus, even agreement among influence operators can conceal substantial context-dependent effects.
Reproducibility. We provide an anonymized supplementary code package implementing context-dependent interaction effects, permutation-based estimators of their mean, variance, and energy, and variance-guided higher-order inspection. Two executable notebooks demonstrate these diagnostics and inspection procedures using analytical and synthetic examples, as well as cached effects from five IMDB sentences and five ImageNet images. Dataset and model sources, preprocessing details, and game specifications are also provided.