acceptodds
Under review as a conference paper at ICLR 2027

SINC: Discovering Global Higher-Order Feature Interactions

Abstract

Understanding how black-box models combine features to make predictions remains a fundamental challenge in explainable AI (XAI). Existing post-hoc methods primarily focus on individual feature importance, pairwise interactions, or local explanations, making it difficult to recover global interaction structure. We propose Synergy Identification via Network Clustering (SINC), a model-agnostic, post-hoc framework for discovering global higher-order feature interactions in trained black-box models. SINC constructs a sparse interaction network from pairwise Friedman’s H-statistics, identifies communities using Leiden clustering with a Constant Potts Model objective, and ranks their predictive contribution using permutation importance. In doing so, SINC recovers the global feature interaction structure from pairwise interaction measurements. Across controlled synthetic and real-world datasets, we demonstrate that SINC recovers complex feature synergies and effectively identifies uninformative features. Compared with SHAP-based interaction analysis and the Archipelago method, SINC reveals higher-order groupings that are difficult to capture using a pairwise method and yields unified global communities, overcoming the grouping variability inherent in aggregating local explanations. Finally, we show that SINC serves as a diagnostic tool, revealing whether a model has successfully learned the underlying interaction structure of the data.

Primary area:
interpretability and explainable AI
Keywords:
Explainable AI, Global Feature Interactions, Black-Box Models, Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dJQhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a posthoc XAI method to identify global higher-order feature interactions. It uses pairwise Friedman H-statistics to measure interaction strengths and Leiden clustering with a Constant Potts Model (CPM) objective to identify communities of interacting features. These communities are then ranked based on their importance using permutation importance. The authors empirically compare their method with SHAP, pairwise Shapley interactions, and Archipelago.

Critical strengths

the general goal of finding the global higher-order interaction structure is interesting

Critical weaknesses

I do not think the the methodological approach to find global higher-order interactions is sound and the respective empirical validation is also not convincing as elaborated below.

1. Methodological weaknesses: SINC aims to identify higher-order feature interactions based solely on pairwise H-statistics. However, it is unclear why communities based on a measure that quantifies only pure pairwise interactions that does not account for higher order interactions should be able to identify higher-order interactions. For example, f(x1,x2,x3)=x1x2+x2x3f(x_1,x_2,x_3)=x_1x_2+x_2x_3 contains only pairwise interactions, but SINC may group all three features into one community implying a three-way interaction between them. Conversely, for f(x1,x2,x3)=x1x2x3f(x_1,x_2,x_3)=x_1x_2x_3 with independent, zero-mean features, all pairwise partial dependence functions vanish, although a three-way interaction exists. SINC would therefore not be able to identify this interaction from its pairwise graph. Furthermore, the H-statistic interaction values are normalized by the pairwise PDPs which differ for different interaction pairs and thus the interaction values measure the percentage of pairwise interaction compared to the two respective main effects they account for within the pairwise PDP. Thus, the H-Statistic values are not comparable as absolute interaction values which makes it unclear how their values should be compared and combined as edge weights in the CPM objective. The paper does not justify why this clustering procedure based on the H-Statistic values should recover higher-order interactions rather than simply connected groups of pairwise interactions. It is also unclear why Friedman’s H-statistic was chosen over alternative measures such as ALE-based interactions [1], particularly given the extrapolation problems of partial dependence when features are correlated.

2. Experimental weaknesses: The experimental evaluation does not sufficiently address these methodological concerns and raises additional questions, such as why small nonzero SHAP interaction scores are highlighted as a limitation, whereas SINC explicitly thresholds small interaction values. The comparison with SHAP itself is not very meaningful since it is an aggregation over all weighted effects of a feature not decomposing it into interaction effects. Only adding pairwise Shapley interactions and saying that the combination is more complicated then the proposed approach is in my opinion no real baseline. Particularly since higher-order Shapley interaction decompositions already exist [e.g., 2–4]. It is unclear why these methods were not considered. The comparison with Archipelago could also benefit from a clearer explanation of the methodological differences. In addition, the synthetic experiments do not investigate cases in which pairwise interactions are insufficient to identify the true higher-order interaction structure, nor do they consider correlated features. These experiments would be particularly relevant for assessing the fundamental assumptions of the proposed method -- particular based on the pairwise H-Statistic.

[1] Apley & Zhu (2020). Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models. Journal of the Royal Statistical Society: Series B.

[2] Fumagalli et al. (2023). SHAP-IQ: Unified Approximation of any-order Shapley Interactions. NeurIPS.

[3] Muschalik et al. (2024). shapiq: Shapley Interactions for Machine Learning. NeurIPS.

[4] Bordt & von Luxburg (2023). From Shapley Values to Generalized Additive Models and back. AISTATS.

Questions for authors

see weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer dJQh (1/2)

We thank the reviewer for the detailed review and the helpful references. Due to the character limit, our response is split into two parts.

"it is unclear why communities based on a measure that quantifies only pure pairwise interactions [...] should be able to identify higher-order interactions"

We agree with the examples provided. In the revised manuscript, SINC is now presented as a method for identifying global communities of mutually interacting features rather than higher-order interactions. To further strengthen the method, we have also replaced the Leiden algorithm with maximal clique detection, so that every pair within a community must interact. While this does not establish that a community corresponds to a higher-order interaction, it yields communities that are more closely aligned with the observed interaction structure. In particular, it addresses the reviewer's question of why features should be grouped rather than "simply connected groups of pairwise interactions", since features are no longer grouped through indirect chains of interactions but instead must be directly connected to every other feature in the community.

"the H-Statistic values are not comparable as absolute interaction values which makes it unclear how their values should be compared and combined as edge weights in the CPM objective"

We partly agree. The H-statistic measures the fraction of a feature pair's joint effect that is due to their interaction, so its values are comparable across pairs as proportions, but not as absolute interaction strengths. Consequently, a pair with a relatively weak joint effect can still obtain a high H-statistic value. In the revised manuscript, the concern is mitigated by replacing the CPM-based community detection with maximal clique detection. As a result, edge weights no longer contribute to an objective function and are only used to decide whether two features interact. The absolute relevance of a detected community is assessed separately by the impact weighting. Communities composed of features with limited predictive relevance therefore receive low importance scores, even if their pairwise H-statistics are high. We now discuss this distinction explicitly in the manuscript.

"It is also unclear why Friedman's H-statistic was chosen over alternative measures such as ALE-based interactions [1], particularly given the extrapolation problems of partial dependence when features are correlated."

We chose the H-statistic because it provides a single, global interaction strength for each feature pair with an established and efficient approximation, which is what the construction of the interaction network requires. We agree, however, that partial dependence varies features independently and can therefore evaluate the model on unrealistic feature combinations when features are correlated, whereas ALE-based interactions avoid this extrapolation by only considering local changes within the data distribution. The two approaches thus describe different aspects of the model, with partial dependence capturing how the model itself responds to its features and ALE capturing its behavior on realistic inputs. We now discuss this trade-off in the limitations. Since the interaction measure is a modular component of SINC, an ALE-based interaction network could replace the H-statistic without changing the remaining stages, which we consider a promising direction for future work.

"why small nonzero SHAP interaction scores are highlighted as a limitation, whereas SINC explicitly thresholds small interaction values"

We agree that this was inconsistent and thank the reviewer for pointing it out. We have removed the remark.

AuthorsOpenReview ↗

Response to Reviewer dJQh (2/2)

"The comparison with SHAP itself is not very meaningful [...] higher-order Shapley interaction decompositions already exist [2–4]. It is unclear why these methods were not considered."

We thank the reviewer for this comment and for the suggestion to include higher-order Shapley interactions as a baseline. We agree that pairwise SHAP interactions alone are not a sufficient baseline. We keep SHAP in the comparison because it is the most widely used explanation method in practice and therefore a natural reference point. Following the reviewer's suggestion, we now also compare SINC with SHAP-IQ, computed with the shapiq library for a global, loss-based game up to order five, and discuss Shapley interaction indices in the related work. We report k-SII, as FSII, STII and Möbius interactions yield the same recovered interaction structure in our experiments.

On the synthetic benchmark, SINC recovers 72.0% of the ground-truth terms on average, compared with 37.7% for SHAP-IQ. SHAP-IQ assigns the effect of each feature set to its pure interaction component, and since the higher-order components of most benchmark terms are small compared with the main effects of their features, it recovers only one higher-order term. On two of the three real-world datasets Califiorna Housing and Bike Sharing, SHAP-IQ identifies the same dominant interactions as SINC, but requires the maximum interaction order to be chosen in advance and scores every feature set up to that order. Since the number of such feature sets grows combinatorially with the number of features and the interaction order, SHAP-IQ quickly becomes computationally prohibitive. On HIGGS, with 28 features, there are more than 120,000 feature sets of order at most five, so that SHAP-IQ could not be computed up to this order, and with a maximum order of three, every feature set contributes less than 2% of the total, so that no grouping of the features emerges.

At the same time, SHAP-IQ has a clear advantage that we now report explicitly. Since it scores each feature set separately, it can distinguish several pairwise interactions from a single higher-order interaction, which SINC cannot (Appendix I). We therefore see the two methods as complementary, with SHAP-IQ decomposing interactions by order and SINC summarizing them as global communities of interacting features.

"the synthetic experiments do not investigate cases in which pairwise interactions are insufficient to identify the true higher-order interaction structure, nor do they consider correlated features"

We fully agree with the comment. For the first point, we have added two functions, x1x2+x2x3+x1x3x_1x_2 + x_2x_3 + x_1x_3 and x1x2x3x_1x_2x_3, which produce the same pairwise interaction network but differ in their higher-order structure (Appendix I). SINC returns the community {x1,x2,x3}\lbrace x_1, x_2, x_3 \rbrace for both, which is correct only for the latter, while SHAP-IQ distinguishes the two. We now state this explicitly as a limitation. Regarding correlated features, we acknowledge that the synthetic experiments use independent inputs. The real-world datasets contain strongly correlated features, for example latitude and longitude in California Housing and the reconstructed masses in HIGGS. We now discuss in Section 5 how SINC's use of partial dependence and permutation can lead to extrapolation for correlated features, and we point to conditional alternatives such as ALE-based interactions.

I thank the authors for their detailed responses to the rebuttal and their work on the revised manuscript.

Since the authors changed the core of the methodology, the paper needs a major revision, which is not possible within this short rebuttal time. Also, I would highly recommend to add a background section that introduces the clique detection approach and explaining the intuition behind its usefulness in their approach in more detail.

I am still not convinced by scaling the H-Statistic value by the pairwise PD and thresholding based on that. I think this is not solved by the clique detection, since you are still considering the pairs that fullfill H_ij > thau. Therefore, you might not consider a feature pairs with a high interaction only because they also have a high main effect, but you consider very small (maybe spurious) interactions if they have no main effects since even very small effects then always have an H-Statistic value of 1. Therefore, accounting for the magnitude afterwards via permutation importance does not work, since you filtered out potentitally importance one already in the step before.

It is also not clear to me, how Shapley-based importance on the communities helps to mitigate the problem of features appearing in multiple communities (in non-disjoint cases) not be counted multiple times. If communities are treated as players, then the interactions between communities are fairly distributed among those communities, but this does not generally add up on a feature level or account for the overlap. This should be clarified by the authors and stated more formally.

I appreciate the additional comparisons with SHAPIQ, however, the repository was not updated, which makes it hard to verify those results. If calculated exactly, SHAPIQ provides the decomposition into pure interaction effects. The experiments for higher-order interactions are all done based on an MLP for which the authors do not know the learned interaction effects, but making claims about the recovery of interactions of the true-data generating function. To actually understand interaction recovery, the experiments should additionally be done based on the oracle itself.

I thank the authors again for their efforts, but I do not think that the paper is ready for acceptance, therefore I remain my score.

We thank the reviewer for the comment and the continued engagement with our work. We have updated the repository to include the current methodology and the comparison with SHAP-IQ. We would also like to clarify that for the synthetic benchmark functions F1–F10, Table 5, includes the results of SINC and the baseline methods on the oracle function, shown in parentheses, as requested by the reviewer. On the oracle, SINC recovers 83.3% of the ground-truth terms, compared with 46.1% for Archipelago and 37.7% for SHAP-IQ.

Reviewer_TrbEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SINC, a model-agnostic, post-hoc framework for discovering global feature interaction groups in trained models. The method constructs a weighted graph using approximate pairwise Friedman’s H-statistics, applies Leiden community detection with a Constant Potts Model objective to identify feature groups, and ranks their predictive importance through group permutation. The authors evaluate SINC on controlled synthetic functions and three real-world datasets, comparing it with SHAP-based interaction analysis and Archipelago. The results suggest that SINC can recover meaningful feature groups in selected settings and help diagnose differences in the interaction structures learned by converged and undertrained models.

Critical strengths
  1. SINC offers a straightforward way to organize pairwise interaction scores into global feature groups. Combining these groups with permutation importance makes the results easier to inspect without modifying or retraining the target model.
  2. The experiments cover a white-box example, ten synthetic functions, and three real-world datasets. The synthetic examples are particularly useful for examining whether the recovered groups agree with explicitly defined dependencies.
  3. The comparison between converged and undertrained MLPs illustrates a useful diagnostic application. The interaction graphs reveal differences in learned dependencies that are not apparent from predictive accuracy alone.
Critical weaknesses
  1. My main concern is the connection between pairwise communities and higher-order interactions. For independent x~U(-1,1), f(x)=x_1+x_2+x_3+x_1x_2x_3 contains a three-way interaction, but all population pairwise H-statistics are zero. Conversely, a community can arise from pairwise terms alone. The method therefore identifies a grouping of pairwise dependencies, while the paper makes a strong claim about recovering higher-order structure without establishing when this is possible.
  2. In Table 2, Archipelago’s attribution and SINC’s permutation importance quantify different aspects of model behavior. Their 50%/50% and approximately 74%/25% allocations can therefore differ without either being incorrect, resulting in an unfair comparison.
  3. Table 5 evaluates explanations against the generating function’s interactions, although the models being explained are trained MLPs. The authors justify this using R^2≥0.95, but accurate predictions do not establish that the same interaction structure was learned. The reported recovery rates mix errors in the explanations with discrepancies between the fitted models and the generating functions.
  4. The real-world discussion mainly establishes that the groups appear plausible, rather than that they faithfully describe the model’s behavior on realistic inputs. Marginal partial dependence and group permutation can break feature dependencies and produce unsupported combinations. This issue is not assessed despite the use of related geographic, weather, and kinematic variables, leaving the interpretation of these communities uncertain.
  5. The computational discussion focuses on reducing the cost of individual H-statistic estimates, but does not quantify the cost of the complete pipeline relative to the baselines. All feature pairs are still evaluated before sparsification, so thresholding does not reduce this initial quadratic cost. Without runtime or model-query comparisons, the claimed efficiency advantage and scalability remain unsubstantiated.
  6. Most figures in the main paper have limited explanatory value. The workflow diagram largely repeats the procedural description, while the interaction graphs provide little visual guidance on how to interpret the recovered groups.
Questions for authors
  1. Under what assumptions can SINC recover higher-order interactions from pairwise H-statistics?
  2. What justifies the importance-ranking comparison in Table 2 and the use of the generating function’s interactions as ground truth for the trained MLPs in Table 5?
  3. How does the real-world analysis account for unrealistic feature combinations introduced by marginalization and permutation?
  4. What runtime or model-query measurements support the claimed efficiency advantage of the complete SINC pipeline over the baselines?
3 comments
AuthorsOpenReview ↗

Response to Reviewer TrbE

We thank the reviewer for the detailed review and for the positive remarks on the synthetic experiments and the diagnostic comparison.

"Under what assumptions can SINC recover higher-order interactions from pairwise H-statistics?"

We agree with the reviewer that, in general, SINC cannot recover arbitrary higher-order interactions from pairwise H-statistics, as illustrated by the reviewer's example. We have therefore revised the manuscript to avoid interpreting the discovered communities as higher-order interaction structures. Instead, SINC is now presented as a method for identifying global communities of mutually interacting features. To further address this point, we have also replaced the Leiden algorithm with maximal clique detection. Under this revised formulation, a weaker but more precise statement holds: if all pairwise interactions associated with a higher-order interaction satisfy H2≥τH^2 \geq \tau, then all participating features will be contained within a detected community. However, the resulting community alone does not distinguish a higher-order interaction from several pairwise ones. We now discuss both limitations explicitly in the manuscript.

"In Table 2, Archipelago's attribution and SINC's permutation importance quantify different aspects of model behavior."

We agree and have revised the manuscript accordingly. The paper now explains that Archipelago's attribution scores and SINC's permutation importance capture complementary aspects of model behavior. We further explain that, under squared loss and independent features, the permutation importance of an additive term equals twice its variance. As a result, SINC's importance allocation reflects each term's share of the model output variance, which for Eq. 3 is exactly 75% and 25%, respectively.

"accurate predictions do not establish that the same interaction structure was learned"

We agree that the recovery rates reported in Table 5 reflect both the accuracy of the explanation methods and discrepancies between the fitted models and the underlying data-generating functions, and we now state this explicitly in the revised manuscript. Since all methods explain the same trained models, such discrepancies affect all methods equally, and the comparison between the methods therefore remains informative. We also include the results obtained using the ground-truth functions as the model, allowing the evaluation to isolate the performance of the explanation methods from model-fitting errors.

"How does the real-world analysis account for unrealistic feature combinations introduced by marginalization and permutation?"

It does not, and we now discuss this explicitly as a limitation in the manuscript. Varying features independently is a deliberate choice, as SINC aims to describe how the model itself responds to its inputs rather than how features co-occur in the data. However, for strongly correlated features, marginalization and permutation may evaluate the model on feature combinations that are absent in the training data, so the detected communities and their estimated importance may partially reflect the model's behavior under extrapolation.

"What runtime or model-query measurements support the claimed efficiency advantage of the complete SINC pipeline over the baselines?"

We agree that the efficiency claim was not substantiated and have therefore removed it. The approximated H-statistics require 2RM+R2M2RM + R^2M model queries per feature pair, which is independent of the dataset size, but the total cost still grows quadratically with the number of features, and thresholding does not reduce this cost.

"Most figures in the main paper have limited explanatory value."

We think some figures add value by illustrating the method; however, we agree that some figures are redundant and have therefore removed them in the revised manuscript.

We again thank the reviewer for the detailed review. We would greatly appreciate it if the reviewer could take a look at our response and the revised manuscript, and we hope that we have addressed the concerns raised.

Thanks for the authors' reply. However, they submitted the reply very late. I cannot carefully check their paper thoroughly with their new results and explanations due to the late submission. However, I still  think the paper is not ready for acceptance.

Reviewer_SU5yOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SINC, a post-hoc, model-agnostic method for identifying global feature interaction structure in trained opaque models. SINC computes a scalable approximate pairwise Friedman (H)-statistics, thresholds them to construct a weighted feature graph, applies Leiden community detection, and finally ranks the resulting communities using permutation importance. The method is evaluated on synthetic benchmarks and several real-world datasets, with comparisons primarily to SHAP interaction values and Archipelago. I find the goal relevant and the resulting visualizations potentially useful. However, I have substantial concerns about the methodological novelty and, more importantly, about the interpretation of the detected communities as higher-order interactions.

Critical strengths

The paper has a clear motivation and presents a simple pipeline that can be applied post hoc to an already trained predictor. The global partition can be easier to interpret than a large matrix of pairwise interaction scores, and the synthetic experiments make the behavior of the method relatively easy to inspect and understand. I find also useful the experiment comparing converged and undertrained models, which is a very nice diagnostic and makes good sense.

Critical weaknesses

My primary concern is that the defined communities of pairwise interactions do not necessarily translate to a higher-order interaction, at least without additional assumptions that make the problem very specific. SINC constructs its graph entirely from pairwise Friedman H-statistics and subsequently clusters this graph. However, graph communities and higher-order statistical interactions are different mathematical objects.

For example, considering only pairwise interaction terms in f(A,B,C)=AB+AC+BC. Having this as a graph creates a triangle which clustering algorithms would naturally tend to group together. In this case though there is no evidence for requiring a genuine higher-order interaction among {A,B,C}, so this should not be interpreted as evidence for a genuine three-way interaction. In addition, the use of the H-statistic to marginalise to obtain a pairwise interaction has additional problems. Considering the interaction f(A,B,C)=ABC with independent centered variables, marginalising over C gives PD_AB(a,b)=E_C[abC]=ab E[C]=0. Therefore, the pairwise AB signal vanishes, and similarly for the other pairs, despite the presence of a true third-order interaction. More generally, higher-order interactions can completely vanish under the pairwise marginalisation.

Therefore, I think claims that SINC recovers higher-order interaction structure need substantial additions. What SINC directly recovers is a partition induced by pairwise interaction strengths. The paper should either establish conditions under which such communities correspond to genuine higher-order interactions or weaken the interpretation accordingly.

Furthermore, for a returned community {A,B,C,D}, this does not distinguish between the different mechanisms that could have generated the cluster. The community therefore indicates that features are connected under the chosen pairwise interaction measure, but does not recover the form or order of their joint interaction. Permutation importance subsequently measures the predictive relevance of the group, not its interaction order. I think the manuscript currently uses “interaction structure,” “higher-order synergy,” and “feature community” somewhat interchangeably, although these concepts are not equivalent.

Finally, the proposed pipeline uses established components such as scalable approximate Friedman H-statistics, graph thresholding, Leiden/CPM community detection, and group permutation importance. The paper itself describes its primary methodological contribution as combining interaction estimation with community detection. I can see the practical value in this composition, but it is unclear to me that it constitutes a sufficiently substantial methodological contribution for ICLR without either stronger theoretical justification or a clearer demonstration that this particular construction enables capabilities unavailable to existing higher-order interaction approaches.

The final communities depend on the approximation parameters (M,R), the interaction threshold (τ), and the CPM resolution parameter (γ). The authors explicitly acknowledge that there is currently no definitive way to select optimal (τ) and (γ), and that these parameters must be chosen empirically. Given that the claimed explanation is itself the output of thresholding and clustering, I would like to see stronger evidence that the conclusions are robust to these choices, particularly on the real-world datasets where there is no known ground truth.

The paper motivates the post-hoc setting partly by arguing that inherently interpretable models may sacrifice predictive performance. I think this claim should be substantially rewritten. There is a substantial literature on self-explainable models in which interpretable representations or explanations are learned jointly with the predictor, including Self-Explaining Neural Networks (Alvarez-Melis & Jaakkola, 2018), ProtoPNet (Chen et al., 2019), ProtoVAE (Gautam et al., 2022), and Pantypes (Kjærsgaard et al., 2024). In particular, ProtoPNet reports accuracy comparable to its analogous non-interpretable counterpart, while ProtoVAE explicitly reports obtaining self-explainability without degrading predictive performance. More recently, Bassan et al. (2025) similarly learn sufficient explanatory feature subsets as an integral part of the predictor while maintaining comparable predictive performance. These approaches do not necessarily address the same feature-interaction problem as SINC and hence are not direct experimental baselines. Even in graph learning, recent intrinsically interpretable methods learn community- or archetype-based representations directly while retaining competitive predictive performance (e.g., AICoG and GraphHull; Nakis et al., 2026). Nevertheless, they weaken the implied dichotomy between accurate opaque prediction and interpretability-by-design, and suggest that the motivation for requiring a post-hoc approach should be stated more carefully.

Questions for authors

please see weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer SU5y

We thank the reviewer for the thorough review, the insightful examples, and the positive remarks regarding the clarity of the paper and the diagnostic experiment.

"graph communities and higher-order statistical interactions are different mathematical objects [...] The paper should either establish conditions under which such communities correspond to genuine higher-order interactions or weaken the interpretation accordingly."

We agree with this assessment and we have revised the manuscript as suggested. Specifically, we have weakened the interpretation and now present SINC as a method for identifying global communities of mutually interacting features, rather than as a method for directly discovering higher-order interactions. The detected communities are therefore intended as an interpretable representation of groups of features that interact with each other, without claiming that they precisely correspond to higher-order interactions. This clarification is now stated explicitly in Section 3.2 and further discussed in the Limitations section.

To further address this concern, we have replaced the Leiden community detection algorithm with maximal clique detection. Under this revised formulation, every pair of features within a community must show a detected interaction. As noted in the revised manuscript, this change does not imply that a detected community constitutes a higher-order interaction, but it provides a representation that is more closely aligned with the underlying interaction structure by ensuring that all features within a community are connected through direct interactions, rather than being grouped merely through indirect connections. Consequently, the identified communities offer a more principled characterization of globally interacting feature groups.

"Considering the interaction f(A,B,C)=ABCf(A,B,C)=ABC with independent centered variables [...] higher-order interactions can completely vanish under the pairwise marginalisation."

We agree and now discuss this limitation explicitly in the manuscript and have included it as an experiment in the revised manuscript. We also clarify that several pairwise interactions and one higher-order interaction can produce the same detected community.

"for a returned community {A,B,C,D}\lbrace A,B,C,D \rbrace, this does not distinguish between the different mechanisms that could have generated the cluster [...] Permutation importance subsequently measures the predictive relevance of the group, not its interaction order. I think the manuscript currently uses 'interaction structure,' 'higher-order synergy,' and 'feature community' somewhat interchangeably"

We agree on all points. The revised manuscript now makes clear that the importance assigned to a community reflects its predictive relevance to the model, rather than the order or form of the interaction among its features. We also revised the terminology throughout the paper, using "community" consistently to denote the output of SINC.

"it is unclear to me that it constitutes a sufficiently substantial methodological contribution for ICLR"

We agree that SINC is a composition of established methodological components. We nevertheless view the contribution as residing in the resulting capability: a practical, model-agnostic framework that discovers global and deterministic communities of interacting features for any trained model and ranks them by predictive impact. This further enables a new form of model diagnosis, allowing the assessment of whether a model has learned expected interaction patterns. By revealing coherent groups of mutually interacting features at a global level, SINC takes a step toward richer characterization of interaction structure in black-box models.

"The final communities depend on the approximation parameters (M,R)(M,R), the interaction threshold (τ)(\tau), and the CPM resolution parameter (γ)(\gamma) [...] I would like to see stronger evidence that the conclusions are robust to these choices"

With the adoption of maximal clique detection instead of Leiden communities, the CPM resolution parameter (γ\gamma) is no longer needed, leaving τ\tau as the only parameter for community detection. We have discussed its role in more detail in the revised manuscript.

"the motivation for requiring a post-hoc approach should be stated more carefully"

We thank the reviewer for these references and agree that the dichotomy was overstated. We have revised the motivation, which now states that self-explaining models can achieve predictive performance comparable to their non-interpretable counterparts, and that the need for a post-hoc approach arises when an already trained or deployed model must be explained. We now cite SENN, ProtoPNet and ProtoVAE in this context.

I would like to thank the authors for their response.

Unfortunately the changes described are not implemented in a revised manuscript that ICLR allows the authors to upload.

Therefore, I am unable to critically assess the changes that seem to be of a major revision status.

I think the paper is moving towards the correct direction, but unfortunately I am unable to raise my score based on the current version of the paper.

Nevertheless, I wish all the best to the authors and I will be interested to reading the paper again in its revised form when this becomes available.

We thank the reviewer for the response and for the interest. We apologize that the revised manuscript was not available. It has now been uploaded with the changes described in our response. The experimental results are being recomputed with the revised method and extended with a comparison to SHAP-IQ, and we will upload a further revised version shortly.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their time and for a constructive discussion, which has substantially shaped how we view and present our work. Below we summarize the strengths noted by the reviewers, the main concerns raised, and how we have addressed them.

Reviewer SU5y found the goal relevant, and Reviewer dJQh found the general goal of finding global interaction structure interesting. Reviewer TrbE noted that SINC offers “a straightforward way to organize pairwise interaction scores into global feature groups”, which makes the results “easier to inspect without modifying or retraining the target model”. Reviewer SU5y noted that the global partition “can be easier to interpret than a large matrix of pairwise interaction scores”. Reviewers SU5y and TrbE also found the synthetic experiments useful for inspecting the method, and highlighted the comparison of converged and undertrained models as a useful diagnostic that reveals differences “not apparent from predictive accuracy alone”.

The main concern, shared by all three reviewers, was that communities built from pairwise interactions do not in general correspond to higher-order interactions. We agree with this assessment and have revised the interpretation accordingly. SINC is now presented as a method for identifying global communities of mutually interacting features.

To further address this concern, we have replaced the Leiden community detection algorithm with maximal clique detection, so that every pair of features within a community must show a detected interaction. This provides a representation that is more closely aligned with the underlying interaction structure.

Following Reviewer dJQh, we added SHAP-IQ as a baseline. On the synthetic functions F1–F10, SINC recovers 72.0% of the ground-truth terms compared with 37.7% for SHAP-IQ, and 83.3% compared with 37.7% when the methods are applied to the generating functions directly. SHAP-IQ assigns the effect of each feature set to its pure interaction component, and since the higher-order components of most benchmark terms are small, it recovers only one higher-order term. On the California Housing and Bike Sharing datasets, SHAP-IQ identifies the same dominant interactions as SINC. However, SHAP-IQ requires the maximum interaction order to be chosen in advance, and it must score every feature set up to that order, which quickly becomes computationally demanding as the number of features grows. On the HIGGS dataset, with 28 features, computing SHAP-IQ with max order five was therefore not feasible within our available compute and time, and at order three every feature set contributes less than 2% of the total, so that no grouping of the features emerges.

The remaining comments are addressed in our individual responses to each reviewer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.