acceptodds
Under review as a conference paper at ICLR 2027

TopoSHAP: Shapley for Explainable and Efficient Topological Deep Learning

Abstract

Topological deep learning (TDL) extends learning on graphs, such as molecules or social networks, to higher-order structures, such as rings in a molecule or groups in a social network. Explaining which of these structures drive a model's predictions, and which architectural components matter most for its performance, remains challenging. We introduce TopoSHAP, an efficient and general explanation framework that addresses both questions across topological domains and model architectures. TopoSHAP-cell assigns Shapley values to topological data cells and extracts compact explanations. These explanations match or outperform graph-based counterparts on GraphXAI benchmark data while running up to 112 faster. They also reveal advantageous topological feature design, leading to measured improvements in explanation recovery and predictive accuracy. On the architecture side, TopoSHAP-neighbor explains how neighborhoods contribute to model performance, and turns these evaluations into a backward-elimination search that identifies smaller, competitive architectures early in training. This is up to 11.3 cheaper than traditional TDL training sweeps.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Topological Deep Learning, Explainable AI, Graph Neural Network, Shapley, XAI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zzhxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TopoSHAP, a cooperative game-theoretic framework for explaining the predictions of topological neural networks and the contributions of their neighborhood modules. TopoSHAP-cell treats cells of different ranks as players, constructs coalitions through masking, and computes Shapley attributions, while using a separate greedy elimination procedure to generate compact explanations. TopoSHAP-neighbor evaluates the contributions of neighborhood modules and generates candidate architectures using masked validation performance after five epochs of training the full model. The authors evaluate explanation quality on GraphXAI, lifted topological datasets, and MANTRA, and assess computational cost and predictive performance through architecture-selection experiments across multiple datasets.

Critical strengths
  1. The paper extends the objects of analysis to cells of different ranks and neighborhood modules within the network. It explicitly defines masking, neighborhood reconstruction, and coalition evaluation procedures, supporting different model families, including GCCN and HOPSE.
  2. The paper connects explanation analysis with model design and provides a relatively extensive empirical evaluation.
Critical weaknesses
  1. FORGE already generates cell-level explanation masks as an intermediate step, whereas this paper primarily emphasizes that its final outputs are node-level attributions, without sufficiently discussing the overlap and differences between the two approaches to higher-order explanations. Furthermore, the main graph-explanation experiments lack comparisons with recent, directly relevant methods such as FORGE and GraphSHAP-IQ. Consequently, the current results do not sufficiently establish TopoSHAP’s additional contributions in explanatory capabilities, explanation quality, and computational efficiency.
  2. Cells of different ranks exhibit structural dependencies and overlapping information. Treating them as independently maskable players therefore affects how the resulting attributions should be interpreted. In Table 4, the relative performance of detachment and feature zero-filling reverses across backbone models, further indicating substantial sensitivity to the choice of intervention.
  3. For architecture selection, the practical pipeline relies on masked scores after five training epochs. However, the optimal-architecture recovery results in the exhaustive evaluation use oracle scores obtained from fully trained architectures, which do not directly validate the reliability of the early-stage scoring mechanism.
  4. Qualitative case studies of explanation quality are limited. Although Figure 3 presents attribution distributions on MANTRA, the paper primarily evaluates explanation quality through quantitative metrics and lacks visualizations that map the explanations back onto the corresponding input structures.
Questions for authors
  1. FORGE already introduces intermediate cell-level explanation masks. Could the authors clarify the overlap and differences between TopoSHAP and this mechanism?
  2. How closely do the masked scores after five training epochs agree with the performance of candidate architectures after continued training? On the existing exhaustive benchmark, how often does the practical low-cost pipeline include the optimal architecture among its candidates, and how does its final selection performance compare with the oracle?
  3. Table 4 shows that, on Benzene, replacing the default detachment strategy with feature zero-filling increases the explanation score for GCN from 0.377 to 0.648, but decreases the score for GIN from 0.969 to 0.569. A similar reversal occurs on Mutagenicity. Could the authors explain this phenomenon and discuss its implications for the applicability of the default masking strategy and the conclusions drawn from the explanations?
  4. The paper primarily compares against established methods such as GNNExplainer (2019), PGExplainer (2020), and SubgraphX (2021), while omitting more recent relevant approaches already discussed in the paper. Could the authors include comparisons with recent relevant methods?
2 comments
AuthorsOpenReview ↗

FORGE Comparison, Continued Training, and Visual Explanations

We thank the reviewer for their helpful feedback. These clarifications and additional experiments strengthen our original contribution: extending explainability to TNNs.

1 What does TopoSHAP add beyond FORGE? | W1 Q1 Q4

We acknowledge that while FORGE does use intermediate cell masks in order to explain graph-level outputs; TopoSHAP markedly and definitively extends beyond FORGE (Sec. 2):

  • TopoSHAP-cell explains configurable TNNs directly over their supplied higher-order cells, across any given topological lifting. FORGE explains GNNs in its restricted lifted-graph framework and returns graph explanations.
  • TopoSHAP-neighbor explains which architectural neighborhoods contribute to a trained TNN’s performance and supports selecting combinations for a specific task. FORGE provides neither neighborhood attribution nor architecture selection.

Following your request, we add two direct comparisons and an exact Shapley reference:

  • GStarX: TopoSHAP attribution achieves lower deletion-AUC on five of eight dataset/backbone pairs under node removal, using full evaluation populations (App. F.1).
  • Exact reference motivated by GraphSHAP-IQ: we enumerate Shapley values with excluded nodes’ features zeroed and connectivity preserved, on 207/1,302 molecules with at most 18 nodes. On matched models and molecules, TopoSHAP has higher mean Fidelity+ on six of eight pairs under node-removal evaluation. This tests attribution quality, not GraphSHAP-IQ’s implementation (App. F.8).
  • FORGE-GNNExplainer: on the same FORGE-trained GIN/GCN models, molecules and atom budget (four datasets, three seeds, up to 100 molecules/run), TopoSHAP with cells as players achieves higher mean Fidelity+ and lower mean deletion-AUC in all eight settings under atom removal, and higher mean motif recovery in seven. Deletion-AUC is lower in all 24 runs. FORGE leads in motif recovery on Alkane-Carbonyl/GCN (0.537 versus 0.459). TopoSHAP takes 2.1–2.9× FORGE’s explanation time under shared GPU load (App. F.10, Tables 40–43).

2 Why does the way we remove cells matter? | W2 Q3

We agree that removal choices affect explanations. Separately masking cells does not assume statistical independence: attribution reflects model behavior across combinations of cells (Sec. 4.1). Zero-fill retains connections, while detachment removes cells from computation, changing information flow.

We reproduce the reported GIN/GCN reversal after retraining: zero-fill improves greedy motif recovery for GCN but reduces it for GIN on both Benzene and Mutagenicity (three seeds; Tables 15–16).

Greedy GEADetachmentZero-fill
Benzene GIN0.890.65
Benzene GCN0.390.58
Mutagenicity GIN0.690.32
Mutagenicity GCN0.330.57

An explanation must therefore specify its intervention. Detachment measures reliance on supplied cells; remove-and-re-lift rebuilds higher-order inputs after base-node removal (Sec. 4.1; App. F.2).

3 Do scores after five epochs predict final performance? | W3 Q2

We ran 198 continued-training experiments: six architectures formed by removing one neighborhood at a time, across 11 settings and three seeds. Each resumes from its five-epoch checkpoint (App. F.9).

Selected models achieve similar mean performance to continuing the full model, often retaining fewer neighborhoods:

ResultClassification (30 runs)ZINC (3 runs)
Selected − full-model mean test performance+0.29 percentage points accuracy+0.0015 MAE (worse)
Fewer neighborhoods retained16/301/3
Early/final Spearman correlation: validation / test0.47 / 0.410.89 / 0.90

Can early scores identify the better candidate? They correctly order the two candidates in 12/24 untied classification runs and 1/3 ZINC runs. Our pipeline trains both further and chooses using continued validation scores. The correlations cover only six architectures chosen using early scores.

How often do we find the benchmark optimum? Against the 64 architectures trained independently from scratch, our two candidates include one within 1σ of the best in 22/33 runs, including the exact best in 5/33. The final selected architecture meets this tolerance in 16/33. Here, σ is the median standard deviation across training seeds for the 64 architectures. In this benchmark, selected architectures average 1.01 percentage points below the best classification accuracy and 0.0180 higher ZINC MAE (Table 39).

The submission already includes continued training and validation-based selection (Sec. 6.1, Table 9).

4 Visualizing explanations on inputs | W4

We add Figure 6 (p. 33) to complement MANTRA: signed atom/bond/ring attributions, greedy selections and annotated motifs on Benzene and Mutagenicity. It shows the best and worst structure-only GCCN cases among ten inspected molecules per dataset (Sec. 5.2), with interpretation in App. F.1.

Thank you for the detailed responses and additional experiments. The direct comparison with FORGE and the continued training experiments strengthen the empirical evaluation and address some of my concerns.

However, some issues remain insufficiently resolved. First, reproducing the GCN/GIN reversal confirms the method’s sensitivity to the choice of intervention, but its cause remains unexplained—at least, I could not readily find such an explanation in the revised manuscript (because the authors did not mark the revisions in colored text). In addition, Figure 6 provides useful qualitative examples, but its readability still needs improvement. Although the color meanings are stated, the text is too small. A color bar with numerical ticks would help readers interpret the results.

Overall, I appreciate the authors’ efforts to improve the paper, but these additions are not sufficient to change my overall assessment. I will therefore maintain my original score.

Reviewer_YbirOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents TopoSHAP, which is a new value function for Topological Deep Learning (TDL). TDL is a, rather niche, special variant of tackling graph-based learning where higher-order structures (complexes and cells) are modeled directly. Therein, this paper actually proposes two value functions: the local attribution setting (called TopoSHAP-cell) and the global explanation setting (called TopoSHAP-neighbour). For both value functions, the paper describes how it is defined and how to compute it with existing Shapley methods. The empirical evaluation then compares it to existing baselines in the graph explanation domain.

Critical strengths
  1. The contribution of adding the new value functions is good: While I think this paper is quite thin of a contribution, overall, I do not think it is a bad thing to have well defined value functions for well-defined model-classes. TDL seems to be an active research field at top-tier venues and the models seem to tie with classical graph-based learning, so having specialized machinery for model interpretation is absolutely warranted. I like such kinds of contributions.

  2. The paper is clearly written and I was able to learn a lot from it. The background on TDL was good for me (and most readers from XAI I suppose), and the related work of Shapley and XAI for graphs also contains all important works (second thought: maybe a bit light on the interaction side).

Minor:

  1. The paper's code is part of the submission (sorry that this is a strength, but it has become one), which was helpful for looking through to understand the empirical evaluation better.
Critical weaknesses

I genuinely think this paper can be quite good and I was contemplating about the final score, but ultimately as it stands now I think I have to opt for a regular rejection. Overall, I see two main weaknesses holding this paper back currently:

1. Very limited contribution:

This paper only presents a new value function. Breaking everything down, the only contribution is that players are now cells inside a TDL environment. While I do not think that this is per se a dealbreaker, it is very little for a venue like this. The paper, does not offer any computational improvements over existing Shapley methods, nor does it connect (lean into) the existing state of the art. Below I list a some issues I see that offer very natural room for future work:

First, the estimator used for TopoSHAP is the very outdated permutation sampling baseline, which is outperformed across the board by more sophisticated methods like KernelSHAP [5], LeverageSHAP [3], or the new line of proxy-based estimators such as RegressionMSR [1] and ProxySHAP [2] that currently set the state of the art. Particularly, the latter methods require an inherently interpretable proxy-model (usually trees), which fit heavily dictates the performance of the estimator (a good proxy-model fit is a good estimation quality). Hence, analyzing whether one could get good estimates with these approximators would be a good start to make this contribution more meaningful. This is for TDL not so straightforward maybe and might even lead to nice performance improvements.

Second, the current paper only looks at first order explanation objects (I am using explanation interchangeably with attribution here and do not mean this papers distinction between the twos: i.e. output 1 and 2). However, in the recent time more and more methods and extensions for higher-order interactions have been proposed, some of which are already cited in the related work. For example, the Shapley interaction index (Grabisch and Roubens, 1999) or GrapSHAP-IQ (Muschalik et al., 2025). Since TDL models higher-order structures directly in the machine learning model, establishing a connection between interactions (or Joint Shapley values, grouped explanations) seems reasonable. Also, take GrapSHAP-IQ as an example: GrapSHAP-IQ is just an algorithm that creates a sparse Möbius representation of the underlying game and then transforms this Möbius representation into lower-order explanation concepts such as attribution or interaction scores. This should be transferrable to TDL and this setting as well. That would offer a computational improvement. A similar idea of these sparse Möbius representations exists outside of GNNs in explaining causal graphs [5].

Third, the explanation (output 2, lines 261-266 and lines 294-299) is very heuristically. Having a value function allows you do this, I understand, but I do not really see the point or any theoretical guarantee of doing this. While not perfect, already having the attribution scores can help making a better selection. Other game-theoretic concepts, such as interactions, possibly Banzhaf interactions, are probably more helpful, as has been done for example with FIxLIP [6].

Fourth, I think it is not so clear how to model the removal aspect of these new value functions. Currently all is modeled with detachment (deleting cells), but I think this does not make sense in every explanation setting TopoSHAP would like to be used. See Q1 for this.

2. Not so convincing results:

Moreover, the empirical evaluation and showcase does, for me at least, not show a convincing case why the explanations proposed here are so meaningful. It feels like, the evaluation done in the existing experiments is comparing apples with oranges, as there are no TDL baselines (because this paper here seems to be the first one). While showing that existing non-TDL baselines fail in the TDL case is good, but also expected as this arises from construction. I would personally be way more interested in seeing how the explanations / attributions created from this new value function helps practitioners identify interesting properties or learn something about the TDL problem at hand. I do not think that the current experiments convey this. Maybe some more "ground-truth" explanations could help. For example, there is this B-XAIC dataset which is a graph-based dataset with molecules and known ground truth explanations. This could serve as a better environment to showcase TopoSHAP's capabilities.

Minor:

  1. The related work on Shapley Interactions (or game-theory interactions) is actually a bit small. This usually is not an issue for XAI / attribution papers, but this paper is on TDL, which aims to model higher-order structures directly as input features. There interactions play a crucial part. A more careful description and delineation of existing game theoretic contepts such as Banzhaf Interacitons, Möbius / Fourier representation of the game, Faithful Shapley interactions, or the Shapley Taylor interaction index would have been better.
  2. This might be just me, but the speed improvements do not really interest me for a submission like this.

References

Questions for authors

I suggest the authors to continue working on this and hopefully be able to connect this work more with the existing state-of-the-art on game-theoretic explanations.

Q1: You model player/feature removal with detachment, i.e. removing a cell. This seems reasonable in cases where you want to do local explanations of what the model actually uses (like remove a person from a social network group, or remove a hyper edge). But if you are interested in explaining what substructures of the data explain the prediction, (like rings in a graph) is this then not a wrong removal strategy? If you keep a ring without its atoms then this makes no sense, no? Would it then not be better to implement kind-of a remove and re-lift strategy, where you define the game on the base graph, remove nodes, then recompute the lifting and the lifted features and then evaluate the model. Missingess would then follow from how the data was generated and a face would only exist if each of its atoms was part of the coalition. This connects to the discussion in the XAI literature about On- and Off-Manifold explanations (like true to the data or true to the model).

2 comments
AuthorsOpenReview ↗

Response to Ybir: Stronger Shapley Estimators, Interaction Analysis, and Additional Baselines

We thank the reviewer for their helpful feedback. These clarifications and additional experiments strengthen our original contribution: extending the explainability game-theoretic toolkit to TNNs.

1 Contribution and use | W1 W2 M2

We respectfully disagree that the contribution is only a new cell value function. TopoSHAP contributes two explanation games (Figure 1) and demonstrates their practical values:

  • TopoSHAP-cell reveals insights about topological lifting design (Table 2) and learning on native topological data (Sec. 5.3). On GNNs, its greedy explanation achieves GEA 0.969 versus SubgraphX’s 0.796 on the same Benzene/GIN models (Table 4). Added GStarX comparisons show higher motif recovery for TopoSHAP attribution on all 4 GIN datasets (Tables 15–16).
  • TopoSHAP-neighbor explains performance with neighborhoods and thus cuts down on traditional TDL training costs up to >11x (Sec. 6.1). This is meaningful for a field bottlenecked by compute (Papamarkou et al, Position: Topological Deep Learning. 2024.).

2 Removing cells | W1 Q1

We implement the reviewer's proposed remove-and-re-lift alternative (App. F.2):

TestRemovesQuestion answered
DetachmentA cellDoes the model use this cell, keeping other cells’ original input features fixed?
Remove-and-re-liftGraph nodesWhat changes if we remove nodes and rebuild the higher-order input?

Re-lifting improves attribution motif recovery in 7/8 and greedy motif recovery in 5/8 dataset/lifting pairs on the same GCCNs: 100 Benzene, 100 Fluoride-Carbonyl, 96 Mutagenicity and 58 Alkane-Carbonyl molecules, with two liftings and three seeds.

3 Stronger Shapley estimators | W1a

Following the reviewer's suggestion, we compare eight estimators on NCI1, NCI109, MUTAG and PROTEINS: 15 complexes each, reused across three structure-only GCCN checkpoints (60 distinct complexes, 180 model–complex pairs; App. F.4).

  • Exact reference: restricted 13/16-player games, mostly vertices and edges; other cells stay present.
  • Budget: results below use a cap of 2,048 distinct coalition evaluations, five estimator seeds and top-six precision.

Pooled results (six estimators shown; all eight in Table 30):

EstimatorMean squared error ↓Ranking agreement ↑Top-six precision ↑
Antithetic permutation0.00140.9730.943
Paired KernelSHAP0.00060.9790.954
Leverage SHAP0.00060.9800.951
Unpaired KernelSHAP0.00120.9720.941
SVARM0.00170.9780.945
Plain permutation0.00220.9700.938

Paired KernelSHAP reduces MSE about 2.4×, while ranking agreement remains high (0.973→0.979). This improves numerical accuracy without changing our explanation framework.

4 Joint contributions | W1b M1

We add the requested interaction analysis: exact pairwise interactions on 100 Benzene molecules and three GCCN seeds per lifting. We evaluate all 8,192 combinations of one ring, six bonds and six atoms, keeping other cells present (App. F.3).

Mean pairwise interactionStandard liftingStructure-only lifting
Ring–boundary bond+0.13+0.38
Bond–endpoint atom+0.08+0.66
Adjacent boundary bonds−0.12+0.67

We thank the reviewer for the helpful feedback - these results reveal how feature lifting changes joint contributions. We find adjacent boundary bonds reinforce each other on average with structure-only features. This complements well our framework of cell attribution, which assigns credit to individual players, including higher-order cells like rings.

Sec. 2 adds Shapley–Taylor, Faith-Shap, Banzhaf and SHAP-IQ/GraphSHAP-IQ. App. F.5 explains why sparse computation needs model-specific locality checks.

5 Why retain greedy pruning | W1c

We agree that greedy pruning has no optimality guarantee, as acknowledged at submission in Sec. 4.1: “Greedy elimination carries no optimality guarantee.”

Its empirical value is shown in Table 2: greedy pruning achieves higher mean motif recovery than attribution ranking in six of eight dataset/lifting pairs. On structure-only Benzene, GEA increases from 0.471 to 0.725, supporting its inclusion as a complementary tool.

6 Why are these comparisons fair/meaningful? | W2

The graph comparisons use the same GNNs for TopoSHAP and existing graph explainers (Table 4). Those explainers do not automatically work on topological complexes. Because more baselines are always helpful, we adapt GNNExplainer to mask cells at every rank and evaluate it on the same retrained GCCNs. TopoSHAP achieves the lowest mean deletion-AUC and highest mean insertion-AUC among all 5 methods on all 4 datasets (Table 26). Insertion tests how quickly highly ranked cells recover confidence when added to an empty complex.

We also add Figure 6, with analysis, to illustrate cases of successful and failed motif recovery.

Reviewer_YbirOpenReview ↗

Overall, I still stand by my initial judgment of this paper.

Overall, I still stand by my initial judgment of this paper and think that it needs to be substantially reworked and streamlined. This has multiple reasons including:

  1. Now, the paper has 10 pages (45 pages in total) with 43 Tables, 6 Figures, and an Appendix F containing 10 sub-sections of disjunct experiments. It is not clear at all what has changed from the original submission to the revised one and the paper is very long now.

From your answers I am also not yet convinced:

  1. There is no novelty in the experiment with the Shapley estimators. You just ran different estimators that are better than the one you had initially and you get better performances in Shapley estimation. This is known and there is no implication for your work. With my comment regarding the proxy-based estimators, I was mainly suggesting that you could rethink your topological structure as something that could help designing a new proxy-based estimator for your domain and actually strengthen the contribution.

  2. Figure 6 could be interesting, but I do not understand it. Regardless, the line of Shapley interactions analysis and structure analysis is in my opinion integral for the whole story of the paper and should not be deferred to some sub-appendix.

  3. Your answer on the heuristic selection "Why retain greedy pruning | W1c" is also not convincing. I do not think greedy-selection is per se an issue. It is just not motivated at all, apart from having empirically some nicer properties in light of some evaluation metric. There is still no theoretical grounding.

  4. Regarding your statement under "1 Contribution and use | W1 W2 M2", I apologize for being unclear in my wording, but the point still stands as the paper just presents a value function in the TDL environment where players are cells or topological structures without new game-theoretic or computational methodology (I even say so myself in the summary of the paper "[...] Therein, this paper actually proposes two value functions [...]").

Note, that I still like the core contribution (see strengths) and do not want to come off as I do not, but there is from my perspective still too little development and connection to existing literature.

Reviewer_6vbyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces TopoSHAP, an explanation technique designed for TNNs. They propose a double use of the technique: explaining models to identify important components of the input; guide model training to reduce training times by only focusing on necessary components (neighborhoods).

The approach is interesting as no previous work focuses on this type of network. Also, results are promising.

Critical strengths

The results support Topo SHAP as the best among different baseline.

Using TopoSHAP provides effective results in explaining and in "Meta Learning"

The authors provide the code to replicate the explanations.

Critical weaknesses

A big part of the evaluation checks alignment with ground-truth substructures . An explanation should first be evaluated for the faithfulness using standard metrics (Fidelity, Inv Fidelity): if a model learns a spurious bias, the correct explanation should reflect that bias. Only after confirming model alignment, comparisons to chemical ground truth are meaningful. Let me cite Agarwal et al, 2023: "trained GNN model may only capture one or an entirely different rationale. In such cases, evaluating the explanation output by a state-of-the-art method using the ground-truth explanation is incorrect because the underlying GNN model does not rely on that ground-truth explanation. In addition, even if a unique ground-truth explanation generates the correct class label, the GNN model trained on the data could be a weak predictor using an entirely different rationale for prediction. Therefore, the ground-truth explanation cannot be used to assess post hoc explanations of such models."

It is not clear why when the authors evaluate deletion AUC, the baselines change. GNNExplainer could have been used also in the case of deletion AUC, and GradXInput could have been used for Explanation Accuracy.

The neighborhood experiments are indeed performing Meta Learning but the authors do not compare with any Meta Learning technique.

Some baselines using SHAP are missing: GStarX beats SubgraphX and it is not used in the paper.

No visual analysis of the explanations. This could offer understanding of why some models fail.

Questions for authors

Can you provide more in depth analysis of the explanations?

Can you add GStarX as a baseline?

Can you provide some Explanation Visualization?

Can you compare with some Meta Learning baselines?

2 comments
AuthorsOpenReview ↗

Response to 6vby: Additional Baselines, Faithfulness Experiments, and Visual Analysis

We thank the reviewer for their time and comments. We believe that the clarifications and additional experiments presented in this rebuttal strengthen our original contribution. We address the reviewer's concerns below and in the updated submission PDF.

1 Do explanations reflect the model’s decisions? | W1

We agree that a faithful explanation should reflect what the model uses, even when it has learned a spurious pattern. Our submitted paper separates this question from motif recovery: Table 2 shows that models can assign higher prediction scores to other substructures than to the annotated motif, while Table 7 directly evaluates faithfulness through deletion-AUC on TNNs. We strengthen this evaluation with more faithfulness metrics and baselines below.

Table 7 already evaluates faithfulness on TNNs with deletion-AUC: it tracks how quickly predicted-class probability falls as important cells are removed. This applies the same removal principle as Fidelity+ across multiple budgets.

Following the reviewer's request, we also add insertion-AUC on the TNNs used for our expanded baseline comparison (higher is better).

Across NCI1, NCI109, MUTAG and PROTEINS, TopoSHAP achieves the highest mean insertion-AUC and lowest mean deletion-AUC among all five evaluated methods. Both metrics use the same models, attribution rankings and 50 test complexes per dataset (47 for MUTAG), across three seeds (results below).

Following the reviewer's recommendation, we add direct faithfulness tests on the graph benchmarks: Fidelity+ measures the confidence drop after removal (higher is better); Fidelity− measures the drop when only the explanation is kept (lower is better). We also report GEF and deletion-AUC under detachment and zero-fill (Sec. 5.1; Tables 18–25). We find that across 8 dataset/backbone pairs, attribution is within 0.02 of the best Fidelity+ and has the lowest deletion-AUC on five.

2 Why the baselines differed, and what we add | W2 W4 Q2

The original baselines differed because they explain different model types: the motif tests explain GNNs, while the complex deletion tests explain topological NNs. These NNs are not interchangeable.

We agree with the reviewer that additional baselines are helpful and therefore adapt GNNExplainer to TNNs, learning cell masks at every rank with a logit-margin objective. We also add insertion-AUC, testing how quickly model confidence recovers as highly ranked cells are added to an empty complex. We find that TopoSHAP achieves the highest mean insertion-AUC and lowest mean deletion-AUC among all five methods on all four TNN datasets. (Sec. 5.3; Table 26).

Insertion-AUC ↑ / Deletion-AUC ↓NCI1NCI109MUTAGPROTEINS
TopoSHAP0.859 / 0.2110.796 / 0.2170.845 / 0.1750.849 / 0.368
Cell-mask GNNExplainer0.596 / 0.4970.576 / 0.4880.640 / 0.4360.688 / 0.559
Occlusion0.818 / 0.2440.768 / 0.2450.783 / 0.2150.806 / 0.455
Gradient×Input0.549 / 0.5910.503 / 0.5220.489 / 0.5340.653 / 0.651
Random0.576 / 0.5510.513 / 0.5140.556 / 0.4940.670 / 0.672

For the GNN comparisons, we add gradient and occlusion to motif evaluation, with random, and GStarX to motif and faithfulness evaluation. GStarX and SubgraphX cover full populations across three seeds (App. F.1). TopoSHAP attribution recovers motifs better than GStarX on all 4 GIN datasets and 3 of 4 GCN datasets. (Tables 15–16).

3 Is this meta-learning? | W3 Q4

We clarify that the neighborhood experiments perform task-specific architecture selection, not meta-learning across tasks. As Sec. 6.1 states, “the procedure is architecture selection in the spirit of low-fidelity search.” We now explicitly state in Section 6.1 that the procedure “selects an architecture for one task and learns no cross-task adaptation rule.”

We address the request for relevant baselines by adding two single-task architecture selection methods (details in App. F.7):

  • Successive halving (Jamieson and Talwalkar, 2016): briefly trains candidate architectures and repeatedly discards the worst half. TopoSHAP-neighbor achieves higher mean accuracy on six datasets, lower on four and comparable on one

  • Differentiable neighborhood selection (inspired by DARTS; Liu et al., 2019): learns a weight for each neighborhood’s output and accordingly selects architectures. TopoSHAP-neighbor achieves higher accuracy on six and lower on five

The baselines use 0.5–1.8× TopoSHAP’s total training FLOPs.

4 What do the explanations teach us? | W5 Q1 Q3

We highlight Secs. 5.2–5.3 of the submission that demonstrate how we can use TopoSHAP explanations to improve features and extract topologically-native (MANTRA) learning insights. We now add additional detail about the exact interaction results in Benzene (see Table 29) and provide visual analysis in Figure 6 of success/failure cases selected by the structure-only GCCN.

Thank you for the rebuttal and the additional experiments. The authors replied to my comments, and I accept the clarification that the neighborhood experiments are architecture selection, not meta-learning.

However, the results do not clearly show TopoSHAP as the best method: it is competitive but not consistently ahead on faithfulness and architecture selection. I will keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Closing response and summary of manuscript revisions

We thank the reviewers for their input and feedback throughout this process, as it led to stronger and clearer empirical evidence of TopoSHAP's contribution (more tests of faithfulness, more baselines, more evaluations and variations on architecture selection). We understand the remaining concerns to center on three questions: where the novelty lies, how faithfully TopoSHAP explains a trained model, and how useful those explanations are for designing TNNs. For brevity, we address them below.

What does TopoSHAP contribute to topological deep learning? We agree with Reviewer Ybir that TopoSHAP builds on established Shapley methods. Our contribution is to define two interventions specific to TDL. TopoSHAP-cell lets us test whether a model relies on a molecular ring as a higher-order cell while its constituent atoms and bonds remain available. TopoSHAP-neighbor lets us test which message-passing neighborhoods contribute to performance on a given task. We believe that making these representation and architecture choices directly testable is a methodological contribution to TDL in its own right. Advances in xAI can then improve the estimates for these games. For example, paired KernelSHAP reduced mean squared error by a factor of 2.35 relative to antithetic permutation sampling on the restricted cell games in Table 30.

Are the cell explanations faithful, and what do they teach us? Reviewer 6vby is right that rankings vary on the graph *benchmarks*. However, we emphasize that the motivation of this work is performance on *topological* benchmarks. There, TopoSHAP achieved the highest mean insertion-AUC and lowest mean deletion-AUC among five methods on each of four datasets, including against existing baselines adapted to higher-order complexes (Table 26). By virtue of its topological domain, TopoSHAP also allows us to stufy the impact of different topological liftings on explanations (Table 29), in line with our contribution of extending xAI to TDL specifically.

How useful are neighborhood explanations for architecture selection? We agree with Reviewer 6vby that TopoSHAP-neighbor is competitive with other low-cost searches. It achieved higher mean accuracy on six of eleven datasets against successive halving and on six against differentiable selection (App. F.7). The practical value is that the neighborhood explanations narrow the search to two candidates after five epochs. On classification tasks, the final selections averaged 1.01 percentage points below the best of 64 architectures (App. F.9). The procedure reduced training cost by up to a factor of 11.3 relative to traditional TDL sweeps (Sec. 6.1).

Per Reviewers Ybir and zzhx's suggestions, we upload a revised version with the interaction finding in the main text (in blue text), a shorter appendix with clearly marked additions, and larger labels and a numerical color bar in Figure 6.

We hope that TopoSHAP can be evaluated for its contribution to TDL. For the first time, practitioners can test whether/how any topological model actually uses the practitioner-selected topological structures and outperforms available baselines. On the practical side, TopoSHAP-neighbor uses TDL-native explanations to execute cheaper and more principled training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.