acceptodds
Under review as a conference paper at ICLR 2027

WOMBAT: Whitebox Oracle for Molecular Benchmarking and Attribution Testing

Abstract

When a graph neural network (GNN) explainer produces an unexpected attribution on a molecule, the attribution alone cannot reveal whether the explainer has failed or the model has learned a shortcut. We introduce WOMBAT, a benchmark of 14 whitebox GNNs, each with message-passing weights set by hand to detect a specific SMARTS motif. Each model’s decision rule is known by construction, providing attribution ground truth against which explainer errors can be identified and studied. We validate the models on millions of PubChem molecules and evaluate post-hoc explainers including GNNExplainer, PGExplainer, and Integrated Gradients. Guided by our qualitative analysis, we construct a model that causes Integrated Gradients to spread attribution across the graph, even though the model reliably detects the intended motif. We release the dataset, models, and evaluation code.

Primary area:
interpretability and explainable AI
Keywords:
XAI, GNNs, chemistry

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_P2nUOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes to use GNNs with handcrafted weights and known ground-truth explanations to evaluate GNN explainers. If an explainer gives an unexpected explanation, it is usually not clear whethe this comes from a the explainer itself or whether actually the model learned something unexpected. Using the proposed approach, unexpected explanations can be attributed solely to the explainer, since the model-ground truth is known. The results show that the readout function used in the GNNs can have a huge impact on the performance of the explainers, especially on that of Integrated Gradients.

Critical strengths

The paper proposes to use GNNs with handcrafted weights and known ground-truth explanations to evaluate GNN explainers. Explaining model decisions gets increasingly relevant as they are used more and more in practice, and also for understanding what is actually relevant for certain tasks. The paper well written and easy to follow.

Critical weaknesses

None

Questions for authors

Q1: In the paper, you essentially evaluate a subgraph search task. Do you think the explainers performance will be similar on real-world tasks?

Q2: There are tables and figures referenced that ate not in the (main) paper, for example S6. Could you add a pointer to where these can be found?

Q3: Table 2 only shows whether the explainer achieved perfect score or not. It would be more interesting to see the actual numbers. Can you please provide them?

3 comments

We thank the reviewer for their review and for recognizing our contributions. We are pleased to hear that our paper is well-written and easy to follow.

Q1: Performance transfer to real-world tasks. WOMBAT cannot assert similarity of performance of explainers on other GNN architectures (which likely would be used in real-world tasks). We observe that preserving our architecture and merely changing weights can alter XAI performance (while the underlying models detect the same motifs), and change of architecture alters these results as well. WOMBAT should mainly be seen as a controlled environment in which we can analyse XAI method’s performance.

As for the applicability of subgraph search tasks in real-world tasks, chemists have been defining subgraph detection tasks as simple proxies for many properties. For example, pan-assay interference compounds (PAINS) are molecules that often produce false positive results in high-throughput screening [F]. A set of molecular patterns was proposed to quickly identify problematic structures. Similarly, some subgraph alerts were defined for mutagenicity and other toxicity-related endpoints [G]. It is also worth noting that in many molecular prediction tasks, tree-based models on molecular fingerprints remain competitive baselines [H], which are essentially sequences of decisions on extracted molecular subgraphs (fingerprint bits).

[F] Baell, Jonathan B., and Georgina A. Holloway. "New substructure filters for removal of pan assay interference compounds (PAINS) from screening libraries and for their exclusion in bioassays." Journal of medicinal chemistry 53.7 (2010): 2719-2740.

[G] Sushko, Iurii, et al. "ToxAlerts: a web server of structural alerts for toxic chemicals and compounds with potential adverse reactions." Journal of chemical information and modeling 52.8 (2012): 2310.

[H] Jiang, Dejun, et al. "Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models." Journal of cheminformatics 13.1 (2021): 12.

Q2: Figure references. We thank the reviewer for reporting this issue. These figures are in the Appendix (e.g. Figure S6 on page 27). In Adobe Acrobat PDF Reader, they are properly marked, and one can navigate to them. In general, clicking on the figure number should navigate to the appropriate figure. This could be an issue with a specific PDF reader; however – could the Reviewer try a different one to see if a problem persists? If the problem still occurs, we would be happy to hear more, reproduce the issue on our end, and fix it if possible.

Q3: AUROC table. The AUROC table was in the Appendix; we’ve moved it to the main text to make it more visible (it’s in blue in the revised manuscript). We thank the reviewer for this suggestion, as we think that the paper’s readability will benefit from this change.

I thank the authors for addressing my concerns. Having read also the other reviews and answers to them, my score remains unchanged.

AuthorsOpenReview ↗

Thank you!

We thank the reviewer for their time, for reading the full discussion, and for their continued support of WOMBAT.

Reviewer_9mkCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This study addresses a challenge in interpreting Graph Neural Networks (GNNs) applied to molecular graphs: when a GNN explainer produces an unusual attribution, it is often difficult to determine whether this stems from a failure of the explainer itself or from the model having learned a "shortcut." This "model-explainer ambiguity" makes it difficult to fairly evaluate the quality of an explanation. To overcome this, the authors created an environment where the ground-truth explanation is definitively known, allowing them to isolate and diagnose attribution errors made by the explainer. Specifically, they constructed 14 "white-box" GNNs, manually setting their weights to detect specific SMARTS motifs. This ensures that the decision-making rules for each model are known by design, effectively turning them into "oracles" where the ground-truth explanation—the precise motif the model must observe to output a prediction of 1—can be directly extracted.
  • To validate their approach, the authors verified that these white-box models themselves serve as a reliable ground truth, reporting perfect classification performance.
Critical strengths
  • This study makes a contribution by designing a benchmark that structurally eliminates model-explainer ambiguity.
  • Another strength is that it provides white-box models based on actual molecular structures, rather than synthetic data.
  • This benchmark set offers the advantage of not only comparing scores but also clearly illustrating failure modes by tracing the causes of failure at the level of the white-box model’s weights and operations.
Critical weaknesses
  • It is unclear how frequently the failure modes observed in WOMBAT occur in GNNs with different architectures.
  • While the study reports performance on a large-scale molecular dataset that might be described as "perfect," this does not constitute formal mathematical verification.
  • The evaluation involves ranking nodes by importance; however, since practical issues—such as setting thresholds or calibrating scores—remain unaddressed, the utility of the reported AUROC values ​​is open to interpretation.
  • There are inconsistencies that detract from the paper’s overall coherence; for instance, the capitalization style used for section titles in the appendix is ​​not uniform.
Questions for authors
  • Is it possible to evaluate the model on other GNN architectures and report its generalization performance?
  • Could the benchmark set also be provided in a format that involves predicting actual values, rather than just ranking nodes by importance?
3 comments

We thank the reviewer for their review and for recognizing the value of separating model failures from explainer failures. We address the concerns and questions below.

W1: Transferability of findings. We agree that investigating the prevalence of these failure modes across different GNN architectures is an interesting direction for future work. However, WOMBAT is designed to establish and diagnose explainer failures under controlled conditions, rather than estimate their frequency across arbitrary trained architectures. Our white-box construction deliberately removes model-explainer ambiguity, allowing us to study failure mechanisms that would otherwise be difficult to isolate. Our GINEConv experiment, where we create a whitebox for a different architecture (Appendix Section F.2), gives us a glimpse into this: a different architecture can substantially change XAI method performance. However, this phenomenon is not limited to architecture: even changing weights alone (while preserving motifs being recognised by the model) can lead to change in XAI method’s performance (Appendix Section F.2). In the end, WOMBAT should be seen as a unit test for XAI methods and not as a definitive answer to the question “which XAI method to use”.

W2: Formal mathematical verification. That is correct. We believe our validation pipeline to be sufficiently rigorous for the purposes of this benchmark. We would like to note that the weights of our networks were entirely human-made with a pen-and-paper approach and were made to be understood by a human. Consequently, their behavior can be inspected and reasoned about directly, rather than treated as a black box. Their actual implementations are relatively simple to comprehend and build intuition about, without formal proofs: our Appendix D explains our 2 whiteboxes in detail (Pattern 4 and Pattern 5), with Appendix F.3 showcasing constructions of whiteboxes for a simpler Pattern 12.

W3: AUROC focus. That is also true. We have focused on AUROC, as it’s the most straightforward metric. AUROC not being equal to 1 means that nodes have been misranked, which – in our eyes – is an important failure of an explanation, as it implies that there does not exist a threshold which could separate all motif nodes from all non-motif nodes. As such, we see perfect AUROC in WOMBAT as a necessary (but probably not sufficient) criterion for a good explanation. Importantly, at this point for all tested XAI methods there exist cases where their AUROC is imperfect. Because the most intuitive metric is still “unsolved”, we leave analysis of other metrics for future work.

W4: Capitalisation style. We thank the reviewer for pointing out this capitalisation style issue. We have addressed it in the revised manuscript.

Q1: Whitebox for another architecture. Yes! We have actually done that in Appendix Section F.2: we tested XAI methods on a whitebox implemented using GINEConv architecture and evaluated XAI methods on it. As stated in response to W1, quantitative results differ, which was expected.

Q2: Other metrics. Yes. We focus on ranking in our analysis, but explainers assign numerical values to nodes, which we then analyse with a ranking-based metric (i.e. AUROC). That said, there is nothing stopping WOMBAT’s users from using their own metrics. To this end, we release all code and data publicly.

I appreciate the authors’ efforts to address several points raised in my initial review. My original score remains unchanged.

AuthorsOpenReview ↗

Thank you!

We thank the reviewer for their time and for acknowledging our responses. We remain happy to clarify any outstanding points during the discussion period.

Reviewer_ewrtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper explores the validity of post-hoc GNN explainers on molecular data. The authors use 14 white-box GNNs with pre-set weights and known ground truth model classification decision reasons. This allows the study of whether the explainers find the attribution correctly, or whether they are picking up signals from shortcuts or incorrect explanations besides the ground truth molecular motif. They motivate the paper with the example that they can cause the Integrated Gradients method to spread the attribution across the entire graph, even though it can correctly identify the class motif. This is a useful paper in determining whether current GNN explainers are useful in practice, and the open code is provided to help move the community forward and continue improvement in this area, however, the benchmark data is not yet available. The reasons for not releasing the full data are valid, but this raises full evaluation/ reproducibility concerns at the moment. This is a very important and useful contribution, however, I find the paper lacking in impact with regards to contributing insights or directions for future research (such as design principles) based on the findings in this paper. Additionally, it is unclear how the baseline models are adapted for fair evaluation since they are node/ graph classification explanation models being evaluated on different downstream tasks.

Critical strengths

S1. The authors identify a real issue in GNN explainability, that there is a lack of ground truth real world data which can be used to evaluate GNN explainers. Synthetic dataset such as BA2-Motifs, BA3-Motifs, Spurious Motifs, etc. exist and are useful for benchmarking, however, they do not fill the need for baselines on real-world data. Some chemical/ molecular datasets do exist with real world ground truth data (MUTAG, and those listed by the authors in Table 1), but what the authors propose here is much more expansive across chemical space.

S2. This paper is the first to combine the use of whitebox GNNs with applications in chemistry/ the molecular domain. It is useful in discovering the failure modes of explainer models, even of those which have high performance. This could be useful in determining whether an explainer is widely applicable, and understanding when it fails such that its use can be cautioned against these cases in real-world settings, and it can be improved directly to account for these cases.

S3. Embedding construction for molecular structure is well-designed and all decisions for whitebox GNN construction, along with other considerations such as how to handle explainer-specific cases (edge → node explanation), are well thought out and addressed thoroughly.

Critical weaknesses

W1. A limitation is that this was only evaluated on acyclic motifs. The authors state that no cyclic motifs were chosen for identification by the whitebox GNNs because of the known issue that it is challenging for GNNs to identify cyclic motifs. However, this remains a pertinent issue especially for the application to chemistry where cyclic motifs are common. Could we infer that an explainer which performs well for acyclic motifs would also transfer this good performance to cyclic motif identification, should the underlying GNN overcome this issue?

W2. The paper would benefit from a section which directly posits insights learned from the explainer successes/ failures studied in order to help draw conclusions as to what would benefit future explainer development. The paper lacks a set of takeaways about how we can use the learnings current explainers to improve research in this area.

W3. It is underaddressed how the previous explainers (GNNExplainer, PGExplainer, SubgraphX, etc.) can have such poor performance on these whitebox GNNs, yet they each report high Fidelity/ AUROC metrics in their reported empirical results. Does this then suggest a failure of the metric in measuring explanation quality correctly? Or, are all of these models picking up on spurious motifs? Did the authors find anything in the attribution results to suggest this? The AUROC is also measured in this paper, how do the authors suggest that the field should approach evaluating explanation quality, and what should the standard metrics be?

W4. The comparison with baseline explainers in Table 2 is not informative because it gives a checkmark indicating the explainer achieved a perfect score (AUROC = 1), but no information on what the x’s mean. These could still be good values, but not a perfect score. Can the authors provide a table with the actual values? For comparison with the evaluation of the explainers in comparison to the whitebox GNN attribution, it would be helpful to know how the explainer methods themselves perform. Also, how were these methods adapted for the downstream task? Most are binary node or graph classification explainers. Therefore, it does not immediately carry that they should have high performance on a motif detection task.

W5. The metric for negative results (requiring uniformity) may wrongly attribute success to an explainer that does not learn anything and thus retains uniform weighting. The case of explainer collapse would remain indistinguishable here.

Questions for authors

Q1. What is the downstream task the whitebox GNNs are validated on? Without training, how are the weights determined such that the model can correctly identify 4860 motifs selected?

Q2. How should the explainers behave correctly in the case that the explanation is the absence of a motif? Were there any examples of this in the PubChem dataset used, or did all molecules have identifying motifs?

Q3. Are the 14 whitebox GNNs generalizable? Can ante-hoc methods be constructed this way as well (for example, the GNNs in GSAT)?

Q4. How sensitive is this analysis to the hand-selected weights? Are there different configurations of weights that give the whitebox GNNs the same performance guarantees and correct ground-truth attribution, but then improve/ worsen the explainer performance?

(Also, please see the questions raised in W1, W3 and W4.)

8 comments

We thank the reviewer for their review. We are delighted to hear that our benchmark is useful and makes a very important contribution.

Before addressing the individual weaknesses and questions, we would like to clarify two aspects of our methodology:

  1. Our whitebox GNNs perform binary graph classification. Each model predicts whether a particular molecular motif is present in the input graph (class 1) or absent (class 0). Models are first validated on this task across millions of molecules, and then attributions created by XAI methods are tested against known ground truth using a subset of validation molecules on the same task. As such, the concern that the explainers are being evaluated on a different downstream task, raised in the summary and W4, seems to be caused by a misunderstanding of our experimental setup.
  2. Each white-box GNN detects exactly one motif, not 4,860 motifs (Q1). Our benchmark consists of 14 models, each designed to recognize a single predefined molecular motif (see Fig. 2).

Furthermore, we clarify that the exact molecules used for XAI evaluation are already available as SMILES in the CSV evaluation artifacts included in our repository. The explainer evaluation can therefore be reproduced using the released code and data. The dataset we could not link while preserving anonymity contains the larger sets used to validate the whitebox models. As stated in the paper, these molecules were obtained from the PubChem FTP server; code used to create our validation datasets from these SMILES is already released in the anonymised repository.

W1: Cycles. Raw MPNN cannot overcome such limitations without architectural interventions. For intuition: imagine an MPNN that correctly detects a cyclopropane and has ll MPNN layers. Assume the network does not receive any additional information (such as node identities or “is in cycle” flag) and is formulated as an MPNN, i.e. message-passing with permutation-invariant aggregation operator. Now consider a path made of CH2 carbons of length 10l10l. Trivially, MPNN will represent some of these carbons with the exact same representation as atoms that would belong to cyclopropane, even though the graph contains no cycles (including triangles) at all. Furthermore, if we imagine cyclohexane, all its nodes would produce exactly the same representations as those of cyclopropane. Similar issues were already studied by [A, B].

[A] Huang, Y., Peng, X., Ma, J., & Zhang, M. (2022). Boosting the cycle counting power of graph neural networks with i 2^ 2-GNNs. arXiv preprint arXiv:2210.13978.

[B] Chen, Z., Chen, L., Villar, S., & Bruna, J. (2020). Can graph neural networks count substructures?. Advances in neural information processing systems, 33, 10383-10395.

Note that including cycle-defined tasks while using standard MPNNs would not automatically produce a valid XAI benchmark: because these architectures cannot in general detect cycles, a model that performs well on a finite dataset may instead rely on degree patterns, graph size, or other shortcuts. An explainer faithfully exposing such a shortcut would then be penalised for disagreeing with the nominal structural ground truth. Cycle-based evaluation therefore requires either a cycle-aware architecture, graph features that explicitly encode cycle information, or a separate construction proving that the model implements the intended predicate – for example, one that exploits unique node identifiers.

To address the reviewer's concern, we constructed an additional whitebox detecting the SMARTS pattern [C;H1;R0]-[C;H1;R], corresponding to an acyclic carbon with exactly one hydrogen connected by a single bond to a cyclic carbon with exactly one hydrogen. To enable detection of this pattern, we augmented the atom features with a binary ring-membership indicator computed using RDKit, similarly to as it is done in literature [C, D]. This allows the pattern to be detected using WOMBAT's existing MPNN architecture without further architectural modifications. We use r2r_2 readout.

[C] Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., ... & Leskovec, J. (2020). Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33, 22118-22133.

[D] Wojtuch, A., Danel, T., Podlewska, S., & Maziarka, Ł. (2023). Extended study on atomic featurization in graph neural networks for molecular property prediction. Journal of Cheminformatics, 15(1), 81.

(continued in the next response)

Due to the time constraints of the rebuttal period, we conducted a simplified validation (one million molecules from the Pattern 12 validation dataset) and found no false positives or false negatives (955580 negatives, 44420 positives). Overall, most explainers performed satisfactorily, with Saliency and Input x Gradient failing due to r2r_2 readout. Interestingly, GNNExplainer worked remarkably well (see Table below), and PGExplainer did not collapse. These observations are generally consistent with our previous experiments, as this pattern is generally easy to detect as long as the “is in ring” flag is provided: in other words, such MPNN with cycle-aware features does not have to architecturally differ from an MPNN without cycle-aware features and (in this case) conclusions from acyclic patterns can actually transfer to cyclical ones.

ExplainerAUROC
PGExplainer0.9819 ± 0.0001
Integrated Gradients1.0000 ± 0.0000
Saliency0.5000 ± 0.0000
SHAP Sampling1.0000 ± 0.0000
Input × Gradient0.5000 ± 0.0000
GNNExplainer1.0000 ± 0.0000
SubgraphX0.9194 ± 0.0148

W2: Takeaways. We thank the reviewer for this suggestion. We have added Section 5.1 (“Takeaways”), marked in blue in the revised manuscript. It discusses our views on main issues with optimisation-based explainers, whether alternative integration paths could address the observed IG failures, and the potential and limitations of explainers based on Shapley values.

W3: Why some methods fail. SubgraphX’s attributions are often imperfect, but its AUROCs do not indicate catastrophic failure (Table 3, moved from the appendix). PGExplainer is likely hindered by many nodes sharing zero final-layer representations (Appendix E.2, Fig. S8); we also observe inconsistent convergence on simpler motifs and shortcut learning. For GNNExplainer, none of the tested configurations succeeds on Pattern 14 (Appendix E.3).

Success on other benchmarks need not transfer to WOMBAT: we demonstrate that changing weights alone can alter explainer performance (see our answer to Q4), while our GINEConv experiment (Appendix F.2) shows even bigger differences across architectures.

These findings are not merely a metric issue. AUROC evaluates attribution rankings against known ground truth, while qualitative analysis helps identify failure mechanisms and assess when they may recur. We therefore advocate using metrics to support qualitative analysis of explainer behaviour and failure modes, consistent with [E]’s call for “evaluation frameworks that are both task-specific and precisely defined”.

[E] Moshkovitz, M., Srinivas, S., Semenova, L., Frost, N., Rashtchian, C., Boreiko, V., ... & Vaughan, J. W. (2026). Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methods. arXiv preprint arXiv:2607.14123.

W4: AUROC table. The table was in the Appendix; we’ve moved it to the main text (Table 3). We thank the reviewer for the suggestion.

As presented above, the models perform a binary graph classification task. We describe the experimental setup in Appendix B: explainers work in “regression” mode where appropriate to facilitate unbounded r1r_1 readout function.

W5: Negative explanations. We follow B-XAIC (NeurIPS 2026). An ideal explainer may return null attributions for a negative example. If a collapsed explainer returns exactly the same attributions, the metric necessarily assigns it the same score. This is why negative evaluation alone is insufficient: we must also evaluate positive examples, where the known motif provides attribution ground truth. We focus mainly on positives because most methods pass the negative criterion on WOMBAT; the exception is an observed case of PGExplainer shortcut learning.

Q1: Weight-setting. As described in the beginning of our rebuttal, GNNs are validated on binary graph classification tasks (predicting whether a given pattern is present or not). The weights were determined by a human with pen and paper, writing down weights manually. The construction process relied mostly on creating “circuits” that would progressively detect more complex patterns and then combining them into even more complex ones. Our Appendix section D.2 provides an intuition behind weight-setting (along with Fig. 3).

Q2: Negative explanations.

  • B-XAIC argues that attribution in such cases should not make any node “overimportant” and hence proposes the IQR criterion. We agree with that assessment and adopt the same methodology for negative explainability.
  • For each of our patterns, PubChem contains millions of molecules that do not contain it. In fact, our validation dataset alone contains at least 5 million negative examples for each of the motifs.

Q3: Construction transferability. Other architectures can also likely be made to detect some patterns by hand-setting weights (we have actually successfully attempted it already with GINEConv; see Appendix F.2).

Q4: Sensitivity to weights. Yes. Changing weights while preserving the message passing architecture and prediction rule can alter explainer performance. This includes the IG attribution “spreading” onto irrelevant nodes noted by the reviewer (Fig. 7), as well as changes that improve or worsen PGExplainer’s performance. We discuss these effects in Section 5 and Appendix F.3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.