acceptodds
Under review as a conference paper at ICLR 2027

Linear Lobotomy For Alien Graph Representations : Probing, Ablation and Recovery of Linear Features in Graph Neural Networks

Abstract

Which graph properties does a graph neural network (GNN) use, and are they the ones a domain expert would name? Instance-level explanations for GNNs are well developed, but tools that answer this model-level question are not. We call a graph-theoretic property that a model encodes and uses, although the literature on the task does not name it, an alien graph representation (AGR), and we make the notion testable by fixing an expert set S2 per task before looking at any result. We then propose linear lobotomy, a pipeline that tests the Linear Representation Hypothesis (LRH) at three levels: linear probing decides whether a property is encoded (Weak), concept erasure with matched random controls decides whether it is used (Strong), and converged retraining of the classifier head decides whether it is exclusively encoded along the erased direction (Exclusive). We compare three erasure operators (LEACE, iterative nullspace projection and a rank-one ablation) and show that the choice of operator, not the property, drives the size of an amnesic drop unless the share of the representation it deletes is reported. Across five datasets, a synthetic motif benchmark, two molecular and social benchmarks and two clinical fMRI connectivity tasks, AGR candidates are the rule at the Weak level, few survive the Strong level, and exclusivity tracks the architecture rather than the property: on the synthetic benchmark a GCN recovers fully from every erasure and a GIN never does. On MUTAG the model relies on branching and size counts rather than the ring counts a chemist would name; on the depression task six spectral, resistance and clique descriptors outside the clinical vocabulary are candidate used AGRs; on the social benchmark, the graph features align with the expert vocabulary.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph neural network, XAI, Mechanistic interpretability, Graph machine learning, Linear probes, Linear Representation Hypothesis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GK46OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes “linear lobotomy,” a pipeline for investigating graph properties encoded in GNN representations. It combines linear probing, concept erasure with random controls, and retraining of the downstream classifier. These stages distinguish decodability, dependence of the frozen classifier on erased information, and recovery of task performance. Properties outside a predefined task-specific expert vocabulary that pass the probing and erasure tests are classified as alien graph representations, with a check for linear proxies of expert properties. Experiments cover a synthetic motif benchmark, molecular and social graph classification, and two functional-connectivity tasks. The principal findings concern sensitivity to the erasure operator, differences in recovery between GCN and GIN configurations, and candidate unexpected structural properties.

Critical strengths

A useful empirical audit of representation-intervention methods on GNN embeddings. The paper compares three erasure operators and reports residual decodability, the number of removed dimensions, random controls, and recovery behavior across several tasks. The comparison shows that erasure operators achieving similar residual linear decodability can produce substantially different accuracy drops, illustrating why raw drops alone are insufficient for interpreting an intervention. The recovery-budget example likewise shows how premature stopping can produce misleading conclusions about recoverability. These observations identify practical failure modes in interpreting GNN representations and are the main reason I find the paper informative. The explicit reporting of failed robustness controls and inconclusive clinical results is also valuable.

Critical weaknesses

The erasure test does not establish property-specific use.

A property passes the paper’s “used” criterion when its erasure reduces accuracy more than random ablations of the same dimension. However, the reported used AGRs lack convincing support from the variance-matched control: no margin on any dataset reaches z=1.1z = 1.1 (Sec. 4.4). This leaves unresolved whether the intervention removes information specific to the named property or simply disrupts important directions in the embedding. Matching the number of removed dimensions does not ensure comparable disruption, particularly when the representation is anisotropic and the graph properties are highly correlated. The results therefore provide evidence of sensitivity to the intervention, but do not establish the property-specific dependence claimed in Sec. 5.

The fMRI results make this ambiguity particularly clear. LEACE already makes the property linearly undecodable and produces almost no accuracy loss. The candidate effects appear only under INLP, which removes approximately seven to nine directions. The additional damage could therefore arise from removing other useful information rather than from more complete removal of the target property. A small fraction of deleted dimensions does not establish selectivity. Moreover, no clinical effect survives Holm correction, and two ASD properties clear the dimension-matched control despite failing the held-out linear-decodability criterion. These findings weaken the interpretation of the Strong test as identifying properties the classifier uses. Since this is the key test supporting the alien findings, the claim that the model “provably depends on” the identified properties is stronger than the evidence warrants.

The analysis does not adequately distinguish alien information from overlapping expert cues.

The paper acknowledges that its property dictionary is highly collinear and excludes an alien property when a linear model predicts it from the expert set with held-out R2≥0.9R^2 \geq 0.9. This is a useful safeguard, but it does not establish that an erasure isolates information beyond the expert cues. A property can escape this flag while sharing task-relevant information with an expert property.

On Grid-House, the cycle rank ∣E∣−∣V∣+1|E| - |V| + 1 determines the label exactly. Although the paper correctly excludes cycle rank as an S2S_2 proxy, it counts edge count, node count and density among the alien properties used by GIN with weight decay. Erasing count-related directions could disrupt the expert-equivalent cycle cue, and the reported drops do not distinguish this explanation from reliance on additional alien information. The algebraic relationship does not prove that the individual counts are expert proxies, but it leaves the attribution unresolved.

On MUTAG, the statement that the model relies on branching and size “rather than ring counts” is also too strong: erasing ring counts still reduces accuracy by 10.5 percentage points. Larger drops for other properties do not establish distinct reliance when the erased information overlaps, as the paper’s own Limitations paragraph notes (“the S2S_2-proxy flag, not the size of a drop, carries the attribution”). The expert vocabulary itself also needs stronger justification: Sec. 4.3 states that the expert vocabulary for mutagenicity is ring structure without citing a source, and Sec. 1 reports expert validation only for the clinical sets. Consequently, the paper’s “clearest” alien case remains dependent on both uncertain attribution and an insufficiently validated expert vocabulary.

Questions for authors

Q1. The dimension-matched and variance-matched controls lead to different conclusions. Sec. 4.4 attributes this to how the random directions are drawn; could you explain the disagreement more fully and report both margins for every property counted as used? An additional control could fit the same erasure operator to repeatedly shuffled training-property values, while accounting for differences in the number of removed directions and representation distortion. Which used-property verdicts remain supported under comparably disruptive controls?

Q2. What evidence distinguishes reliance on an alien property from disruption of overlapping expert information? On Grid-House, this concerns edge count, node count and density versus cycle rank; on MUTAG, it concerns branching and size descriptors versus ring counts. Comparing joint erasure of the expert properties with joint erasure of expert and alien properties could be informative, provided the additional damage is appropriately controlled and shared information is considered when interpreting a negative result. Please also justify the MUTAG expert set and explain whether plausible alternative expert sets change the main AGR verdicts.

Q3. Could you distinguish recovery of label accuracy from recoverability of the erased property, for example by evaluating held-out nonlinear prediction of that property after erasure? Please also clarify whether the Grid-House results use one trained encoder checkpoint per configuration, with five seeds only for subsequent analyses. Given the differences in depth, width, pooling and classifier heads, what supports attributing the GCN/GIN contrast specifically to aggregation? In particular, does “GIN’s sum readout” in Sec. 4.1 mean neighborhood sum aggregation, since Sec. 3 specifies mean graph pooling?

Q4. What changed between the original and corrected runs? Please distinguish which choices, including expert sets, thresholds and the use of INLP for the fMRI interpretation were fixed before the initial results, and which were revised after inspecting earlier results. This would clarify which findings are exploratory and which evaluate choices fixed independently of those findings.

Reviewer_CUAMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether the properties extracted and used by graph neural networks (GNNs) are graph-theoretic properties and whether they align with those considered important by domain experts. The authors introduce Alien Graph Representation (AGR) to describe properties learned by neural networks that fall outside the scope of domain experts’ interests. They also propose Linear Lobotomy, an analytical framework. The authors compare three erasure methods, including LEACE, INLP, and rank-one erasure, and evaluate them on Grid-House, MUTAG, IMDB-B, and datasets for ASD and MDD.

Critical strengths

This paper distinguishes between whether a property can be decoded from hidden representations and whether the classifier actually relies on that property. Through the Detect, Remove, and Recover procedure, the paper further examines whether properties are actually used, enabling a more rigorous interpretation of GNN hidden representations than probing alone. The experiments also show that, across most tasks, probe performance is only weakly correlated with the drop in classification accuracy after removal, demonstrating the practical significance of this methodological distinction.

Critical weaknesses

W1. The assignment to the Strong tier, and the conclusion that a property is actually used by the model, are sensitive to the choice of random control. The paper primarily determines Strong status using Z-scores relative to dimension-matched random ablations. However, most effects weaken substantially under the more stringent variance-matched control. Consequently, the current experiments do not adequately distinguish between two explanations: whether the performance drop results from removing information about a specific graph property or from removing an important direction that accounts for substantial representational variance. W2. Because the paper’s central concept of AGR emphasizes properties that the model actually uses, rather than merely those that can be decoded from its representations, the sensitivity of the Strong classification to the choice of control directly affects its core conclusions. The authors should further justify why dimension-matched controls should be preferred or report the stability of AGR classifications across different reasonable control conditions.

Questions for authors

The paper uses dimension-matched random ablation to determine Strong status, yet many effects weaken substantially under variance-matched controls. Could the authors clarify why matching the number of removed dimensions is more appropriate than matching the amount of removed representational variance for determining whether a classifier actually uses a particular graph property? If a property were required to satisfy the criteria under both dimension-matched and variance-matched controls, how many of the currently reported genuine AGRs would still qualify?

Reviewer_9UrtOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors investigate which properties GNNs encode and use, and whether these properties are the ones a human or the literature would name. A property that the model encodes and uses and is not named by the literature is called an alien graph representation (AGR), which is the complement of an expert set defined beforehand. They propose a “linear lobotomy” pipeline, that (1) decides whether a property is encoded using a linear probe, (2) detects if the concepts is used using a concept erasure methods, and (3) conclude if the property was exclusively encoded along the erased direction by retrain the classifier head on their representation. The authors use this pipeline for classifying the AGR candidates, which they conclude are mostly weak (i.e., they are encoded by the model, but not actually used.)

Critical strengths
  • The proposed goal of defining alien representations for graphs is interesting and can improve understanding of which kind of representations GNNs encode internally.

  • The introduced pipeline is well designed, and the exclusivity level proposes a relevant test on whether downstream layers can compensate for erased representations.

Critical weaknesses
  • zvarz_\text{var} is reported as a robustness test, but in Table 3, the main text table that reports it, it stays below 1 for every model. Although this is noted by the authors, this test weakens the Strong level claims. Additionally, almost all other tables report only zdimz_\text{dim}. The conclusions drawn in Section 4 frequently refer to results and statistics reported only in the appendix, even though they are needed to support the paper’s main claims, so the main text cannot be evaluated on its own.

  • Alien representations have already been discussed in interpretability literature, and the authors note that two out of the three tests in linear lobotomy are already well studied. The novelty therefore mainly lies in applying the concept to GNNs through a pre-fixed expert set, sequencing graph probing and causal interventions, and adding the retraining-based exclusivity test.

  • In the introduction, the authors comment on the limitations of the LRH and state that they test them, but it is not clear whether the experiments actually addressed these limitations.

  • The text reads like a draft and needs substantial revision. For instance, the abstract is hard to parse, as it introduces several new terms before explaining what the pipeline does. A suggestion is to describe the probing, erasure and retraining steps in plain terms first, and introducing the terminology afterwards. In the experiments, implementation details are provided but no overall conclusion is available. There are also conflicting excerpts. The paragraph “What the architectures do differently” attributes GIN’s behaviour to its sum readout, whereas Section 3 and Table 3 state that the GIN uses mean pooling. The offered untested interpretation would also fit better in the Discussion.

Further comments

  • Appendix B contains several unresolved citations, reading “(?)”.

  • Bringing up the research question earlier in the text could aid readers' understanding, as it currently only appears in page 4.

  • The first contribution item is presented as a definition of AGRs, but it mainly lists protocol features and refers to details in the appendix.

  • Sections 2.2 and 2.3 mix in results, which makes the method harder to follow and blurs which design choices were fixed before seeing results.

Questions for authors
  • Why was zdimz_\text{dim}. chosen as the Strong criterion instead of zvarz_\text{var}.? The only justification given by the authors is that the variance matched control is sensitive to how random directions are drawn. How are these directions sampled, and do the Strong level results change if zvarz_\text{var} is used instead?

  • What are the “corrected runs” mentioned in Section 2.4? Definitions and thresholds in Table 1 seem arbitrary.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.