acceptodds
Under review as a conference paper at ICLR 2027

To be or not to be: Mutually Exclusive Features in Language Models

Abstract

Sparse autoencoders have become a widespread tool for discovering interpretable features in language models and to intervene upon them. Yet the standard practice still treats features in isolation, leaving the structure of their relationships unexplored. We study these relationships through feature co-activation graphs, focusing on a particular suppressive relationship that we call mutual exclusivity: features that occur in similar contexts but not together. Across a wide range of models varying in size, we consistently find that mutually exclusive features form interpretable groups whose members represent alternative values of a common semantic type, such as numerals, pronouns, and months, to name a few. These groups exhibit consistent geometric structure: their decoder directions are highly similar, placing alternative values within narrow regions of activation space. Finally, we show that these features are causally relevant to model behavior: replacing a feature with another ME alternative predictably moves the model’s output toward the concept represented by the replacement feature. Together, our findings demonstrate that relationships among features reveal structure in model representations and mechanisms that are difficult to discover by studying them in isolation

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic interpretability, sparse autoencoders, co-activation, mutual exclusivity, representation geometry, causal intervention

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_NMQeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how sparse autoencoder (SAE) features relate to each other, instead of looking at each feature on its own. The authors build a co-activation graph from SAE features. They then focus on one kind of relationship they call "mutual exclusivity" (ME): features that appear in similar contexts but never fire together. The authors study four models and three SAE architectures, and observe that ME features form interpretable group whose members are alternative values of the same type, such as digits, pronouns or months. Their decoder directions point in similar directions, forming narrow cones. The authors also run "intervention" experiments showing that these features have causal roles: swapping one ME feature for another from the same group shifts the model's prediction toward the swapped-in value.

Critical strengths
  • Experimental breadth: the analysis covers four models, three SAE architectures and every layer. The steering experiments are compared against several sensible alternatives, so the effect is shown to be specific to ME features.
  • New characterization of SAE features: most prior work, to the best of my knowledge, on feature interactions looks at features that fire together. This paper studies instead features that exclude each other, making a clear connection to distributional semantics.
Critical weaknesses
  • Limited significance? The paper offers a careful and well-executed analysis that would likely interest the NLP and interpretability communities. In particular, the link between mutually exclusive features and linguistic categories (closed word classes, paradigmatic relations from distributional semantics) is a nice bridge between interpretability and linguistics, and it could be of interest in a venue like ACL or EMNLP. For ICLR, though, I'm less sure the findings tell us much that is new about how language models learn or compute. Most cliques are closed word categories (digits, pronouns, months, prepositions, modal verbs), for which it is fairly expected that features for different words don't fire together.

===

Minor:

  • "Larger cliques are more interpretable" may be an artifact? A bigger set gives the LLM judge more examples to spot a pattern, so the trend in Fig. 2 may partly reflect how the judge works.
  • fix citations format
Questions for authors
  • IIUC, many ME features are highly token-specific (high token purity), and each position holds only one token, so features for "3" and "4" can hardly fire on the same token. How much of the ME signal do the authors think reflects structure the model has learned, as opposed to a consequence of tokenization? Would the findings hold or change if the analysis was about low-purity features?
3 comments

Thank you for taking the time and reviewing our manuscript. We are glad that you find the breadth of our experiments a strength, and that you recognize the novelty in analyzing interactions between features. We first address your questions and return to the weakness in the end.

Could the clique-size trend come from the judge?

We applied the same prompt to ME cliques and several controls, including uncoupled groups matched for size and token purity. The YES rate consistently rises with group size only for ME cliques. We have revised Figure 2 and added Figure 15 to include these results, and added a description highlighted in blue in Section 4.2.The controls give the judge equally large groups, so having more features to look at alone does not explain the trend.

Token purity and tokenization

We follow your suggestion to look at low-purity cliques. We found several semantically coherent cliques among features with low token purity (Table 1, and additional highlighted text in 4.2). These include cliques whose members represent features that correspond to different programming languages, different regions of Canada, or different written language such as Spanish, Russian, etc.

These found cliques also support steering. For example, swapping the Russian feature for the Spanish feature on a Russian prompt changed the model’s continuation from Russian to Spanish (Figure 23). We will expand this evidence further in our next revision. Thank you for this suggestion! The low-purity examples strengthen our evidence that ME cliques capture meaningful relationships among features.

Citation formats

We fixed the issues in the citation formats. Thank you

Significance of findings

The paper is mainly an interpretability contribution, and the ideas described in it can be applied to any modality. We clarify it below and will revise the introduction and discussion to better clarify this point.

1. The contribution is a move from single features to structure among features. SAE-based interpretability today works feature by feature. Each feature is labeled and steered on its own, as if it were one entry in a flat dictionary. But how a model computes depends on how its representations relate to each other, not only on what each one means alone. Our paper takes a first step toward interpreting at that level. We show that one relationship, mutual exclusivity, recovers higher-level types on its own: a variable (e.g., “which direction”, “which pronoun”, “which language”, “which Canadian region”...) together with the values it can take. A single feature cannot show this, because a feature’s meaning as “one of the alternatives for X” exists only in relation to the other alternatives. This adds a layer of meaning that isolated feature labels miss. It is a step from reading a model one unit at a time toward reading the variables it computes with. We hope this is a first step towards interpretability that analyzes higher-level structure and more complicated mechanisms, beyond low level features.

2. Finding closed classes is not trivial, and it supports the method. Rarely firing together is not enough on its own: mutual exclusivity also requires shared context (Figure 5). ME-based swaps are causal, beating the controls and working in later layers where standard feature selection fails (Figure 8), and recovering known categories without supervision validates the method before it is used where we can’t check the results. As our results on impure-cliques highlight, this goes much beyond simple word features (Table 1).

We see ME features as the first evidence of the hypothesis in a broader modality-agnostic interpretability program, namely that analyzing how features co-occur can help to uncover the compositional structure of models.

Our contribution fits into the broader AI community and therefore to ICLR. We are happy to add this additional context and framing to the manuscript.

We hope that this addresses your concerns and are happy to continue the discussion.

Dear reviewer, thank you again for your feedback. With the discussion period ending on November 18, we would appreciate hearing whether our response addresses your main concerns or whether you have any other thoughts. We would be happy to clarify any remaining questions.

I thank the authors for their response. As I think this can be a good interpretability contribution, I am raising my score.

Reviewer_qEpxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper studies how features found by SAEs relate to one another. It builds a graph over features from how often they fire together, and singles out pairs that appear in similar contexts but never fire together. The authors call such pairs mutually exclusive. These pairs form groups that read as values of one type, such as digits, months, or pronouns. Members of a group point in nearly the same direction in activation space, and replacing one member with another moves the model's output toward the replacement.

Critical strengths
  • (S1) Calibrating the selection threshold against shuffled graphs gives the criterion a clear statistical footing.
  • (S2) That the mutually exclusive features steer best, and keep working deeper into the network, is the strongest evidence that the discovered structure matters for behavior.
Critical weaknesses
  • (W1) The payoff of the mutually exclusive feature analysis stays implicit.
    • The conclusion is that relationships between features deserve study, but the paper does not show a setting where the analysis affects some decision. The most convincing cliques are digits, months, and pronouns. Prior work (e.g., in Transformer Circuits) already documented special geometry for numbers and calendar terms, and already showed such features are causal in arithmetic. What the criterion adds on top of that record is left for the reader to guess.
    • Figure 1 only increases that tension for me. The digit clique previews all three claims, which makes for good exposition, but as evidence it leans on the least surprising case. The non-numeric cliques, such as compass directions and sports, carry more of the novelty and could be foregrounded.
    • A concrete use would help, too. Candidates include mutual exclusivity as a proxy for dictionary quality, or as a selection rule on a task where activation-based selection fails.
  • (W2) The positioning overstates how little is known about suppressive feature interactions.
    • Attribution graphs model excitatory and inhibitory interactions between transcoder features directly (Ameisen et al., 2025 [1]; Lindsey et al., 2025 [2]), and transcoder-based feature circuits do the same through MLP layers (Dunefsky et al., 2024 [3]). Those works study computation for single prompts, a different object from the corpus-wide statistics used here, but the claim that suppressive relationships have received little attention is too strong. Several recent papers also build type-and-value structure into the SAE itself (Muchane et al., 2025 [4]; Bussmann et al., 2025 [5]; Cao et al., Tree SAE [6]; Luo et al., 2026 [7]). A clique of mutually exclusive features is post-hoc evidence for the same organization.
  • (W3) The interpretability conclusions depend on LLM-written descriptions and an LLM judge. Semantic similarity between features is measured by word overlap of their Neuronpedia descriptions, and clique interpretability is decided by a GPT-4o-mini judge. Both depend on description quality, which is known to be uneven for SAE latents (Puri et al., 2025) [8]. The paper itself notes that top-activating examples cover only the tail of a feature's behavior. A small human check of a sample of cliques, or one description-free control, would firm up the claim that cliques are semantically coherent.

[1] Circuit Tracing - Revealing Computational Graphs in Language Models (Ameisen et al., 2025)
[2] On the Biology of a Large Language Model (Lindsey et al., 2025)
[3] Transcoders Find Interpretable LLM Feature Circuits (Dunefsky et al., 2024)
[4] Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures (Muchane et al., 2025)
[5] Learning Multi-Level Features with Matryoshka Sparse Autoencoders (Bussmann et al., 2025)
[6] Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders (Cao et al., 2026)
[7] From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoders (Luo et al., 2026)
[8] FADE: Why Bad Descriptions Happen to Good Features (Puri et al., 2025)

Questions for authors
  • (Q1) Clique members have the most similar decoder directions of any pair class in the paper. Could the same groups be found by clustering decoder directions on cosine similarity alone, without activation statistics? What does the co-activation graph add?
  • (Q2) Standard activation-based selection ties the ME criterion in early layers, where both pick mostly the same features. Where exactly does the ME criterion win, and is there a task property that predicts when?
3 comments

Thank you for taking the time to read and evaluate our work. We are happy that the calibration against shuffled graphs was named as a strength and the steering results as evidence that the discovered structure matters for model behavior. We address the concerns and questions below.

(W1) What the analysis adds

We agree that the non-numeric examples make the contribution clearer. In our next revision, we will foreground a language clique in Figure 1, where different features correspond to generating in different languages. We found this example effective in steering as well, as we demonstrate in Figure 23. We also added further examples of cliques in Table 1 (bottom block). In these cliques, features activate across many different tokens (low purity features) but correspond to clear values of the same type: distinct programming languages, sports, or Canadian regions. This highlights that the found cliques can also discover higher-level concepts than numerical digits.

A concrete use of the graph is to identify which features to swap. Our steering experiments (Figure 8) include types not explored before, like compass directions and sports. This approach allows finding structures in an unsupervised manner that can be seen as "levers" for modifying model behavior.

(W2) Positioning relative to prior work

We agree with the suggested framing and will revise the related work to make these distinctions clear in our next revision, citing the suggested body of work.

(W3) Reliance on automated descriptions

We revised Figure 2 to now include several controls, including uncoupled groups matched for size and token purity. Using the same judge, the rise in YES verdicts with group size is specific to ME cliques in the three larger models.

We also manually labeled five low-token-purity cliques from their activation contexts, complementing the examples we already had in Table 1. The features in these cliques correspond to higher-level concepts. For example, we discover a clique of Canadian regions, where each feature in the clique peaks on seemingly unrelated tokens, but the surrounding text distinguishes British Columbia, Ontario, and Atlantic Canada (Table 5).

(Q1) What the co-activation graph adds beyond cosine similarity

A previous work by Engels et al. (2025) attempted to discover features with similarity. They recovered calendar representations by clustering features using decoder similarity and manually inspecting many of the resulting clusters. They also suggested combining cosine and Jaccard similarity in future work (Appendix F.2). This is what motivates to look beyond simple direction similarity metrics. We will add this discussion to our analysis in the next revision.

(Q2) Where ME selection improves steering

We now include Figure 20 to show how often the rules agree, differ, or select the same vanilla feature for both values. We would also like to note that vanilla sometimes chooses the same feature for both values (e.g., same feature is chosen for the “3” and the “5” digit), resulting in a swap with zero effect.

We revise Figure 8 to focus on cases where the rules differ and vanilla selects two distinct features. ME’s steering advantage is apparent in these cases.

We have yet to establish a task property that predicts when ME will win. Figure 23 illustrates one reason vanilla can fail: a feature can activate strongly on a few tokens and still be a poor choice for changing the language of the continuation.

The found cliques also tell us which features are candidates for a swap before the source and target values are specified. In our comparison, vanilla starts with the values identified from the clique and selects a feature for each independently.

We hope that our answers addressed your concerns and we are happy to continue the discussion if there’s anything else.

Thank you for the revisions. The language clique is a compelling addition, and the responses to W2, Q1, and Q2 address my concerns well.

(W1) The proposed use of "identifying which features to swap" hints at an application, but it does not explain which problem requires the global co-activation graph. Similarly, "finding structures in an unsupervised manner" remains very broad. Could you specify (1) the structure that the method is intended to discover, (2) the downstream setting that requires unsupervised discovery, and (3) what the global graph adds on top of the ME cliques used for steering?

(W3) Compared to the initially submitted Figure 2, the revised analysis adds useful controls and shows that the increase in YES verdicts with clique size is specific to ME cliques in the three larger models under this judge. However, every condition is still scored by the same LLM, so this does not address my main concern about the validity of the automated judgments. The five author-labeled cliques are selected demonstrations on already-known examples and cannot substitute for an independent human evaluation. A small evaluation with human annotators blinded to the clique condition, using randomly sampled cliques and reporting human–LLM agreement, would address my concern more directly.

Given that I already supported an accept in the first place and W1 and W3 are what keep me from seeing this as an exceptional paper, I maintain my score.

We thank the reviewer for their timely response.

(W1) While we focus on a specific relationship between features in this work (ME), we are excited to continue this investigation to better characterize the global co-activation graph. Just to clarify, the discovery of ME cliques is facilitated by the global co-activation graph.

(1) What structure is the method intended to discover? The method is not specifically aimed at discovering ME cliques, and we are hopeful that it will continue to yield interesting findings.

(2) What downstream setting requires unsupervised discovery? We believe that unsupervised discovery is a necessary step of interpretability at scale, so we see our method as a contribution to that end.

(3) What does the global graph add on top of the ME cliques used for steering? As far as what the global graph contributes for steering, we hope to offer clear answers to many broader questions about co-activation in the future.

(W3) We would like to affirm the reviewer’s criticism of LLM judges in general and agree our results offer limited guarantees about the validity of automated judgements. We agree that a more robust test would involve independent human validation, but given the time constraints for this rebuttal period we do not find it realistic to supplement our findings with such an experiment.

We thank the reviewer for their suggestions to elevate our paper.

Reviewer_7fzUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper uses negative Ising couplings and similar co-activation neighborhoods to identify groups of mutually exclusive SAE features. It studies their meanings and geometry in four language models and tests whether swapping features changes predictions.

Critical strengths
  1. The graph criterion finds recognisable groups of SAE features, including digits and pronouns, across layers in the three larger models without using semantic labels for selection. Their decoder directions are more aligned on average than those of the comparison groups (Fig. 6, Fig. 17).

  2. I lean to weak acceptance because of the structural and behavioural findings. Across substantial layer ranges in all 3 larger models, swapping digit features at the source token often brings the next-token distribution close to that obtained by changing the input digit (Eq. 4-5, Fig. 8). The tested compass-direction swaps also work well (Fig. 9, Fig. 19).

Critical weaknesses
  1. Fig. 2 gives limited evidence for how commonly ME groups share a semantic type. Its semantic verdicts come from a single prompted judge whose prompt states the hypothesised relationship, alongside a negative example and rejection instructions (see lines 1083-1089 and 1112-1147). No independent validation or matched non-ME comparison of these verdicts is reported.
  2. Evidence that ME improves steering over vanilla is limited. In Fig. 8, ME often ties vanilla selection and improves on it at some layers, but Fig. 19 shows small median effects for most other tested types. Appendix I excludes swaps whose vanilla source and target are the same feature. The appendix does not report exclusion counts or say whether both methods use identical source/target pairs in the reported aggregates (see lines 1285-1293). Excluding zeros could improve vanilla’s average.
Questions for authors
  1. Has Fig. 2 been checked without telling the judge the proposed relationship or by comparing ME groups with non-ME groups matched for size and token purity? This would help me assess how much to trust the reported semantic rates.
  2. On identical source/target pairs, how does ME compare with vanilla before and after these exclusions? Please give the exclusion counts to clarify the added steering benefit.
3 comments

We thank the reviewer for their careful feedback. Thank you for highlighting that our method recovers recognizable groups of SAE features, and we are pleased that you think the study recovers structural and behavioural findings.

Semantic validation of ME cliques (Figure 2, Section 4.2)

We applied the same judge prompt to ME cliques and several non-ME controls, including uncoupled feature groups matched for size and token purity. In the three larger models, ME cliques received more YES verdicts than the controls and were the only groups whose YES rate consistently increased with clique size. We have revised Figure 2, it now shows the rate of YES verdict as a function of cliques for both the ME cliques and the baselines for one model (gemma-2-9b). Other models are reported in Figure 15. The prompt still states the hypothesized relationship, so this comparison addresses whether the pattern can be explained by group size or token purity under the same prompt.

Steering comparison with vanilla selection

We would also like to clarify the feature selection for comparison: a source-target pair selected by the vanilla and the ME method can be (1) same pair (2a) different, but the vanilla chooses exactly the same feature for the source and the target (e.g., same feature is chosen for the “3” and the “5” digit, making the steering degenerate), or (2b) different from ME in at least one of the features and the pair is also different from each other. We provide statistics for these cases in Figure 20. Vanilla sometimes chooses the same feature for both source and target, resulting in a swap with zero effect.

The interesting cases to look at are indeed (2b), where vanilla chooses a different pair of features. We have adjusted Section 4.4 of the manuscript to discuss this (highlighted in blue) and we updated Figure 8 to only contain (2b). We observe that ME generally steers better in this comparison, even after excluding the zero-effect vanilla swaps.

We would like to highlight that, even when the rules agree, ME discovered which features belong together without being given the tokens in advance. This can be seen as an unsupervised approach for finding good feature candidates for steering. Vanilla selection starts with the tokens represented in the discovered clique and finds a feature for each one independently.

We thank the reviewer for raising these questions and helping us improve our manuscript. We are happy to discuss further.

Dear reviewer, thank you again for your feedback. With the discussion period ending on November 18, we would appreciate hearing whether our response addresses your main concerns or whether you have any other thoughts. We would be happy to clarify any remaining questions.

Thank you for your response! I see that in larger models the judge finds a common semantic type in ME groups than in non-ME groups more often. This answers my question about control, although the prompt tells the judge what relationship to look for. I will change my assessment from weak to clear acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.