acceptodds
Under review as a conference paper at ICLR 2027

Cross-Model SAE Coactivation Survives Length, Topic, and Lexical Controls

Abstract

Cross-model matches of sparse-autoencoder (SAE) features have been judged against random pairings or neuron baselines, neither of which re-runs the pairing rule. Gemma-2-9B and Mistral-7B, from different families with different tokenizers, each have a released-SAE obituary feature, and the two share their top five documents: is it the same feature? Our controls support a more specific answer: matched features of these two models respond to the same inputs beyond chance and pair selection, and that shared response survives the topic and vocabulary controls we ran; this is coactivation, not feature identity. Such matches are cheap: two sets of 1,500 features reciprocally match about half of each other under idealized chance, and any document property both models encode (length, topic, vocabulary) can make features coactivate. We run the pairing rule inside a document-permutation null, score frozen pairs on documents the rule never saw, and add controls that each fix one named nuisance. In a non-pre-registered follow-up, all nine cross-family cells (Gemma-2, Mistral, Qwen3, Llama-3) hold on disjoint C4 text (p = 1/2001 each, Bonferroni over the 17-cell roster). The sharing survives length- and coarse topic-matched nulls (one cell); on three representative cells, every control leaves at least 0.88 of the excess its reference arm leaves. Regressing out a unigram TF-IDF lexical model, which a pre-registered gate shows is not inert, leaves 92–94% of the excess: the shared-vocabulary explanation is narrowed, not closed. A registered token-aligned test shows the sharing is not reducible to document means (it beats a within-document word-order shuffle; mean pair correlation 0.522/0.398/0.335) and, in a leakage-clean design, keeps R_tok^clean = 0.92–0.99 of its token-level excess after conditioning on the document and the current word. The claim does not extend to compositional or syntactic sharing, a capability trend (swapping only one model’s SAE width moves its cell from z = 22.1 to 14.1), or shared computation (both cross-model ablation probes were inconclusive). The controls can say no: they rejected three decisive-looking secondary results, all reported (a cross-lingual metric that tracks network depth rather than language, its depth-corrected residual, and a 70M-to-7B match null on natural text). Cross-model matches should be scored against a null that re-runs the matching, on documents the matching never saw, one nuisance per control; code, nulls, receipts and registrations are in the anonymous supplement, to be released upon publication.

Primary area:
interpretability and explainable AI
Keywords:
cross-model feature sharing, sparse autoencoders, representational universality, control experiments, lexical residualization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vMsbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks whether SAE features learned independently in different language models show statistically meaningful agreement in their activations. The authors argue that existing cross-model matching methods are statistically vulnerable: choosing each feature’s highest-correlating partner inflates similarity, while comparisons with randomly paired features fail to account for this.

To address these issues, the authors introduce a stricter testing procedure that accounts for the matching process, evaluates feature pairs on unseen documents, and controls for shared input properties. They find significant coactivation across nine pairs of models from different families. Their conclusions are limited to coactivation, rather than shared feature identity or computation.

Critical strengths

The paper addresses a methodological weakness in some previous attempts to match SAE features across language models. Selecting the most highly correlated features can exaggerate their apparent similarity. Pairing each feature with its most highly correlated counterpart can exaggerate apparent similarity, and a random-pairing baseline does not account for this. The authors correct for this by repeating the matching process when estimating chance similarity and by testing the resulting pairs on unseen documents. Under these stricter tests, feature coactivation persists across all nine cross-family model pairs on natural text.

Critical weaknesses

First and foremost, the paper's (presumed) over-reliance on AI for writing makes it close to impossible to read.

Besides that, the authors argue that shared document properties, such as length, topic and vocabulary, can cause features to coactivate "with no shared feature behind them". It is unclear why these properties should be treated as confounds. If two independently trained models learn similar topic or vocabulary detectors, that could itself be evidence of feature universality. Appendix L explicitly names “two models’ topic detectors co-firing” as the alternative being excluded. The controls test whether coactivation extends beyond these properties, but they may also discount genuine shared representations. The paper needs a clear definition of “shared feature” and an explanation of why each excluded form of coactivation should not count towards universality.

Questions for authors

What additional evidence would establish feature identity rather than mere coactivation? Both ablation experiments in Appendix K were inconclusive. Could the authors develop a more reliable intervention method, first validating it on features known to have similar causal effects, and then testing whether matched features across models produce similar changes in predictions?

The paper treats coactivation driven by shared document properties, such as topic or vocabulary, as a potential confound. But if two models independently learn features that detect the same topic or words, wouldn't this itself be evidence of feature universality? Could the authors clarify what these controls are intended to rule out, and why such agreement should not count as genuine feature sharing?

Reviewer_8VKQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work investigates whether matched SAE features from different LLM families fire on the same texts beyond chance. It pairs features that are each other’s best match by correlation. These pairs are compared against a baseline that shuffles the texts and performs the pairing again. The authors select the pairs using half of the texts and test on the other half, with controls for length, topic, and word overlap. The evaluation uses nine model pairs from four model families (i.e., Gemma-2, Mistral-7B, Qwen3, and Llama-3.2), one mid-layer residual SAE per model, and 2,000 short C4 texts (i.e., the first 300 characters of each document). The authors report that the matched features fire together even after these controls, but they do not claim that the models share features or computation.

Critical strengths

This work uses a stricter test for matching SAE features across models. Its baseline shuffles the texts and performs the pairing again, so it counts the matches that appear by chance (i.e., about half of the features under pure chance). The pairs are also tested on the other half of the texts. Prior work judged matches against random pairs (e.g., Lan et al., 2024), which ignores that each feature is paired with its best match. In my opinion, this is the main strength of the paper, which may guide future studies about making claims on shared features across models.

Critical weaknesses

W1: The paper is hard to follow. It uses heavy private jargon (e.g., "rungs", and "verdict-bearing"), and some captions include script and file names. It also mixes three studies (i.e., SAE matching, a cross-lingual measure, and a failed attribution-graph pipeline). As a result, it is difficult to find the main claims and the evidence behind them.

W2: Prior work has already studied matched SAE features across models by checking whether they fire on the same texts (e.g., Lan et al., 2024). The test in this work combines standard statistical tools (e.g., shuffle baselines and held-out texts). The paper does not show that the models share features or computation. The causal tests were inconclusive. Similarly, a probe moved from one model to the other fails when the matching is done on different texts. In my opinion, it is not clear what new finding this work offers.

W3: The setup is small. Each text is only the first 300 characters of a web page, and the main test uses 1,000 texts. Each model has only one mid-later SAE.

Questions for authors

See the weakness section.

Reviewer_2LX5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper asks whether cross-model SAE feature matches reflect shared features or only shared document properties. Features are matched by reciprocal nearest neighbors and scored against a permutation null that re-runs the matching, with pairs frozen on one half of C4 and evaluated on a disjoint half. Controls for length, topic, vocabulary, and the current word are read as retention ratios against a reference arm. All nine cross-family cells survive on held-out documents, and on three representative cells every control retains at least 0.88 of the excess. The authors interpret this as coactivation on shared inputs rather than feature identity, and also report three rejected secondary signals.

Critical strengths
  1. The evaluation protocol fixes a real flaw in existing feature-matching practice (Sections 3, 5.1, Appendix M). A null that re-runs the pairing rule and frozen pairs scored on disjoint documents remove selection optimism by construction (Propositions 1–2), and the re-implementation of Lan et al. (2024) shows that part of the signal above random pairings comes from the pairing rule itself (up to 94% on templated prompts). This is consequential because it gives a concrete, reusable standard for cross-model universality claims.

  2. The controls are shown to be able to say no, and claims are carefully scoped (Sections 5.1–5.2, Table 1). The same battery rejects the Pythia-70M↔Mistral-7B match on natural text, shows size- and sparsity-matched random directions fall below their null, and finds that a cross-lingual metric tracks depth rather than language. Because the instruments demonstrably reject, the surviving positive carries weight, and the paper is explicit about what it does not show.

Critical weaknesses
  1. The surviving claim is narrow, and the main contribution is largely standard statistical practice (Sections 3, 5.1, 7). After all controls, the positive is that mean-pooled features of different models respond to the same documents beyond length, coarse topic, and unigram vocabulary; the paper itself says the matched features respond to "common lexical and topical document properties" and that this is not feature identity, composition, or shared computation, and both cross-model ablation probes are inconclusive. Meanwhile, the protocol's core ingredients (avoiding circular analysis, held-out splits, plus-one permutation p-values) are established practice (Kriegeskorte et al., 2009; Phipson & Smyth, 2010). This matters because it is unclear what the reader learns about representational convergence beyond "existing protocols should use proper nulls."

  2. Key controls are weaker than the retention ratios suggest, and the headline evidence is not fully selection-clean (Sections 4, 5.1, 6, Table 4). The lexical nuisance model (stopword-stripped unigram TF-IDF) has a median per-feature R² of at most 0.14, so high retention partly reflects how little the nuisance explains, as the authors acknowledge ("narrowed, not closed"). The nine-cell held-out follow-up was not pre-registered and its screen saw both halves, the lexical arm selects and scores on the same 20,000 documents, and each C4 "document" is only its first 300 characters. Combined with an extremely dense presentation that mixes registered, non-registered, and audit results, this makes the strength of the central positive hard to assess.

Questions for authors
  1. Beyond establishing that matches should be scored against a re-running null on held-out documents, what does the surviving coactivation tell us about representational convergence that shared document statistics would not already predict? Is there any analysis showing that matched pairs share something more specific than lexical and topical properties?
  2. Given that the unigram TF-IDF nuisance explains at most a median R² of 0.14, can you add a stronger lexical or semantic nuisance model (e.g., bigram features or a sentence-embedding model) and report whether retention stays above 0.88?
  3. Can the nine-cell held-out result be repeated with a screen that sees only the selection half, and with full-length documents rather than the first 300 characters?
  4. The abstract and Sections 5–6 pack many numbers and registration labels into dense prose. A short table separating pre-registered, non-pre-registered, and audit results, and a simpler abstract, would make the paper much easier to follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.