acceptodds
Under review as a conference paper at ICLR 2027

Frequency, Direction, and Feature Sharing: Dissecting Co-activation in Sparse Autoencoders

Abstract

Co-activation in sparse autoencoders (SAEs) is often used to infer concept hierarchies, but shared features and parent–child direction require different evidence. We show that on non-tied pairs, reciprocal coverage orients concepts solely by support-set size. In a complete noiseless chain model with an unknown full-rank dictionary, every ordering of L nonzero states admits an exact representation, and an L₀ penalty favors frequent states at shallow depths rather than identifying semantic order. On 5,572 WordNet pairs with Llama-3.1-8B/JumpReLU, direction accuracy is 79.1% when the child-to-parent corpus-frequency ratio is below one and 25.3% otherwise. In a separate matched-control analysis, a lexical-frequency baseline scores 66.1% versus 61.1% for the support rule, while hierarchical pairs show greater excess overlap than frequency-matched random controls (Cohen's d=0.91). Replacing JumpReLU with TopK at fixed encoder weights and approximately matched mean activation counts reduces the frequency-regime gap from 53.8 to 6.4 points. Our results distinguish frequency-associated orientation from semantic sharing and give sufficient conditions for latent-order recovery within the chain model.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, co-activation, feature sharing, hierarchical direction, latent-order identifiability, reciprocal coverage, corpus frequency, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mp73OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies co-activation patterns in sparse autoencoders and how they relate to concept hierarchies, separating two questions that are often mixed together. The authors show that a common rule for choosing the parent mostly comes down to comparing how many features each word activates. They prove that co-activation alone cannot identify which concept is the parent. On WordNet pairs with Llama-3.1-8B, the predicted direction mainly follows word frequency rather than the true hierarchy. Still, hierarchical pairs share more features than unrelated ones. The authors conclude that feature sharing is a useful signal, but co-activation does not reliably reveal hierarchy direction.

Critical strengths
  • Co-activation is often used to read concept hierarchies off SAE features. Showing that the inferred direction largely follows word frequency is a useful warning for the interpretability community.
  • The authors compare against frequency-matched random pairs and related word pairs (that are not hierarchical), and report confidence intervals and threshold sensitivity. They also work on isolating where the frequency effect comes from by swapping the feature-selection mechanism while keeping the SAE weights fixed.
Critical weaknesses
  • Clarity: the paper is very hard to follow. Figure 1 is meant to give the intuition early on, but it is hard to understand without definitions that only come later. The text is dense and heavily hedged, with many terms never explained in plain words ("non-tied pairs", "support rule", "slot agreement", "regime contrast"). The main message, that the inferred direction mostly follows word frequency (if I understand correctly!), is simple but gets lost. It is not even clearly understandable from the abstract.
  • Limited and mixed results: almost all of the main evidence comes from one SAE, trained on Llama-3.1-8B at layer 16, with a short check on two other Llama layers. The authors also test GPT-2 and Gemma, but there the pattern doesn't hold.
Questions for authors
  • Based on these findings, what should practitioners who want to extract concept hierarchies from SAEs do differently, beyond comparing against frequency baselines?
4 comments
AuthorsOpenReview ↗

Response to Reviewer mp73: Clarity Revisions, Cross-Model Scope, and Practitioner Guidelines

We thank Reviewer mp73 for recognizing the practical value of our warning and our experimental controls (frequency-matched and related non-hierarchical pairs, CIs, threshold sweeps, and fixed-weight gate isolation).

Your understanding is spot on: parent–child direction read off SAE co-activation tracks word frequency (79.1% when rcorpus<1r_{\mathrm{corpus}} < 1 vs. 25.3% when rcorpus≥1r_{\mathrm{corpus}} \ge 1), whereas shared feature count reflects genuine semantic relatedness (d=0.91d = 0.91 vs. frequency-matched controls).

1. W1: Clarity—Abstract, Figure 1, hedged prose, and plain-word definitions Thank you for the diagnostic examples; we revised all of them:

  • Abstract & §1: Rewritten to state the plain-language takeaway in sentence two before formal notation, cutting hedged phrasing.
  • Self-contained Figure 1 caption: Rewritten without forward symbols (|S_P|, excess overlap, reciprocal coverage): both panels share core features {2,3} (teal) and active-count product (3×4=4×3=123\times 4 = 4\times 3 = 12), keeping chance-corrected overlap identical, while changing only unshared features (gray) flips active counts from (3,4)(3,4) to (4,3)(4,3) and reverses which concept has more of its features covered.
  • Plain-word definitions on first use & unified labels:
    • Non-tied pairs (§3): Pairs outputting a verdict rather than abstaining (I>0I > 0, ∣SP∣≠∣SC∣|S_P| \neq |S_C|).
    • Support rule (§5.1): Rules comparing aggregated feature sets—M1 (reciprocal coverage) and M2 (support size), identical on non-tied pairs by Prop. 2.
    • Slot agreement (§6.5): On non-hierarchical controls (no true parent), the rate placing slot 1 above slot 2 under frequency-matched slot assignment.
    • Regime contrast / Δ\Delta (§6.1, Fig. 4, App. B.3): Unified as Δ=Acc(rcorpus<1)−Acc(rcorpus≥1)\Delta = \mathrm{Acc}(r_{\mathrm{corpus}} < 1) - \mathrm{Acc}(r_{\mathrm{corpus}} \ge 1) (Eq. 12).

2. W2: Concentration on Llama-3.1-8B L16 and behavior on GPT-2 / Gemma We clarify scope across three levels (§6.6, §7, App. B.4 Table 8):

  • Universal across all three families: Prop. 2 (M1 ≡\equiv M2 on non-tied pairs) holds on Llama-3.1-8B (60.8% vs. 60.8%, Δ=+53.8\Delta = +53.8 vs. +53.9+53.9), GPT-2-small (46.0% vs. 46.1%, Δ=−28.8\Delta = -28.8 vs. −29.0-29.0), and Gemma-2-9B (49.4% vs. 49.4%, Δ=+0.7\Delta = +0.7 vs. +0.7+0.7). Within Llama Scope, skew persists across layers 8/16/24 (Δ=47.3/53.8/38.3\Delta = 47.3 / 53.8 / 38.3 pp), widths 32K/128K (53.8/47.853.8 / 47.8 pp), and all K=2–5,τ∈{.05,.10,.15}K=2\text{--}5, \tau\in\{.05,.10,.15\} cells (47.5–53.847.5\text{--}53.8 pp).
  • Why GPT-2 (Δ=−28.8\Delta = -28.8) and Gemma (Δ=+0.7\Delta = +0.7) differ: As noted in §6.6 and Table 8, our causal gate claim is strictly localized to the fixed-weight Llama swap (53.8→6.453.8 \to 6.4 pp) on weights never trained under fixed L0L_0. By contrast, GPT-2 OpenAI TopK was trained end-to-end with k=32k=32: direction stays frequency-coupled (36.1%36.1\% at rcorpus<1r_{\mathrm{corpus}}<1 vs. 64.9%64.9\% at rcorpus≥1r_{\mathrm{corpus}}\ge 1, Δ=−28.8\Delta = -28.8 pp, opposite sign) and fails recovery (46.0% overall). GemmaScope L16 131K shows descriptive support saturation: at τ=0.10\tau=0.10, words activate ∼16×\sim 16\times larger supports than on Llama (∣SP∣,∣SC∣=809.0,804.0|S_P|, |S_C| = 809.0, 804.0 vs. 49.3,55.149.3, 55.1; Table 8), compressing coverage differences to chance (49.4% overall, Δ=+0.7\Delta = +0.7). In none of the three families does co-activation recover hierarchy direction, and §7 scopes the positive Δ\Delta sign rather than claiming cross-family sign invariance.

3. Q1: What should practitioners do differently, beyond frequency baselines? Beyond frequency baselines (predicting the higher-frequency word scores 66.1% vs. 61.1% for M1; balanced accuracy 50.0% vs. 52.4%), §7 gives four practices:

  1. Use undirected sharing with matched controls: Measure excess overlap or Jaccard against frequency-matched random and neighborhood controls (d=0.91d = 0.91; +3.7+3.7 features over neighborhood).
  2. Prefer fixed-count (TopK) selection for supports: Re-gating JumpReLU to TopK at fixed weights removes nearly all sparsity skew (Δ:53.8→6.4\Delta: 53.8 \to 6.4 pp) without hurting reconstruction cosine.
  3. Audit support saturation (∣S(w)∣|S(w)|) upfront: Broad supports (∼800\sim 800 features/word in GemmaScope 131K) compress directional differences to noise; checking ∣SP∣,∣SC∣|S_P|, |S_C| diagnoses saturation early.
  4. Supply Prop. 6's auxiliary channels to orient edges: Since binary supports and single-pass rank (R4 =49.7%= 49.7\% at rcorpus≥1r_{\mathrm{corpus}}\ge 1; 64.1%64.1\% vs. 76.8%76.8\% at rcorpus<0.5r_{\mathrm{corpus}}<0.5) cannot orient reversed pairs, combine SAEs with causal clamping (do(ZA=0)\mathrm{do}(Z_A=0) vs. do(ZB=0)\mathrm{do}(Z_B=0), Prop. 6b), orthogonal/tree priors (Prop. 6c), or multi-context covariance (Prop. 6a).
Reviewer_mp73OpenReview ↗

Reply to rebuttal

I thank the authors for their effort in improving the readability of this work, and for taking care of the entire list of terms I had highlighted. I also appreciate that they highlighted the "practical takeaways" of the work, as I had requested. As these were the weaknesses that I pointed out and the authors made an effort to address them, I am bumping my grade. Nonetheless, I want to clarify that I still remain a bit uncertain wrt how assess this work properly, as it is a bit far from my expertise.

AuthorsOpenReview ↗

Thank You for Spotting Our Core Message: Clearer Notes on Readability, Models, and Practical Steps

Dear Reviewer mp73,

Thank you sincerely for your thoughtful review, for recognizing the value of our warning and controls, and above all for capturing our main message so clearly in your summary. Frankly, your two-sentence summary stated our core point more clearly than our original abstract did! Looking back at both our initial submission and our November 13 reply, we realized we still leaned too heavily on shorthand and symbols, so we wanted to follow up clearly on all three of your points, including a new practical workflow in our November 17 revision.

1. Cleaning up readability from page 1 (Weakness 1) Our mechanism comes down to one simple fraction check: if animal activates 3 features, dog activates 4, and 2 are shared, comparing 2/3 against 2/4 cancels out the 2 shared features on top and simply picks whichever word activates fewer total features as the parent. We rewrote the Abstract, Introduction, and Figure 1 caption so this intuition comes first without forward-looking symbols or hedged prose, and we explained every technical term you flagged right where it first appears (when a pair gives a directional verdict vs. a tie, set-based rules, how non-hierarchical control pairs are scored, and the accuracy gap between common-parent and common-child pairs).

2. Why GPT-2 and Gemma look different, yet tell one consistent story (Weakness 2) Separating two takeaways resolves what looked like mixed results across models:

  • Universal across all three model families: On Llama-3.1-8B (layers 8/16/24, 32K/128K), GPT-2-small, and Gemma-2-9B, conditional coverage reduces to raw active-feature count within 0.1% (60.8% vs. 60.8%; 46.0% vs. 46.1%; 49.4% vs. 49.4%), and in none of the three models does co-activation recover hierarchy direction (46.0% on GPT-2, 49.4% on Gemma, and 52.4% balanced accuracy on Llama).
  • Why the frequency tilt differs across training recipes:
    • Llama Scope (JumpReLU) uses a variable per-token threshold, so rarer words activate more features and get flipped below common children (25.3% vs. 79.1%). Freezing those Llama weights and switching only the inference gate to TopK cuts that 53.8-point gap to 6.4 points.
    • GPT-2 (OpenAI TopK) was trained end-to-end with TopK=32. Across contexts, common words light up more varied feature subsets and accumulate larger union sets, so direction still tracks frequency, just tilted the opposite way (36.1% vs. 64.9%), and still fails overall (46.0%).
    • GemmaScope (131K) hits feature-set saturation: words activate roughly 800 features on average (16x Llama's ~50), drowning relative size differences in noise (49.4%, coin flip).

3. What practitioners should do differently, beyond frequency baselines (Question 1) Prompted by your question, revised Section 7 lays out a concrete Two-Stage "Skeleton + Orientation" Workflow:

  • Stage 1 (Use co-activation only for an undirected concept map): Check average feature counts upfront to avoid Gemma-style ~800-feature saturation, apply TopK selection at inference time on variable-gate SAEs to remove per-token count skew, and filter pairs via excess overlap against frequency-matched random words (where real WordNet pairs show a strong signal, Cohen's d = 0.91).
  • Stage 2 (Orient parent vs. child with an asymmetric signal): Across 9 static internal statistics we tested, none beat chance on common-child pairs. Instead, on a frozen LLM + SAE, run asymmetric causal interventions by suppressing A's enriched SAE features to test whether B drops downstream vs. the reverse (reaching 99.1% accuracy across 540 new noisy overcomplete SAE runs in Table 12). Or, when training SAEs, build in tree or orthogonality priors (80%-100% under low-to-moderate noise, as in Tree-SAE, MP-SAE, and TaxoSAE).

Thank you again for showing us where our writing got in the way of a crisp, useful message. We hope the revised manuscript is much easier and more rewarding to read.

AuthorsOpenReview ↗

Thank You Very Much!: A Quick Note on Our New Follow-Up Comment

Dear Reviewer mp73, Thank you so much for re-reading our revision, recognizing the readability improvements, and raising your score! !!We truly appreciate your openness and fairness.

Reviewer_PdKYOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper studies whether co-activation in SAEs can be used to infer parent-children relations. They show via an idealized chain construction and several experiments that this is not the case, and that this is instead related to occurrence statistics,

Critical strengths

The mathematical model is well defined, albeit idealized, and from it derives a solid proof.

The author use various experimental angles to sketch a convincing image of their propositions. They start with toy problems for which various factors can be accurately modeled and then go to relatively recent LLM models.

Critical weaknesses

The premise of the research is not established thoroughly. The authors strive to prove that shared features and parent-child directions require different evidence. While they prove this directly, they fail to establish the relevance of this question. They do not provide any motivation as to why this question needs to be answered (it would be help to cite papers that starts from the explicit premise of determining parent-child directions). In a similar vein, the actual practical impacts of this paper are limited. By not providing a proper motivation of this paper, the authors also fail to properly conclude what kind of advantage their results can lead to. This makes it difficult to establish the scientific merit of this paper.

The whole of the text is quite sparse, significantly hindering readability. Examples of these are various. There are a lot of experiments, but motivation behind the different experiments is not always clearly stated in the paper, rather each experiment is accompanied by a small block of text describing just the results. This includes the discussions and limitations section as well, where rather than giving an overarching discussion. The limitations is just a single sentence. Some parameters and equations are not defined in the text. f.e., eq 3, 4 lack a small descriptive note, similarly some of the methods of Table 2 are never properly defined in text.

Questions for authors
  • Can you motivate more thoroughly the existential reason of this paper, and its impact.

  • Why did you choose the experiments in this paper. What does each experiment strive to answer?

  • What are the overarching observations, and a more critical discussion of the limitations?

4 comments
AuthorsOpenReview ↗

Response to Reviewer PdKY: Prior Work Grounding, Practical Impact, and Presentation Fixes

We thank Reviewer PdKY for recognizing that our mathematical model is well defined with solid proofs and that our progression from synthetic settings to recent LLMs builds a coherent empirical picture.

(Note on Soundness: We noticed Soundness: 1 alongside your positive assessment of our proofs and empirical progression; if any technical soundness concern remains beyond the motivation and presentation points below, please let us know so we can address it directly.)

1. W1 & Q1: Establishing the premise, citing prior parent–child work, and clarifying practical impact We agree our original draft compressed the motivation too tightly. Revised §1–§2 ground the premise in two active lines:

  • Post-hoc parent–child and co-activation audits on trained SAEs: Tree-SAE (Cao et al., 2026) states our exact target assumption—"Existing works attempt to identify hierarchical relationships within independent feature sets by relying on activation coverage, the assumption that child feature should only activate when its parent feature activates"—and reports false positives on unrelated concepts, which our Prop. 2 and Thms. 3–4 explain formally. Likewise, HiPACE (Zhang et al., 2026) evaluates parent–child structure over WordNet families in trained SAEs; Chanin et al. (NeurIPS 2025) analyze parent–child feature splitting and absorption (following Bricken et al., 2023; Templeton et al., 2024); and Li et al. (2025) and Deng et al. (ACL 2026) map concept structure via SAE co-activation. (Reviewer mp73 similarly notes: "Co-activation is often used to read concept hierarchies off SAE features.")
  • Hierarchy-aware SAE training: Matryoshka SAE (Bussmann et al., ICML 2025), MP-SAE (Costa et al., NeurIPS 2025), HSAE (Luo et al., 2026), and TaxoSAE (Guo et al., 2026) modify SAE training precisely because flat post-hoc readouts lack taxonomic guarantees.

Practical advantages (expanded in §7):

  1. Evaluation standard: Separates valid sharing audits (WordNet pairs exceed matched random and neighborhood controls; d=0.91d = 0.91, +3.7+3.7 features) from confounded direction claims—where predicting the more frequent word scores 66.1% vs. 61.1% for M1 (66.1% of decided pairs have rcorpus<1r_{\mathrm{corpus}}<1; balanced accuracies across regimes are 50.0% vs. 52.4%, and real vs. random orientation is equivalent within ±5\pm 5 pp, TOST p=3×10−4p = 3\times 10^{-4}).
  2. Actionable gate choice: Swapping JumpReLU for TopK at fixed weights cuts Δ\Delta from 53.8 to 6.4 pp while preserving reconstruction cosine (.7991→.8205.7991 \to .8205).
  3. Design blueprint: Prop. 6 proves sparsity alone leaves L!L! depth orders tied, pinpointing the channels (interventions, orthogonality/tree priors, amplitude covariance) needed to orient hierarchies.

2. W2 & Q2: Experiment motivations, Eqs. (3)–(4), and Table 2 definitions We fixed every flagged presentation gap in the revised PDF:

  • Motivating question added at the start of §6.2–§6.7: (6.2) test Prop. 5's reversal and Prop. 6's recovery on 320 continuous SGD SAEs; (6.3) test frequency invariance and Prop. 2 (M1 ≈\approx M2) on 5,572 WordNet pairs; (6.4) rule out threshold (K,τK,\tau) or single-layer artifacts; (6.5) test whether sharing survives matched random/neighborhood controls where direction collapses; (6.6) causally isolate inference-time gating at frozen weights; (6.7) test whether subspace rank (R4) overcomes the binary-support barrier at rcorpus≥1r_{\mathrm{corpus}}\ge 1.
  • Eqs. (3)–(4) & Table 2 baselines (§3, §6.1): Added prose explaining d(P,C)d(P,C) (forward minus reverse containment, abstaining at 0) and o(P,C)o(P,C) (intersection minus uniform expectation ∣SP∣∣SC∣/M|S_P||S_C|/M), plus explicit formulas for M0 (enrichment argmax rj(w)/gjr_j(w)/g_j + global firing-rate order), M1 (sign d\mathrm{sign}\, d), M2 (∣SP∣<∣SC∣|S_P| < |S_C|), M3 (gjg_j-weighted inclusion), and Mean token L0L_0 (∥f(hC)∥0‾>∥f(hP)∥0‾\overline{\|f(h_C)\|_0} > \overline{\|f(h_P)\|_0}).

3. Q3: Overarching observations and critical limitations (§7) Revised §7 synthesizes our findings alongside four explicit limitations: (i) Thms. 3–4 assume noiseless chains with exact prefix implication rather than noisy overcomplete multi-parent DAGs; (ii) rcorpusr_{\mathrm{corpus}} is an observational proxy rather than latent rr; (iii) cross-family GPT-2/Gemma contrasts co-vary model, layer, and training recipe and remain descriptive; and (iv) control matching uses one seed, neighborhood pairs do not equalize continuous cosine similarity, and Eq. (4) uses a uniform null.

Dear authors,

Thanks for this revision. The flow of the paper reads significantly better than before. With this modification, it is much more clear what the motivation and scientific merit of this work is. I am bumping my grade. However, I still have some concerns on the actual practical applications of this research, given that (1) the theoretical results are only valid for exact noiseless chains, and (2) there is no practical downstream recommendations; the authors do not provide another technique on how to recover these hierarchies.

AuthorsOpenReview ↗

Thank You for Re-evaluating: Noisy/Overcomplete Extensions (Cor. 7, Table 12) & Two-Stage Hierarchy Recovery (§7)

Dear Reviewer PdKY,

Thank you very much for reading our revision closely and raising your score. Prompted by your two remaining concerns, we uploaded an updated PDF (9.8 pages main text; all original numbers Def. 1–Prop. 6, Eqs. 1–12, Figs. 1–6, Tables 1–11 unchanged) adding Corollary 7 (§5.2, App. A.2), a 540-run noisy partial-implication overcomplete SAE benchmark (§6.2, App. B.8 Table 12), and a Two-Stage Hierarchy-Recovery Workflow (§7).

1. Beyond Exact Noiseless Chains (§5.2, App. A.2 Corollary 7, App. B.8 Table 12)

  1. Proposition 2 has zero chain, noiseless, or full-rank assumptions: Reciprocal-coverage cancellation (sign d(P,C)=sign(∣SC∣−∣SP∣)\mathrm{sign}\, d(P,C) = \mathrm{sign}(|S_C| - |S_P|) for I>0I > 0) is pure finite-set algebra. It holds verbatim for supports from any noisy, continuous, overcomplete SAE over multi-parent DAGs—verified across Llama-3.1-8B (60.8% vs. 60.8%), GPT-2 (46.0% vs. 46.1%), and Gemma-2-9B (49.4% vs. 49.4%; Table 8).
  2. Observation noise, partial implication (δ>0\delta > 0), and overcompleteness (Corollary 7):
    • Arbitrary noise (Cor. 7a): Pushing clean states through any Markov noise kernel (e.g., x~=x+ε\tilde{x} = x + \varepsilon) preserves μ~I=μ~II\tilde{\mu}_{\mathrm{I}} = \tilde{\mu}_{\mathrm{II}} identically, keeping observation-only minimax error ≥1/2\ge 1/2 at every sample size nn.
    • Partial implication / noisy chains (Cor. 7b): Allowing child firing without the parent at rate δ=Pr⁡(ZC=1,ZP=0)>0\delta = \Pr(Z_C=1, Z_P=0) > 0 under N(0,σ2Id)\mathcal{N}(0,\sigma^2 I_d) noise (σ>0\sigma > 0) gives TV(μ~I(δ),μ~II(0))=δ(2Φ(∥b∥2/2σ)−1)≤δ\mathrm{TV}(\tilde{\mu}_{\mathrm{I}}^{(\delta)}, \tilde{\mu}_{\mathrm{II}}^{(0)}) = \delta(2\Phi(\|b\|_2 / 2\sigma) - 1) \le \delta and finite-sample Le Cam error ≥12(1−n TV)+\ge \frac{1}{2}(1 - n\,\mathrm{TV})_+.
    • Overcomplete exact equivalence (Cor. 7c): In 3-atom coding, DI=[a,b,c]D_{\mathrm{I}} = [a, b, c] and the antipodal split DII=[a+b,−b,b]D_{\mathrm{II}} = [a+b, -b, b] induce the exact same noisy partial-implication law at all nn for every δ≥0\delta \ge 0, swap single- vs. two-feature coding on {a,a+b}\{a, a+b\}, and retain CI−CII=pa(r−1)C_{\mathrm{I}} - C_{\mathrm{II}} = p_a(r - 1).
  3. 540-run noisy, leaky, overcomplete SAE check (§6.2, Table 12): Across m=8m=8 L1L_1 SAEs in d=64d=64 with U(0.6,1.4)U(0.6,1.4) amplitudes, δ∈{0,.05,.10}\delta \in \{0, .05, .10\}, and σ∈{.01,.05,.10}\sigma \in \{.01, .05, .10\} (σd∈{.08,.40,.80}\sigma\sqrt{d} \in \{.08, .40, .80\}), primary-feature support scores 100.0% at r=0.5r=0.5 in every cell yet drops to 27.2% mean accuracy at r=4.0r=4.0 (0.0%–10.0% across σ=0.10\sigma=0.10 cells).

2. Concrete Hierarchy-Recovery Techniques & Downstream Recommendations (§7)

Paralleling causal discovery (Verma & Pearl, 1990) and unsupervised disentanglement (Locatello et al., 2019), Theorem 3 and Corollary 7 explain why another static single-pass support heuristic cannot orient unrestricted flat SAEs (confirmed across 9 static predictors in Tables 2, 10, 11, where R4 reaches 49.7% at rcorpus≥1r_{\mathrm{corpus}} \ge 1). Revised §7 prescribes a Two-Stage "Skeleton + Orientation" Workflow:

  • Stage 1 — Undirected skeleton (post-hoc on flat SAEs): (i) Verify unsaturated supports (∣S(w)∣≪Mτ|S(w)| \ll M\tau); (ii) apply fixed-weight TopK gating matched to mean token L0L_0 (Δ:53.8→6.4\Delta: 53.8 \to 6.4 pp; cosine .7991→.8205.7991 \to .8205); (iii) filter candidate edges via excess overlap o(P,C)o(P,C) against frequency-matched controls (d=0.91d = 0.91, +3.7+3.7 features over siblings on Llama Scope, Table 4).
  • Stage 2 — Edge orientation via Proposition 6:
    • Frozen models + SAEs (Prop. 6b): Apply asymmetric causal interventions—clamping AA's enriched layer-ℓ\ell SAE features to test downstream suppression of BB vs. the converse, or counterfactual ablations (latent do(ZP=0)\mathrm{do}(Z_P=0) readouts through frozen SAEs reach 99.1% across all 540 noisy leaky runs in Table 12)—or asymmetric LLM entailment probes.
    • Training SAEs (Prop. 6c): Impose tree or orthogonality priors during training (80.0%–100.0% across rr at σ≤0.05\sigma \le 0.05 in Table 12), as in MP-SAE (Costa et al., 2025), Tree-SAE (Cao et al., 2026), and TaxoSAE (Guo et al., 2026).

Three downstream applications (§7): (1) Automated ontology & feature labeling—avoids inverting taxonomies when child terms outnumber hypernyms (rcorpus≥1r_{\mathrm{corpus}} \ge 1, e.g., dog above canine, where M1 scores 25.3%); (2) Concept steering & safety unlearning—verifies upstream causal control rather than ablating frequent narrow features selected by coverage; (3) Hierarchical SAE benchmarking—mandates rcorpusr_{\mathrm{corpus}}-stratified Δ\Delta, the zero-SAE frequency baseline (66.1% vs. 61.1%), and matched-control equivalence checks.

AuthorsOpenReview ↗

Thank You Again: A Clearer Walkthrough of Our Noisy/Overcomplete Results and Recovery Workflow

Dear Reviewer PdKY,

First, thank you again sincerely for reading our revision so carefully, recognizing the clearer motivation and scientific merit of the work, and raising your score. Your constructive engagement has made this paper substantially stronger.

Looking back at the technical reply we posted earlier today, we realized we rushed out a dense wall of notation (and OpenReview's math parser even mangled two lines in Section 1). We wanted to leave a much cleaner, more readable walkthrough of how the November 17 manuscript addresses both of your remaining concerns.

1. Going beyond exact, noiseless chains (Section 5.2, Corollary 7, Table 12)

  • The coverage-to-size cancellation needs zero chain or noiseless assumptions: Whenever two concepts share features, those shared features sit in the numerator of both forward and reverse coverage (for example, 2/3 vs. 2/4) and cancel out. Direction always reduces to comparing total active feature counts, whether the SAE is noisy, continuous, overcomplete, or evaluated on multi-parent graphs. That is why coverage direction and raw feature-set size match within 0.1% on real LLMs across Llama-3.1-8B (60.8% vs. 60.8%), GPT-2 (46.0% vs. 46.1%), and Gemma-2-9B (49.4% vs. 49.4%).
  • Noise, exceptions, and overcompleteness make static direction harder, not easier: Corollary 7 formally relaxes all three assumptions. Intuitively, if two opposite parent-child hierarchies already produce the exact same activation distribution before noise, adding random reconstruction noise cannot help an observer tell them apart, much like blurring two identical photographs. Even when a child concept occasionally fires without its parent (5%-10% of the time) and the dictionary is overcomplete (more features than concepts), an overcomplete SAE can split a concept across redundant features so opposite hierarchies still match in distribution while sparsity favors whichever concept is more frequent.
  • Tested across 540 noisy, leaky, overcomplete continuous SAEs (Table 12): Across 8-feature continuous L1 SAEs in 64 dimensions with random amplitudes, Gaussian noise, and imperfect implication, support direction scores 100% when the parent is more common, yet drops to 27.2% mean accuracy when the child is 4x more common (and 0%-10% at higher noise).

2. A concrete technique to recover hierarchies & downstream recommendations (Section 7) Much like causal discovery separates finding connected variables from orienting arrows, revised Section 7 provides a practical Two-Stage "Skeleton + Orientation" Workflow:

  • Stage 1 (Undirected concept map on any flat SAE): Check that feature sets are not saturated, apply fixed-count TopK gating on frozen SAE weights (cutting Llama's frequency skew from 53.8 to 6.4 points without retraining), and keep pairs whose shared-feature count beats frequency-matched controls (Cohen's d = 0.91; +3.7 features over siblings).
  • Stage 2 (Orienting parent vs. child via Proposition 6):
    • On a frozen LLM + SAE (no retraining): Use asymmetric causal clamping by suppressing concept A's key SAE features to test whether concept B drops downstream, versus suppressing B and testing A (blocking animal disrupts dog far more than the reverse). Across all 540 noisy, leaky, overcomplete runs in Table 12, interventional readouts through frozen SAEs reach 99.1% accuracy.
    • When training a new SAE: Add tree or orthogonality constraints during training (80%-100% accuracy under low-to-moderate noise in Table 12), explaining why hierarchy-aware methods (Tree-SAE, MP-SAE, TaxoSAE) succeed where flat co-activation fails.

Section 7 also applies this workflow to three downstream tasks: (1) automated feature labeling & ontology building (avoiding inverted pairs like dog above canine), (2) concept steering & safety unlearning (verifying upstream causal control before ablation), and (3) hierarchical SAE benchmarking (reporting common-child accuracy alongside the 66.1% word-frequency baseline). Thank you once again for pushing our paper to be much more complete and practically useful!

Reviewer_Z64EOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is about whether feature co-activation in sparse autoencoders (SAEs) can reliably reveal semantic hierarchies. The paper claims that feature sharing and parent–child direction identification are fundamentally different problems. This paper first shows that reciprocal coverage mathematically reduces to comparing feature support sizes. Then it establishes that latent hierarchical ordering is not identifiable in a noiseless chain model with an unknown, unrestricted dictionary, and demonstrates that sparsity regularization can favor more frequent states rather than the true semantic ordering. Empirically, the authors evaluate SAE representations using WordNet parent–child pairs across several language models. Their results show that direction prediction is strongly associated with lexical frequency, while semantically related concepts exhibit greater feature overlap than frequency-matched controls. Furthermore, replacing JumpReLU with TopK feature selection substantially reduces the frequency-dependent accuracy gap. These findings suggest that co-activation can provide evidence of semantic sharing but is insufficient on its own to establish hierarchical direction.

Critical strengths
  1. The authors establish several formal results concerning reciprocal coverage, latent-order ambiguity, and the influence of sparsity regularization.

  2. The paper also addresses a meaningful interpretability problem by distinguishing feature sharing from hierarchical direction identification.

Critical weaknesses
  1. The theoretical results are established under a highly simplified noiseless chain model with a full-rank, unrestricted dictionary and binary prefix codes. In contrast, practical SAEs operate on high-dimensional continuous representations, often with overcomplete dictionaries, reconstruction errors, and more complex feature interactions.

  2. The experiments demonstrate a strong association between lexical frequency and predicted hierarchical direction. However, corpus-level word frequency is not equivalent to the latent-state probabilities used in the theoretical analysis.

Questions for authors
  • To what extent do these conclusions extend to practical SAEs with overcomplete dictionaries, continuous activations, and reconstruction errors?

  • Could the authors directly examine whether feature activation probabilities or other representation-level statistics better explain the observed direction bias?

  • The focused-rank approach reduces systematic direction reversal but achieves approximately chance-level accuracy in the challenging frequency regime. What additional information or practical constraints would be necessary to achieve reliable semantic hierarchy recovery from trained SAEs?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Z64E: Bridging Theory to Practical SAEs, Controls, and Beyond-Support Limits

We thank Reviewer Z64E for the accurate summary and for recognizing both our formal characterization and the separation between feature sharing and parent–child direction.

1. W1 & Q1: Extending conclusions from the noiseless chain model to practical SAEs (overcomplete, continuous, noisy) Our analysis connects to practical SAEs at three levels (made explicit in revised §5):

  • Universal set algebra (Prop. 2): Reciprocal-coverage cancellation (sign d(P,C)=sign(∣SC∣−∣SP∣)\mathrm{sign}\, d(P,C) = \mathrm{sign}(|S_C| - |S_P|) for I>0I > 0) assumes neither the chain model, binary codes, nor exact reconstruction. It holds for finite supports from any overcomplete, noisy, continuous SAE, verified on Llama (M1 60.8% vs. M2 60.8%), GPT-2 (46.0% vs. 46.1%), and GemmaScope (49.4% vs. 49.4%; Table 8).
  • Minimal lower-bound logic (Thms. 3–4): As in standard identifiability theory (Locatello et al., 2019; Nelson et al., 2026), Definition 1 isolates a minimal counterexample where two depth orders induce the exact same observation law. Since any fixed map or randomized training preserves equality in law (App. A.2), reconstruction noise or extra overcomplete atoms cannot create order identifiability from observations alone. Conversely, we explicitly scope Thms. 3–4 to chains with exact prefix implication; multi-parent DAGs and partial implication remain open (§4, §7).
  • Tests on continuous SGD & overcomplete LLM SAEs: First, §6.2 (Table 1) trains 320 continuous SAEs via SGD in d=64d=64 with nonzero reconstruction error across 4 sparsity families, reproducing Prop. 5's reversal on decided L1L_1 support certificates (100% at r=0.5r=0.5 vs. 0% at r≥1.5r\ge 1.5). Second, Eq. (1) and Prop. 5 (CI−CII=pa(r−1)C_{\mathrm{I}} - C_{\mathrm{II}} = p_a(r-1)) predict that variable-count gates retain an active-count preference for frequent states, whereas fixed-count TopK holds per-token ∥f(h)∥0\|f(h)\|_0 constant and removes that differential. Swapping JumpReLU →\to TopK at fixed encoder weights on Llama Scope SAEs confirms this: Δ\Delta drops from 53.8 to 6.4 pp (32K, mean L0L_0: 43.671→44.00043.671\to 44.000) and 47.8 to 5.9 pp (128K, 40.370→40.00040.370\to 40.000).

2. W2: Corpus lexical frequency (rcorpusr_{\mathrm{corpus}}) vs. latent state probability (rr) We agree and state in §1, §4, §6–§7 that rcorpus=freq(C)/freq(P)r_{\mathrm{corpus}} = \mathrm{freq}(C)/\mathrm{freq}(P) is an observational proxy rather than latent r=pb/par = p_b/p_a, showing an association rather than a pretraining intervention. Still, the representation-level mediator is directly measurable: as rcorpusr_{\mathrm{corpus}} crosses 1, support-size difference ∣SC∣−∣SP∣|S_C| - |S_P| drives M1, and TopK re-gating contracts mean ∣SC∣−∣SP∣|S_C| - |S_P| from +5.8+5.8 to −0.1-0.1 features alongside Δ:53.8→6.4\Delta: 53.8 \to 6.4 pp.

3. Q2: Do firing probabilities or other representation-level statistics better explain direction bias? Yes—we audited firing-probability, count, and spectral predictors in Table 2 and Appendix B.3, B.5–B.7:

PredictorOverall Acc.Gap Δ\Delta (pp)Reference
M1 / M2 (support coverage / size)60.8% / 60.8%+53.8 / +53.9Table 2
Mean token L0L_0 count60.7%+55.9 (20-tmpl) / +55.3 (30-tmpl)Tables 2, 11
M3 (global firing-rate-weighted inclusion)55.4%+31.9 [29.3,34.5][29.3, 34.5]Table 2, App. B.1
M0 (firing-rate enrichment argmax)49.9%+4.5 [1.8,7.3][1.8, 7.3]Table 2, App. B.1
Effective rank / Singular-value ratio44.7% / 48.5%+17.4 / +16.2Table 11
L0L_0 coefficient of variation39.3%$-$24.4 (reversed)Table 11
Focused effective rank (R4)57.9%+12.4 (rcorpus≥1r_{\mathrm{corpus}}\ge 1: 49.7%)Table 10

Top-firing conditional supports also suffer 3,721/5,572 collisions and stratify from 83.0% (rcorpus<0.5r_{\mathrm{corpus}}<0.5) to 26.4% (rcorpus≥2r_{\mathrm{corpus}}\ge 2; Table 9), while logistic regression gives a nonsignificant hierarchy slope (−0.069-0.069, SE 0.0600.060, p=0.258p=0.258) vs. log-frequency ratio (−1.913-1.913, SE 0.0340.034; App. B.3). No tested single-pass statistic is both above chance overall and frequency-invariant.

4. Q3: What is needed for reliable hierarchy recovery beyond R4 (≈49.7%\approx 49.7\% at rcorpus≥1r_{\mathrm{corpus}} \ge 1)? Table 10 shows R4 removes reversal at rcorpus≥1r_{\mathrm{corpus}}\ge 1 (26.7%→49.7%26.7\% \to 49.7\%, CI [47.4,51.9][47.4, 51.9]) while lowering rcorpus<0.5r_{\mathrm{corpus}}<0.5 accuracy (76.8%→64.1%76.8\% \to 64.1\%), as 30-template subspaces contain background variance absent from noiseless Prop. 6(a). Reliable recovery requires Prop. 6's extra channels: (i) causal interventions (do(ZA=0)\mathrm{do}(Z_A=0) vs. do(ZB=0)\mathrm{do}(Z_B=0), Prop. 6b), (ii) training-time tree/orthogonality priors (Prop. 6c), or (iii) multi-context conditional covariance with explicit background separation.

AuthorsOpenReview ↗

Thank You for Your Insights: Clearer Answers & New Noisy/Overcomplete SAE Results

Dear Reviewer Z64E,

Thank you very much for your accurate summary of our paper and for recognizing why separating feature sharing from parent-child direction matters. Looking back at our November 13 reply, we realized we packed too much dense notation into our answers. Since then, inspired directly by your questions on practical SAEs and hierarchy recovery, we also uploaded an updated manuscript (November 17) with new theoretical and experimental results. We wanted to walk through your three questions in much clearer terms.

1. Extending from the clean chain model to practical overcomplete, continuous, noisy SAEs (W1 & Q1) Our updated manuscript (Sections 5.2, 6.2, Corollary 7, Table 12) connects the theory to practical SAEs at three levels:

  • Zero simplifying assumptions for coverage cancellation: If animal activates 3 features, dog activates 4, and 2 are shared, comparing 2/3 with 2/4 cancels out the 2 shared features and picks whichever word activates fewer features. Because this is pure finite-set counting, it holds for any continuous, noisy, overcomplete SAE, verified on Llama-3.1-8B (60.8% vs. 60.8%), GPT-2 (46.0% vs. 46.1%), and Gemma-2-9B (49.4% vs. 49.4%).
  • New theory for noise, imperfect hierarchies, and overcompleteness (Corollary 7): If two opposite parent-child hierarchies already produce the exact same activation distribution before noise, adding random reconstruction error cannot make them distinguishable. Even when children occasionally fire without parents and the dictionary is overcomplete, redundant features can split concepts so opposite hierarchies still induce the exact same noisy data distribution while sparsity favors more frequent states.
  • New 540-run continuous overcomplete SAE check + real LLM gate swap: Across 540 continuous, 8-feature overcomplete L1 SAEs in 64 dimensions with random amplitudes, Gaussian noise, and leaky child activations (Table 12), support direction scores 100% when the parent is more common yet drops to 27.2% when the child is 4x more common (0%-10% at higher noise). On 32K and 128K Llama SAEs, swapping only the inference gate from JumpReLU to TopK on frozen weights cuts the frequency gap from 53.8 to 6.4 points (and 47.8 to 5.9 at 128K).

2. Corpus frequency vs. latent probability, and internal SAE statistics (W2 & Q2) We completely agree that corpus word frequency is an imperfect proxy for internal concept probability, and we state this explicitly. Importantly, we can directly measure the internal mediator: when a child word is more common than its parent (dog vs. canine), the parent activates +5.8 more SAE features on average, flipping coverage; TopK gating shrinks that internal count gap from +5.8 to -0.1 features and removes the bias. Testing 9 internal predictors, including weighting by global feature firing probabilities (55.4% overall, 31.9-point gap) and ordering top-enriched features by firing rate (49.9%, chance), confirms that no single-pass internal statistic beats chance overall while staying invariant to frequency (Tables 2, 10, 11).

3. What is needed for reliable hierarchy recovery beyond ~50%? (Q3) Focused effective rank removes systematic reversal on common-child pairs (26.7% to 49.7%), but context variance across natural templates keeps it near chance. Revised Section 7 distills a Two-Stage Workflow backed by Proposition 6 and tested in Table 12:

  • Stage 1 (Undirected skeleton): Use TopK gating + frequency-matched overlap controls on flat SAEs to detect which concepts are linked (Cohen's d = 0.91).
  • Stage 2 (Orienting arrows): Supply either (i) inference-time causal interventions on frozen SAEs by clamping A's enriched features to test downstream suppression of B vs. the reverse (99.1% accuracy across all 540 noisy overcomplete runs in Table 12), or (ii) training-time tree/orthogonality priors (80%-100% accuracy under low-to-moderate noise, as in Tree-SAE, MP-SAE, and TaxoSAE).

Thank you again for your constructive questions, which directly shaped these new additions to the paper!

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response: Summary of Revisions, Plain-Language Takeaway, and Core Contributions

We thank the Area Chair and all three reviewers for their constructive feedback. Across reviews, there were no objections to the correctness of our proofs or controlled measurements; shared concerns centered on presentation clarity, literature motivation and practical impact, and bridging theory with cross-model scope. We uploaded a revised PDF (9.1 pages main text, within ICLR's 10-page limit; title and all theorem/equation/table numbers unchanged) addressing every point.

1. Plain-Language Takeaway

Shared SAE features between two concepts carry genuine semantic signal—WordNet parent–child pairs share far more features than frequency-matched random pairs (Cohen's d=0.91d = 0.91) and taxonomic-neighborhood pairs (+3.7+3.7 features, 95% CI [3.4,4.0][3.4, 4.0])—but parent–child direction inferred from co-activation does not. By Proposition 2, on every decided pair, reciprocal coverage algebraically cancels shared features and reduces to comparing how many features each word activates. On Llama-3.1-8B (JumpReLU SAE, 5,572 pairs), accuracy is 79.1% when the child is rarer than the parent (rcorpus<1r_{\mathrm{corpus}} < 1) versus 25.3% otherwise (rcorpus≥1r_{\mathrm{corpus}} \ge 1; Δ=53.8\Delta = 53.8 pp). Replacing only the inference-time JumpReLU gate with TopK at fixed encoder weights and matched mean L0L_0 cuts Δ\Delta from 53.8 to 6.4 pp.

2. Contributions Beyond a Frequency Warning

  1. Exact algebraic reduction (Prop. 2): Any non-tied reciprocal-coverage rule is a support-size rule—verified on Llama (M1 60.8% vs. M2 60.8%), GPT-2 (46.0% vs. 46.1%), and GemmaScope (49.4% vs. 49.4%).
  2. Exact orbit & recovery conditions (Thms. 3–4, Props. 5–6): All L!L! depth orders of LL nonzero states admit exact full-rank chain dictionaries (worst-case observation-only error ≥1/2\ge 1/2), L0L_0 sparsity ranks orders by state probability, and Prop. 6 gives three sufficient conditions (amplitude covariance, causal interventions, dictionary priors) restoring identifiability.
  3. Directional falsification alongside positive sharing (§6.5, App. B.3): M1 direction on real pairs is statistically equivalent to slot agreement on frequency-matched random pairs (diff −0.56-0.56 pp, 90% CI [−2.7,1.6][-2.7, 1.6], TOST p=3×10−4p = 3\times 10^{-4} within ±5\pm 5 pp), and hierarchy has a nonsignificant logistic slope (−0.069-0.069, SE 0.0600.060, p=0.258p = 0.258) vs. log-frequency ratio (−1.913-1.913, SE 0.0340.034).
  4. Theory-predicted causal gate localization (§3, Prop. 5, §6.6): Variable-count gates retain an active-count differential while fixed-count TopK holds per-token ∥f(h)∥0\|f(h)\|_0 constant; swapping JumpReLU →\to TopK on frozen weights confirms this (53.8→6.453.8 \to 6.4 pp at 32K; 47.8→5.947.8 \to 5.9 pp at 128K).
  5. Audit of beyond-support statistics (§6.7, App. B.5–B.7): Across 9 representation-level predictors, focused effective rank (R4) removes high-ratio reversal (26.7%→49.7%26.7\% \to 49.7\%) while lowering rcorpus<0.5r_{\mathrm{corpus}}<0.5 accuracy (76.8%→64.1%76.8\% \to 64.1\%), showing why Prop. 6's extra channels are required.

3. Plain-Language Glossary (Added to §3 & §6)

  • Non-tied pair: A pair with d(P,C)≠0d(P,C) \neq 0 (I>0I > 0 and ∣SP∣≠∣SC∣|S_P| \neq |S_C|), so the rule outputs a directional verdict rather than abstaining.
  • Support rule: Direction rules comparing aggregated feature sets—M1 (reciprocal coverage) and M2 (support size), identical on non-tied pairs by Prop. 2.
  • Slot agreement: On non-hierarchical controls (which lack a true parent), the rate at which M1 places slot 1 above slot 2 under the frequency-matched slot assignment.
  • Regime contrast (Δ\Delta): Acc(rcorpus<1)−Acc(rcorpus≥1)\mathrm{Acc}(r_{\mathrm{corpus}} < 1) - \mathrm{Acc}(r_{\mathrm{corpus}} \ge 1) in percentage points (Eq. 12).

4. Key Manuscript Revisions

  • Abstract, §1, Fig. 1: Plain-language takeaway upfront; self-contained Fig. 1 caption without forward symbols; explicit citations to post-hoc co-activation/WordNet hierarchy audits (Cao et al., 2026; Zhang et al., 2026; Chanin et al., 2025; Li et al., 2025; Deng et al., 2026) and hierarchical SAEs.
  • §3, §5–§7, App. B: Prose explanations for Eqs. (3)–(4) and all Table 2 baselines (M0–M3, Mean token L0L_0), motivating questions opening §6.2–§6.7, unified Δ\Delta, universal-vs-lower-bound bridge in §5, and expanded §7 practitioner guidelines and four scope limitations.
Area_Chair_ioqiOpenReview ↗

Discussion

Dear All,

Thank you for reviewing the paper. Please read the authors' response and leave a comment accordingly.

Yours,

AC

AuthorsOpenReview ↗

A Clearer Summary of Our Core Story, Manuscript Revisions, and How We Addressed Your Concerns

Dear Area Chair and Reviewers,

Looking back at both our original submission and our first-round replies, we realized we leaned too heavily on compact notation and statistical shorthand, making a straightforward finding much harder to read than it needed to be. Before the discussion period closes, we wanted to share a brief, readable summary of what our paper shows, what we changed in the revised manuscript (updated November 17, 9.8 pages), and how we worked to resolve your concerns.

What our paper shows at heart Sparse autoencoder (SAE) feature co-activation is widely used for two things at once: checking whether two concepts are related, and orienting which one is the broader parent (such as animal above dog). We show that co-activation genuinely succeeds at the first task, but fails at the second because of a simple fraction trap.

Suppose animal activates 3 SAE features, dog activates 4, and 2 of them are shared. Then dog covers 2/3 of animal's features, while animal covers 2/4 of dog's features. Because the 2 shared features sit on top of both fractions, they cancel right out. Whenever two concepts share at least one feature, comparing conditional coverage has nothing to do with their shared features: it simply calls whichever word activates fewer total features the parent.

In variable-threshold SAEs (like JumpReLU or ReLU), the sparsity penalty naturally spends fewer active features on more common inputs. When the parent word is more common than the child (animal above dog), this count rule gets the direction right 79.1% of the time across 5,572 WordNet pairs on Llama-3.1-8B. When the child word is more common in everyday text than its formal hypernym (dog above canine, or person above organism), the same rule flips the hierarchy upside down (25.3% accuracy). Guessing "the more common word is the parent" without any SAE already scores 66.1%, beating co-activation's 61.1%. By contrast, undirected feature overlap carries real semantic signal over frequency-matched random pairs (Cohen's d = 0.91) and sibling concepts (+3.7 features). And swapping only the inference gate from JumpReLU to fixed-count TopK on the exact same frozen Llama weights cuts that 53.8-point frequency swing down to 6.4 points.

What we changed in the revised paper

  1. Clearer writing and literature grounding (Abstract, Sections 1-3, Figure 1): We rewrote the Abstract, Introduction, Figure 1 caption, and section openings so the core intuition comes first without undefined symbols or heavy hedging, explained all baselines on first use, and grounded the paper in recent SAE hierarchy work (Tree-SAE, HiPACE, feature absorption, Matryoshka/MP-SAE/TaxoSAE).
  2. Extensions to noisy, imperfect, overcomplete SAEs (Section 5.2, Corollary 7, Table 12): We proved and tested across 540 new continuous, noisy, overcomplete SAE runs (including leaky hierarchies where children fire without parents) that realistic messiness does not rescue static co-activation direction: accuracy still drops from 100% when the parent is more common to 27.2% when the child is 4x more common (0%-10% at higher noise).
  3. A practical Two-Stage Hierarchy-Recovery Workflow (Section 7): Rather than stopping at a negative result, Section 7 gives practitioners a concrete alternative: (i) use fixed-count TopK selection on frozen weights plus frequency-matched overlap controls to extract a clean undirected concept map, and (ii) orient parent-child arrows using either inference-time asymmetric causal feature clamping on frozen models/SAEs (99.1% accuracy across our 540 noisy runs) or training-time tree/orthogonality priors (80%-100% under low-to-moderate noise).

We take full responsibility for how dense and difficult our initial submission was to read. Your feedback pushed us to clear away the jargon, test our claims in messy realistic settings, and build a constructive workflow for practitioners. Thank you sincerely for your time and care, and we hope you might give the revised paper a fresh, generous look.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.