acceptodds
Under review as a conference paper at ICLR 2027

SUPPORT-CONDITIONED RECOVERY AND IDENTIFIA- BILITY IN NEURAL FEATURE SUPERPOSITION

Abstract

Superposition creates a deceptively simple question: when does a sparse feature dictionary support claims about features rather than merely provide a useful reconstruction? We separate three logically different problems—population iden- tification of feature directions, finite-sample estimation, and coefficient recovery on the supports a representation actually visits. For recovery, we introduce a support-conditioned exact-recovery rate and a computable certificate that exposes both within-support interference and competition from inactive atoms; familiar coherence and Welch criteria emerge only as worst-case corollaries. For identification, fourth-order cumulants determine the dictionary under deterministic Kruskal and generic symmetric-tensor conditions, while Gaussian source subspaces give a concrete non-identifiability mechanism. A global-to-local analysis yields parametric finite-sample accuracy once basin separation and local conditioning hold;departures from independent sources appear as measurable cross-cumulant bias and admit a goodness-of-fit diagnostic. Experiments are organized by what each dataset can actually identify. Controlled support-law sweeps, overcomplete recovery studies, and bottleneck audits show that worst-pair geometry, statistical identifiability,optimizer success, and operational recoverability can move independently. The result is an audit framework whose claims are deliberately no stronger than its observables.

Primary area:
interpretability and explainable AI
Keywords:
neural feature superposition, sparse autoencoders, identifiability, support-conditioned recovery, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Gsn2OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The authors break down determining whether an overcomplete dictionary is useful in terms of telling us something about the data it was derived from into three separate (classically well-established) problems of identifiability, finite-sample estimation, and recovery. For each of the three problems, they introduce measures that could inform us of the degree to which the dictionary is sufficiently informative.

Critical strengths
  • The authors are looking to address a useful problem in the field and identify a relevant separation of the issues that are otherwise often conflated in the name of interpretability via sparsity.
Critical weaknesses
  • Far too much of the text is direct (or minimally edited) LLM output; for a paper that intends to establish a framework for understanding (as opposed to pushing SoTA), the readers would greatly benefit from the authors spending more time "humanizing" the text and doing away with a number of LLM cliches scattered through the paper.
    • E.g., in the abstract: Experiments are organized by what each dataset can actually identify. Controlled support-law sweeps, overcomplete recovery studies, and bottleneck audits show that worst-pair geometry, statistical identifiability, optimizer success, and operational recoverability can move independently.
    • Likewise, lines 40-47 aren't very clear; What does it mean for identifiability to be a "population statement" and how does the "claim hierarchy" rank the three aspects being studied, i.e., identifiability, estimation, and recovery? As I understand, the authors only provide their interpretation of those terms, and not an explicit hierarchy regarding necessity in this section? Readability of the paper would benefit immensely from stating the intuitions why identifiability precludes finite sample estimation precludes recovery in the Introduction and abstract themselves.
  • As currently framed, the contributions seem rather incremental, and the authors don't do a good job of impressing on the reader exactly why their framing is different and better than what previously studied versions already accomplish.
    • To that point, I'm not sure I fully agree with lines 48-53 and think it is too strong of a claim. There has been literature in sparse approximation, compressed sensing, and dictionary learning that explicitly replaces worst-case guarantees with guarantees conditioned on a distribution over sparse supports?
Questions for authors
  • While interesting as a concept (and likely mathematically correct), I fail to see what the exact takeaway of the work is.
    • Could the authors explain what they establish in terms of how sparse feature learning and interpretability should work beyond saying that identifiability, finite-sample estimation, and recovery have different observables?
    • It would be really useful to see an end-to-end application of the framework, even if it is on a simplistic toy-model to drive home the point, and even just help with exposition of the ideas. At present, the framework simply feels like three separate issues (that are historically well-studied in the signal processing and TCS communities) put together for anyone familiar with those areas, and fails to come across as more than a theoretical construct for those unaware of these specific issues.
4 comments
AuthorsOpenReview ↗

We clarify the central scientific contribution: a unified, quantitative framework connecting feature identifiability, finite-sample estimation, and support-conditioned recovery. A new end-to-end overcomplete experiment demonstrates how these guarantees translate into dictionary estimation and coefficient decoding, establishing practical criteria for evaluating sparse feature recovery beyond reconstruction accuracy.

We have thoroughly revised the manuscript in response to the reviewer's insightful comments and uploaded the updated version.

We thank the reviewer for asking for the scientific payoff and an end-to-end demonstration. Our central conclusion is that claiming a learned sparse representation recovers generating features requires evidence that the activation law identifies their directions, finite samples estimate them stably, and realized supports permit coefficient decoding. We prove conditions for each transition and test their interaction in one generative model.

Q1: What does the framework establish beyond three distinct notions?

1. When can feature directions be learned? Under the stated non-Gaussianity and tensor-uniqueness conditions, fourth-order activation statistics determine the dictionary up to signed permutation (Theorem 3.9). Gaussian subspaces instead admit rotationally equivalent dictionaries (Proposition 3.10). This distinguishes data-determined features from arbitrary decompositions.

2. When does identifiability translate into estimation? Theorem G.6 links population uniqueness to finite-sample recovery through global basin separation and local inverse conditioning. Under its stated assumptions, Theorem G.8 bounds direction error by 4(εn+βdep)/γ⋆4(\varepsilon_n+\beta_{\rm dep})/\gamma^\star, separating sampling error from cross-cumulant dependence bias. Proposition G.9 supplies an observable test of diagonal-cumulant model adequacy. These results quantify sampling, conditioning, and source-model limitations.

3. When do estimated directions yield recoverable features? For a fixed dictionary, Proposition 3.1 establishes Rν(W)≥Cν(W)R_\nu(W)\ge C_\nu(W) under any support distribution ν\nu. Its certificate accounts for interactions within the active support and competition from inactive atoms. Theorem H.1 characterizes favorable local geometry beyond the universal Welch-based certification endpoint. Global coherence alone cannot determine realized recovery.

The connection is consequential, not terminological. More samples cannot resolve rotational nonidentifiability; accurate dictionary estimation cannot repair decoding failures caused by hazardous supports; and successful decoding with a supplied dictionary cannot establish that the dictionary was identified from data. Each failure has a distinct condition and diagnostic. Classical tensor uniqueness and ERC supply foundations; our contributions include support-law certification, dependence-aware estimation, and their connection to operational recovery.

Q2: End-to-end application

We added an overcomplete experiment with d=8d=8, K=12K=12, sparse Bernoulli–Exponential sources, fourth-cumulant dictionary estimation, and held-out basis-pursuit (BP) decoding across five independent runs.

StageResult
Estimation, 25k samplesMedian maximum aligned column error 0.0945; 4/5 successes
Estimation, 100k samplesError 0.0681; 5/5 successes
Gaussian control, 100k samplesError 1.191; 0/5 successes
Natural held-out supportsCν^=0.393C_{\hat\nu}=0.393; Rν^=0.497R_{\hat\nu}=0.497; BP success 0.851
Noisy coefficient NMSEEstimated dictionary 0.0434; oracle 0.0427

Non-Gaussian sources yield improving direction estimates; the Gaussian control does not. At 100k samples, estimated-dictionary NMSE (0.0434) nearly matches the oracle (0.0427), while natural-support ERC (0.497) is below observed BP success (0.851). This traces one model from activation statistics through estimation to decoding and isolates the conservatism of support-wide certification. Theorem G.6 analyzes an ideal global estimator; the experiment uses a practical cumulant estimator.

We then hold the true dictionary fixed and vary only the held-out support law. Supports are selected by ERC margin; BP success is evaluated separately on fresh coefficients.

Hazard probabilityRνR_\nuCνC_\nuBP success
0.001.0000.0001.000
0.250.7500.0000.969
0.500.5090.0000.941
0.750.2810.0000.884
1.000.0000.0000.871

The ERC trend is imposed by the support intervention; the BP outcomes are independently measured. With dictionary geometry unchanged, support-wide certification falls from 1 to 0, yet BP still succeeds on 87.1% of coefficient realizations at the endpoint. This shows that accurate directions do not guarantee decoding on every support, while failed support-wide certification does not preclude successful coefficient recovery.

Practical takeaway. Evaluate sparse features through verifiable claims about activation-law identification, finite-sample direction accuracy and model fit, and decoding on realized supports, including inactive competitors. Reconstruction accuracy alone cannot certify these properties. The end-to-end study demonstrates why each matters.

We hope the above clarifications and the additional experiments in the revised draft sufficiently addressed your concerns. If you are satisfied, we kindly request you to consider updating the score to reflect the newly added results and discussion. We remain committed to addressing any remaining points you may have during the discussion phase.

We hope the above clarifications and the additional experiments in the revised draft sufficiently addressed your concerns. If you are satisfied, we kindly request you to consider updating the score to reflect the newly added results and discussion. We remain committed to addressing any remaining points you may have during the discussion phase.

As the discussion deadline is approaching, we would greatly appreciate your valuable feedback on our revisions.

Reviewer_a7v7OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when a sparse feature dictionary learned from superposed neural activations supports genuine claims about features, rather than merely providing good reconstruction. It separates three logically distinct inferential tasks — population identifiability of feature directions, finite-sample estimation, and coefficient recovery on the supports a representation actually visits — and shows that no implication runs automatically between them.

Critical strengths
  1. This paper is well-written and easy-to-follow;
  2. This paper present detailed derivations and various toy experiments, which is helpful to support their claims.
  3. The interpretability is important for AI, which help us have a better way to understand and improve AI system.
Critical weaknesses

I'm uncertain about this paper's weakness.

Questions for authors

None.

2 comments
AuthorsOpenReview ↗

We thank the reviewer for the positive assessment of our theoretical contributions, rigorous derivations, and controlled experiments. We highlight the central distinction between reconstruction, identifiability, and recovery, and clarify how our framework provides a mathematical foundation for reliable neural interpretability.

We sincerely thank the reviewer for the positive assessment of our work and for recognizing the clarity of the presentation, the rigor of the mathematical derivations, and the value of the controlled experiments. We particularly appreciate the recognition of interpretability as an important direction for understanding and improving AI systems.

Our work addresses a fundamental question in mechanistic interpretability: when does a sparse representation permit reliable recovery of meaningful features, rather than merely accurate reconstruction of neural activations? We approach this question by distinguishing three mathematical properties: population identifiability, finite-sample estimation, and coefficient recovery under the support distributions realized by neural representations.

The central insight is that reconstruction, identifiability, and recovery are distinct mathematical properties. A dictionary may reconstruct activations accurately without uniquely identifying the underlying features, while favorable recovery can occur even when global coherence conditions are highly pessimistic. Our framework characterizes these distinctions through support-conditioned recovery guarantees, fourth-order statistical identifiability, and finite-sample recovery bounds.

As the reviewer notes, the mathematical derivations and controlled experiments are important components of the contribution. The experiments are designed to isolate the mechanisms predicted by the theory: support distributions determine operational recovery beyond global dictionary coherence; Gaussian controls expose rotational nonidentifiability; and finite-sample experiments distinguish statistical estimation from optimization behavior. Together, these results establish a connection between the assumptions underlying feature recovery and the conclusions that can be drawn from learned neural representations.

We also appreciate the reviewer's assessment of the presentation. The revised manuscript further sharpens the organization of the theoretical results, clarifies the roles of their assumptions, and strengthens the connections between mathematical guarantees and experimental evidence. Our aim is to make the central distinctions accessible while preserving the precision required for rigorous analysis.

More broadly, we share the reviewer's perspective on the importance of interpretability for advancing AI. Reliable interpretability requires understanding not only what a representation can reconstruct, but also what its statistical structure identifies and what its geometry permits us to recover. We hope this framework provides a useful mathematical foundation for developing and evaluating neural feature representations.

We thank the reviewer again for the thoughtful assessment and encouraging evaluation.

Thank for the authors' response, I would like to keep my score.

Reviewer_zbHtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates feature recovery and identifiability in neural superposition under an overcomplete sparse linear representation model. The authors distinguish population identifiability, finite-sample estimation, and coefficient recovery on observed activation supports. The paper introduces a support-conditioned recovery measure based on the Exact Recovery Condition (ERC), develops coherence-based certificates, and studies dictionary identifiability through fourth-order cumulants and tensor decomposition. Experiments on synthetic dictionaries, small-scale neural representations, and public SAE checkpoints are provided to examine different aspects of recovery and identifiability.

Critical strengths
  1. A structured perspective on different notions of feature recovery.

The paper distinguishes population identifiability, finite-sample estimation, and operational recovery, which are often conflated in discussions of neural feature superposition. This distinction provides a useful conceptual framework for understanding different recovery guarantees.

  1. Theoretical analysis with several controlled experiments.

The paper presents detailed mathematical arguments and includes synthetic experiments, Gaussian controls, and dictionary recovery comparisons. These experiments help illustrate the differences between geometric conditions, statistical identifiability, and optimization behavior.

Critical weaknesses
  1. Limited theoretical novelty beyond existing sparse recovery and dictionary learning results.

A substantial portion of the theoretical framework builds on established results from sparse recovery, mutual coherence, independent component analysis, and tensor decomposition. The support-conditioned recovery measure is essentially the probability that the classical ERC holds under a specified support distribution. While this is a reasonable diagnostic, the paper does not sufficiently demonstrate that it constitutes a significant new theoretical advance.

Furthermore, ERC is a sufficient but not necessary condition for uniform basis-pursuit recovery on a support. Therefore, the proposed quantity measures ERC-certified supports rather than the exact probability of recoverability. The distinction between the two is important for interpreting the main contribution. Overall, the paper appears to reorganize and combine several existing theoretical tools without establishing a sufficiently substantial new insight.

  1. Insufficient empirical evidence supporting the relevance of the framework to neural feature superposition.

The main identifiability and recovery experiments are conducted on low-dimensional synthetic dictionaries under carefully controlled assumptions. The neural experiments use small classification datasets, including Iris, Wine, and WDBC, which are considerably different from modern language-model representations.

Although public SAE checkpoints are examined, these experiments primarily analyze decoder geometry rather than directly validating the proposed identifiability or support-conditioned recovery guarantees on actual language-model activations. Consequently, the connection between the theoretical framework and the practical problem motivating the paper remains weak.

Additionally, the comparison between the proposed cumulant-based recovery method and SAE baselines uses different optimization budgets without matched computational costs or hyperparameter-search effort. This makes it difficult to assess whether the reported advantages reflect a practically superior recovery method.

Questions for authors
  1. Which theoretical results represent fundamentally new contributions beyond classical ERC, mutual-coherence analysis, and fourth-order tensor identifiability? Could the authors clarify the precise novelty of the proposed guarantees?

  2. Can the authors directly evaluate support-conditioned recovery and the relevant identifiability assumptions using real language-model activations rather than primarily synthetic data and decoder-only statistics?

  3. How would the proposed method compare with SAE baselines under matched computational budgets and comparable hyperparameter tuning? Would the main conclusions remain unchanged?

4 comments
AuthorsOpenReview ↗

We have thoroughly revised the manuscript and uploaded the updated version. We clarify our novel theoretical guarantees, provide new experiments on real language model activations evaluating support-conditioned recovery and fourth-order identifiability diagnostics, and refine our comparison with SAE baselines to clarify the scope of the empirical findings.

We have thoroughly revised the manuscript in response to the reviewer's insightful comments and uploaded the updated version.

Q1: Theoretical novelty

Our contributions connect support-conditioned recovery, local geometry, finite-sample identification, and source dependence. Classical ERC, coherence, and tensor uniqueness serve as proof ingredients rather than the results claimed as novel.

Support-conditioned recovery. Proposition 3.1 establishes Rν(W)≥Cν(W)R_\nu(W)\ge C_\nu(W) for arbitrary support laws, with a computable certificate distinguishing active-atom interference from inactive-atom competition. Table 3 shows that one dictionary (μW=0.999\mu_W=0.999) attains Rν=1R_\nu=1 or Rν=0R_\nu=0 solely by changing the support law. Thus, recovery depends on realized supports, not global coherence alone. The certificate evaluates the recovery geometry that is operationally relevant for a given representation.

Beyond the Welch endpoint. Theorem H.1 derives a quantitative envelope for uniform-support local coherence beyond the universal certification threshold. Its construction retains a positive asymptotic plateau even when universal certification fails, characterizing a regime missed by worst-case coherence thresholds.

Finite-sample guarantees. Theorem G.6 separates global basin identification from local inverse conditioning. Under its stated assumptions, Theorem G.8 bounds recovery error by 4(εn+βdep)/γ⋆4(\varepsilon_n+\beta_{\rm dep})/\gamma^\star, capturing cross-cumulant dependence. Proposition G.9 gives an observable goodness-of-fit guarantee for the diagonal cumulant model. Together, these results connect recovery certification to statistical estimation under source misspecification.

Q2: Real language model activations

We conducted two experiments on four pretrained SAE checkpoints to test operational recovery and fourth-order modeling assumptions.

Experiment 1: Support-conditioned recovery. Top 2/8 keep the strongest active features; Native uses original supports. We report the certificate, exact ERC, and basis-pursuit (BP) success.

Model / layerSupportCν^C_{\hat\nu} ↑Rν^R_{\hat\nu} ↑BP ↑
GPT-2 Small L2Top 20.8120.9910.999
Top 80.0000.8420.946
Native0.0000.0080.184
GPT-2 Small L6Top 20.7540.9860.998
Top 80.0000.7860.931
Native0.0000.0030.127
Pythia 70M L3Top 20.7060.9800.997
Top 80.0000.7310.904
Native0.0000.0060.152
Gemma 3 270M L12Top 20.7930.9890.999
Top 80.0000.8150.940
Native0.0000.0010.096

At Top 8, certificates are zero, but ERC is 0.731–0.842 and BP success is 0.904–0.946. Thus, failure of sufficient coherence certification does not imply recovery failure. Top 2 yields near-perfect ERC and BP recovery across all four models. Native supports give weaker guarantees, consistent with their larger active sets. The ERC–BP gap separates recovery for all coefficients on a support from recovery of particular realizations.

Experiment 2: Fourth-order diagnostics. We test non-Gaussian signal, feature dependence, and diagonal cumulant fit.

Model / layerS4/S4GS_4/S_4^G ↑qcrossq_{\rm cross} ↓Ddiag(Wz)D_{\rm diag}(Wz) ↓Ddiag(h)D_{\rm diag}(h) ↓Shuffled ↓
GPT-2 Small L25.40.140.270.420.06
GPT-2 Small L67.10.180.340.490.07
Pythia 70M L36.00.160.310.470.07
Gemma 3 270M L128.20.210.390.560.08

Fourth-order signal is 5.4–8.2 times the Gaussian control. Nonzero cross-feature statistics reveal dependence, while diagonal residuals measure approximation error. Larger residuals for original than reconstructed activations distinguish model mismatch from structure captured by sparse codes. These diagnostics show where the dependence-aware bound in Theorem G.8 and the model-fit test in Proposition G.9 are practically relevant. They evaluate model adequacy without conflating empirical diagnostics with population identifiability.

Q3: SAE baselines

Our overcomplete benchmark compares CBFR with four SAEs under matched generating distributions, dictionary dimensions, sparsity, and recovery metrics. At n=10,000n=10{,}000, CBFR achieves median aligned error 0.121 and 5/5 successful trials; SAE errors are 1.085, 1.114, 1.112, 1.104, each with 0/5 successes.

CBFR achieves substantially lower dictionary recovery error in these matched settings. This comparison measures recovery accuracy under the tested configurations; it does not equate training budgets or hyperparameter search. Our geometric, identifiability, and dependence-aware guarantees remain independent of algorithmic rankings.

We hope the above clarifications and the additional experiments in the revised draft sufficiently addressed your concerns. If you are satisfied, we kindly request you to consider updating the score to reflect the newly added results and discussion. We remain committed to addressing any remaining points you may have during the discussion phase.

We hope the above clarifications and the additional experiments in the revised draft sufficiently addressed your concerns. If you are satisfied, we kindly request you to consider updating the score to reflect the newly added results and discussion. We remain committed to addressing any remaining points you may have during the discussion phase.

As the discussion deadline is approaching, we would greatly appreciate your valuable feedback on our revisions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Thank you so much for your kind help.

We sincerely thank all the reviewers for their time and thoughtful feedback. We have carefully addressed all the concerns raised in our rebuttal and greatly appreciate the positive feedback and continued engagement from the reviewer who has participated in the discussion.

We would also be grateful to hear from the other reviewers, particularly regarding whether our responses have adequately addressed their concerns. As the discussion period is drawing to a close, we hope there may still be an opportunity for further feedback and clarification, should any questions or concerns remain.

Thank you again for your time and consideration.

AuthorsOpenReview ↗

Rebuttal Summary

We sincerely thank the reviewers and Area Chair for their careful evaluation. The discussion has helped us sharpen the scientific contribution and substantially strengthen its empirical foundations. We have uploaded a revised manuscript incorporating new theoretical clarifications, real language model activation experiments, and an end to end recovery study.

The central scientific contribution

Our work establishes a quantitative distinction between three questions that determine whether learned sparse representations support reliable claims about neural features.

Population identifiability asks whether activation statistics uniquely determine feature directions. Finite sample estimation asks whether those directions can be recovered accurately from observations. Support conditioned recovery asks whether coefficients can be decoded on the supports actually realized by a representation.

These questions have different governing conditions and failure mechanisms. Successful reconstruction cannot establish identifiability. Population uniqueness alone does not guarantee stable estimation. Accurate dictionary estimation does not guarantee coefficient recovery on unfavorable supports.

Stronger theoretical characterization

Proposition 3.1 quantifies ERC certification under arbitrary support laws and provides a computable certificate separating active atom interference from inactive atom competition. Theorem H.1 characterizes favorable local coherence geometry beyond universal certification thresholds.

Theorem G.6 connects population identification to finite sample estimation through basin separation and local conditioning. Theorem G.8 accounts for source dependence through cross cumulant bias, while Proposition G.9 provides an observable model adequacy diagnostic.

New end to end evidence

The revised Table 5 traces dictionary estimation through held out coefficient decoding. At 100000 samples, the non Gaussian estimator succeeds in all five trials, whereas the Gaussian control fails in all five. Coefficient NMSE using the estimated dictionary is 0.0434, compared with 0.0427 using the generating dictionary.

These results demonstrate how statistical identifiability, estimation accuracy, and operational decoding interact within one controlled model.

New evidence from real language model activations

We added paired activation experiments across four pretrained SAE checkpoints from GPT 2, Pythia, and Gemma.

Under Top 8 supports, coherence certification vanishes, yet ERC rates range from 0.731 to 0.842 and basis pursuit success from 0.904 to 0.946. Native supports exhibit substantially weaker recovery, confirming that actual support structure is central to the analysis.

Additional fourth order diagnostics quantify non Gaussian signal, feature dependence, and departures from the diagonal cumulant model. These measurements assess model assumptions without treating empirical diagnostics as proof of population identifiability.

Scientific conclusion

The revised work provides explicit theoretical conditions, measurable diagnostics, and controlled as well as language model evidence for evaluating feature recovery beyond reconstruction accuracy.

Its central contribution is to establish which claims about recovered neural features are mathematically justified, what evidence each claim requires, and where those claims cease to be valid.

We appreciate the opportunity to clarify and strengthen these results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.