acceptodds
Under review as a conference paper at ICLR 2027

Identifiable Interventions from Nonidentifiable Sparse Codes

Abstract

Sparse autoencoders may admit multiple nonnegative sparse explanations of one activation, so a feature deletion can depend on the chosen explanation. We study identifying the activation change rather than recovering a unique code. Signed circuits characterize all exactly identifiable diagonal interventions, and a sharp modulus quantifies their noise amplification. An exact quotient theorem shows that certification and minimum-cost selection commute with arbitrary replication of decoder directions. On forest Gram graphs, a balanced-subtree algorithm certifies stability in arithmetic operations with rational counterexamples; positive paths admit optimal binary deletion over all feature subsets. For single-activation codes on arbitrary dictionaries, optimal attenuation has an explicit shortest-path formula. A separate construction proves an unbounded advantage over every stable binary deletion for any sparsity budget and arbitrarily overcomplete, nonidentifiable codes. Conversely, exact certification of one feature is coNP-complete even for well-conditioned, weakly correlated rational dictionaries. Uniform amplitude bounds transfer the structured certificates to densely perturbed dictionaries and control actual collateral change. Reproducible synthetic validation includes exact algorithm checks, 120 whole-input certification instances, and checkable refusal witnesses.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, mechanistic interpretability, intervention identifiability, sparse inverse problems, robust certification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wNw8OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies an intriguing possibility: can interventions be identifiable and stable even when the underlying nonnegative sparse code is not unique? Rather than tying the analysis to the output of a particular encoder, the authors work with a fixed decoder and consider all admissible nonnegative kk-sparse explanations. That framing shift alone is a real strength. The main results are substantial: a signed-circuit characterization of exactly identifiable diagonal interventions, a sharp target-specific stability modulus, and an exact quotient theorem for repeated decoder directions. Beyond these, the paper delivers an O(mk4)O(mk^4) certification algorithm on forest Gram graphs, optimal binary selection on positive paths, and a shortest-path formula for attenuation when k=1k=1. Further results establish an unbounded binary-to-continuous collateral ratio on a structured family, coNP-completeness of general certification, and input-specific certificates under dictionary perturbations. On the validation side, the paper runs synthetic checks against exhaustive enumeration, rational counterexample verification, numerical LP comparisons, and 120 whole-input certification instances. This checks implementation consistency and certificate coverage thoroughly, though it stops short of evaluating interventions on trained SAEs.

Critical strengths
  1. The paper gives a precise and genuinely useful formulation of intervention ambiguity. The distinction between recovering a unique explanation and determining its intervention is one worth making, and the authors do far more than state it: they connect it to circuit structure, quantitative stability, and computational certification. Of the technical contributions, the budget-preserving forest algorithm and the prescribed-deletion hardness result are the ones I find most compelling. They give the framework real substance and are the main reasons I regard this submission as a borderline case rather than a clear rejection.

  2. The treatment of exact certification is careful throughout. Singular states, rational witnesses, and the difference between upper bounds and impossibility certificates are all handled explicitly, which is exactly what one wants to see in this kind of work. The reported exhaustive checks and independent witness verification are appropriate as well. I also appreciate that the authors resist the temptation to read the 43 uncertified robust instances as unstable. Overall, the paper is notably disciplined about what its results do and do not establish.

Critical weaknesses
  1. The manuscript does not yet make a sufficiently strong case for the originality of its overall contribution. A substantial part of the framework is assembled from established ingredients: conformal circuit decomposition, Lipschitz extension, duplicate-column aggregation, distance envelopes, and low-dimensional copositive representations. Their application here is certainly useful, but the number of theorems should not be confused with the number of independent advances. As I read it, the strongest novelty is concentrated in the sparsity-budget-aware forest algorithm, the intervention-specific hyperbolic reduction, and the prescribed-deletion hardness result. I was hoping for a sharper comparison with the closest existing results, particularly for the circuit characterization and the structured certification problem. The current positioning leaves me uncertain whether the paper represents a sufficiently substantial advance beyond a careful collection of specialized adaptations, and that uncertainty weighs heavily in my recommendation.

  2. The positive results are also too structurally limited, and their usefulness beyond those structures is not sufficiently established. Forest Gram graphs come with strong rank restrictions. The all-sparsity separation construction relies on orthogonal two-dimensional blocks and replicated columns, so its arbitrary overcompleteness and coding nonuniqueness arise primarily from replication. This is a perfectly valid existence result, but it does not touch the difficult higher-order dependencies among distinct directions that actually motivate the SAE setting. The perturbation certificates offer a route beyond exact structure, yet the experiments do not show how tight or scalable those certificates really are. The whole-input suite is small, and it increases reconstruction tolerance together with the perturbation through ε=δ+10−3\varepsilon=\delta+10^{-3}, so the coverage curve never isolates sensitivity to dictionary perturbation. There is also little empirical information about runtime, rational bit-length growth, or usefulness at larger sparsity budgets. Let me be clear that my concern is not simply the absence of a language-model experiment; a theory paper need not demonstrate steering. What the submission does need is a stronger case that its tractable regimes or transferable certificates illuminate a broader class of problems than the relatively simple constructions currently evaluated. At present, that case is not convincing enough for acceptance.

Questions for authors
  1. What are the closest prior results for the circuit characterization and the forest certification algorithm? Which parts require genuinely new arguments, rather than an application of conformal decomposition, kernel preservation, or standard Schur-complement dynamic programming? A precise comparison here could materially change my originality assessment.

  2. Do the existing whole-input records contain feasible-pair lower bounds that can be compared with the certified upper bounds? Can the available results indicate whether coverage loss is driven mainly by the perturbation term, the increasing ε\varepsilon, or conservatism in the amplitude bound? Let me stress that I am asking about evidence already available, not requesting a new experimental suite.

  3. Could you identify the existing scripts that check the original stability inequalities independently of the transformed formulations? Are runtime, memory, or rational bit-length measurements available for the submitted implementation, and what is the largest nontrivial instance it has been tested on? This would affect my assessment of the constructive contribution. I should add that broader dictionaries and trained-SAE evaluations would be welcome future work; my questions above concern the strength of the claims and evidence in the present submission.

Reviewer_trZnOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper focuses on the usage of sparse autoencoders (SAE) to explain neural activations and intervene (e.g., edit) the network. A central challenge of interventions is that each activation may be polysemantic and have multiple sparse representations. The central idea is to determine when interventions can be uniquely identified even when the underlying sparse representations cannot. The paper attempts to investigate how much the activation changes induced by an intervention can differ across alternative sparse representations of the same activation, providing a polynomial time algorithm for deciding if the amplification will be at most some value K when the dictionary induces a forest Gram graph, and showing that the decision problem is coNP-complete for general dictionaries. The paper demonstrates the performance of the polynomial time algorithm on synthetic examples.

Critical strengths

The paper identifies a meaningful distinction between the identifiability of sparse representations and the identifiability of interventions applied using those representations. Arguably in practice we are more interested in the uniqueness of the intervention, which may be achievable even when the representation is not unique.

The paper develops several concrete theoretical results, including a characterization of identifiable interventions, a polynomial-time certification algorithm under structural assumptions on the dictionary, and a computational hardness result for the general problem.

Critical weaknesses

Major weaknesses:

The paper as written is difficult to read and/or follow, which makes it difficult to evaluate its contributions. For example:

  • The introduction does not define terminology that’s central to the paper, and extensively discusses central concepts before they are defined. For example in the introduction, three separate questions are identified as addressed in the paper on lines 47 and 48. The first two questions contrast the “noiseless model” and the presence of “noise”. No preceding sentences define (i) what the “model” is (e.g., is it the classical SAE itself, the network that the SAE is interpreting, a model introduced by the paper, etc.) or (ii) how noise enters. As a non-expert for SAEs, I had to read the papers in the prior work section to find definitions of centrally-used terms to fully understand the question(s) the paper identifies and attempts to answer
  • Some results are presented without any context. For example, Section 6 is just (i) the statement of the theorem and (ii) a sketch of the proof without any surrounding context or interpretation.
  • Because the paper has space available for adding context, this would greatly improve the paper’s readability, especially for stressing the paper’s impact.

Given the apparent goal of steering an SAE, the lack of experiments demonstrating that the method is relevant for steering trained SAEs is a significant weakness. Although this is acknowledged explicitly as a limitation in the final section of the main body (lines 412 through 414), the limitation is a core weakness of the paper. It is to be shown if this is also a limitation of the method, especially since there is limited evidence from the paper (e.g., through references to prior work) that the synthetic examples in Table 1 reasonably predict performance in SAE steering.

  • My understanding is that the paper attempts to look at worst-case amplification over the set of sparse activations. It’s not clear to me (from the paper) how related this is to practical amplification for real interventions.
  • For example, although the amplification factor decision problem is in general coNP-complete, this is for particular K and Gram graphs with particular structure.
  • If the dictionary is randomly initialized, then all pairs of dictionary elements are non-orthogonal almost surely, so the Gram graph is complete almost surely, which is not the same structure as used in the clique reduction. Thus, to me it seems important to justify that the procedure through which the dictionary is assembled can produce these clique-like examples to make the argument relevant to practical SAE training.

The polynomial-time algorithm requires the Gram graph to be a forest, meaning that the graph of nonzero pairwise inner products must be acyclic. It’s not clear to me whether/how ordinary SAE training would produce such a dictionary.

  • Again, at a mathematical level (numerical precision aside), randomly initialized dictionary elements will be non-orthogonal almost surely. To use the algorithm for SAEs, I would want to see a concrete argument that you will get dictionary elements which are either (i) pairwise orthogonal or (ii) allowed to be pairwise non-orthogonal but not form cycles.
  • If it’s true that ordinary SAE training doesn’t produce such a dictionary, a training modification should be proposed, or an approximation-algorithm type analysis should be performed. Otherwise, I don’t see how the polynomial time algorithm is practically relevant.

The paper’s position in the literature is unclear. As a non-expert in SAEs, the proposed results are not contextualized against prior work within the paper itself, requiring that a user be familiar with what has been achieved in existing work:

  • There is no clear preliminaries section, meaning it’s unclear where discussion of prior work ends and claimed novel contributions begin.
  • There is limited effort to contextualize (what I interpret to be) the main body against the literature. In particular, there are two citations total in Sections 2 through 7: on line 255 to note that envelope argument for Theorem 7 is standard (the result itself is not contextualized in the literature), and on line 298 for the standard Karp (1972) paper reducing SAT to Clique (and other problems). If earlier sections (e.g. Section 2) are ought to provide preliminary information, citations to pre-existing work discussing the objects more foundationally would be very helpful for the reader to understand both (i) why those particular objects are chosen and (ii) what is novel.
  • For another example, the equivalence in Theorem 1 on lines 91 through 94 of Section 2 is a modified Lipschitz stability result, where instead of the classical ∥Dh∥≤C∥h∥\|Dh\| \leq C\|h\| the paper studies ∥DWh∥≤C∥Dh∥\|DWh\| \leq C\|Dh\|. There is no context given on how this result is positioned in the literature: is it entirely new (e.g., has no one considered stability bounds of this form), do such bounds exist but yours has (for example) less limiting/more appropriate assumptions for your setting, etc.?
  • the runtimes and bounds provided in the paper are not compared against similar algorithms or bounds in the literature
  • the explicit contrasts in the limitations section (e.g., “The complexity lower bound has an inverse-polynomial rather than constant gap”) do not establish if the contrasted runtimes/gaps are either (i) feasible and/or (ii) achieved by existing literature

Minor weaknesses:

  • The statement of Theorem 9 and the limitations section both emphasize the inverse-polynomial gap (in place of a constant-factor approximation gap). It would be helpful to discuss the precise implications of this.
  • The in-line proof sketch of Theorem 9 is very difficult for me to follow. It would be far easier to follow if you concretely establish that you are reducing Clique to the complement problem at the beginning of the proof, concretely show what the operator norm will be (since it explicitly has this “dangerous direction” that you refer to in words), etc. Please reference Appendix F explicitly as the location of the full proof, rather than only referencing that Appendix F contains a specific construction for Γ\Gamma.
Questions for authors

Please refer to the major weaknesses, namely:

  • How are your polynomial runtime algorithm and coNP-completeness reduction positioned in literature?
  • Do you expect either to be relevant for practical SAE dictionaries and why?
Reviewer_rZ7bOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies sparse autoencoder-based intervention on the activations of neural networks. Prior works study exact reconstructions of the sparse codes to interpret neural activations. This work aims for targeted interventions to the activations, circumventing the need for code recovery. Towards this goal, they establish some structural results and propose algorithms for certain structured input classes. They also establish computational hardness over broader input classes.

Critical strengths
  1. The problem seems natural and well-timed.

  2. The mathematical claims are correct as far as I could verify.

Critical weaknesses
  1. The presentation of the paper requires a lot of improvement. The current manuscript renders both the conceptual content and the technical details hard to follow.

  2. The paper reads like a collection of disparate mathematical claims. There is neither a well-defined technical question nor a proper roadmap through the inquiry. Consequently, it is hard to understand what the paper achieves beyond a list of theorems.

  3. Most of the terms are not defined clearly in the paper. What is a dictionary? What is a forest-gram graph? The paper needs a well-defined preliminaries section to be accessible.

  4. The technical presentation requires a more rigorous approach. For a simple example, at the beginning of Section 2 (Lines 77-78):

        (a) Which norm are the columns ‘unit-norm’ in? 
    
        (b) In the definition of $\Sigma^+_k$, what is $z$? 
    
        (c) What do you mean by “deletes group $G$”?
    
  5. The related work discussion seems very compressed. This can be addressed through a proper problem formulation.

Questions for authors

See the "Critical Weakness" section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.