- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper shows that covariance tests which refit coordinate-wise variances can miss axis-aligned features, even when sparse autoencoders can represent and recover them. It also derives alignment certificates and guarantees for preserving PCA-estimated directions, with positive synthetic results but no recovery advantage on the tested language-model residual backgrounds. The paper theoretically characterizes a detector specific blind spot and guarantees alignment under explicit constraints (known as anchor constraints), while empirically showing that SAEs can recover detector-invisible features and that PCA anchoring helps in synthetic settings but offers no recovery-rate advantage on the tested language-model residuals.
Critical strengths
I think it is nice that this paper separates statistical detection, decoder representability, and actual recovery through training. The problem is well motivated
Critical weaknesses
Limited novelty PCA eigenvalue/eigenvector transitions are classical spiked-covariance results, and finite-sample alignment bounds follow from standard eigenvector perturbation theory. Here, Theorem 4 applies that machinery and then freezes the estimated direction; Theorem 5 adds a bounded angular displacement. These are useful constructions, but do not establish a new recovery threshold or explain SAE learning dynamics.
Insufficient substantive comparison with dictionary learning Spielman–Wang–Wright establish recovery of sparsely used dictionaries, and Arora–Ge–Moitra give guarantees for learning incoherent, overcomplete dictionaries. Those works address identifying unknown dictionary directions from sparse observations—a more direct version of the motivating feature-recovery problem. Their assumptions and algorithms differ from modern SAEs, so they do not automatically subsume this paper, but the authors should explain precisely what unresolved problem their results address.
Detector is Narrow Contribution The visibility coefficient is a clean calculation, but exact axis blindness follows because the alternative belongs to the composite diagonal-covariance null. The potentially distinctive contribution is its explicit geometric characterization and juxtaposition with SAE recovery—not the general observation that nuisance fitting removes statistical information.
Limited practical evidence Synthetic successes occur in favorable settings, while the matched language-model residual experiments show no recovery-rate advantage. The theory’s relevance to realistic SAE failures therefore remains unestablished.
Experimental Limitations
The experimental evidence is too narrow to establish a substantive advance in SAE feature recovery. The main synthetic experiments demonstrate that a deliberately restricted covariance detector can fail where an SAE succeeds, but do not explain realistic SAE failures; the anchor experiments largely test preservation of a direction already recoverable by PCA, without establishing a clear benefit beyond PCA itself. Moreover, the matched GPT-2/Pythia experiments show no recovery-rate advantage, and the evaluation lacks a systematic study of weak targets amid stronger competing features, correlated activations, and varying noise and sparsity. Although the authors transparently report negative results and limitations, the experiments primarily illustrate the conceptual distinction rather than establish a broadly useful recovery mechanism.
Questions for authors
-
What is the precise theoretical novelty beyond established dictionary-learning and spiked-covariance results? Which theorem provides a new recovery insight, rather than an application of standard perturbation bounds or a consequence of preserving an estimated direction?
-
What practical SAE procedure corresponds to the proposed detector? Why should blindness of a test that refits coordinate variances help us understand failures of reconstruction-trained SAEs?
-
What does SAE training add beyond PCA in the anchor experiments? Can you demonstrate improved feature recovery, latent-code accuracy, or reconstruction relative to PCA alone and an unconstrained PCA-initialized SAE?
-
Does the approach recover weak features that are not leading principal components? How does performance change with stronger competing features, correlated latent activations, and nonorthogonal dictionary directions?
-
Why does the anchor advantage disappear on language-model residuals? Is the failure attributable to initial PCA estimation, subsequent optimization, or the usage-based atom-selection rule?
-
How does the subspace-alignment certificate translate into individual feature recovery? What prevents a certified subspace from containing the target without any atom—or its activations—meaningfully representing it?
-
Can the main detector–decoder comparison be repeated with the correctly calibrated correlation statistic and equal sample budgets? This would separate the conceptual distinction from calibration and data-budget differences.
-
How robust are the empirical conclusions across dimension, dictionary width, sparsity, and training budget? Can you report activation-level recovery alongside atom cosine, with chance baselines adjusted for dictionary size?
4 comments
Response to Reviewer D64C (2/3): beyond PCA, and weak non-leading features (Q3–Q4)
Q3 / What SAE training adds in the anchor experiments. The anchor experiments test whether training preserves a data-estimated direction while fitting a full sparse dictionary—the prerequisite for any anchor-based recovery guarantee. Two comparisons address your question directly:
Against random anchors, with everything else matched (data, seed, optimizer, non-anchor initialization; 192 pairs): training-PCA anchors yield 183/192 recoveries versus 21/192 (paired difference +0.844, 95% CI [0.797, 0.891]). Against an unconstrained PCA-initialized SAE (dose-response control, eight independent anchor groups): every mechanism recovers 8/8, but the unconstrained SAE lets residual atoms leak into the anchor direction (maximum residual–anchor dot product 0.734 versus ≤ 0.102 for every constrained mechanism) and has the largest and most variable held-out loss (0.1414 ± 0.0077 versus ≤ 0.1373). The structured family therefore delivers controlled leakage and stable reconstruction on top of recovery.
Unlike PCA, the SAE also learns sparse codes for a full overcomplete dictionary. We tested this directly on SynthSAEBench-16k (768 dimensions, 16,384 features, correlated and hierarchical firing; five seeds): BatchTopK reaches MCC 0.754, versus 0.629 for FastICA and 0.171 for PCA, and learns an atom aligned at cosine ≥ 0.80 for 73 of 1,280 pre-specified target features, versus 1 for FastICA and 2 for PCA. Our pipeline reproduces the public SynthSAEBench checkpoints (MCC 0.753 vs. 0.757–0.758).
Q4 / Weak non-leading features, competing and correlated features. Our residual-background experiments address this directly. We inject a known feature into real GPT-2-small and Pythia-70M residual-stream activations, where it must be learned among many naturally interfering, correlated, and non-orthogonal features. One injected direction is the 32nd principal direction of the empirical background, the weakest of the 32 retained components rather than a leading one. At , the trained dictionaries contain an atom aligned with this non-leading direction (cosine ) in 8/8 (PCA anchor) and 6/8 (random anchor) GPT-2 conditions, and in 8/8 and 6/8 Pythia conditions. These alignments are not chance coincidences: random 64-atom dictionaries in this space reach cosine 0.80 in 0 of 20,000 draws. Recovering such a feature therefore does not require it to be a leading principal component.
Response to Reviewer D64C (3/3): LM residuals, certificate, equal-budget experiment, and robustness (Q5–Q8)
Q5 / Why the anchor advantage disappears on LM residuals. We re-scored the archived runs (48 matched pairs per model; no retraining), counting conditions with absolute target cosine ≥ 0.80 at three points: the initial anchor, the best final atom, and the usage-selected atom.
GPT-2, training-PCA anchor: 40/48 initial anchor, 32/48 best final atom, 0/48 usage-selected atom. GPT-2, random anchor: 0/48, 30/48, 0/48. Pythia, training-PCA anchor: 44/48, 41/48, 1/48. Pythia, random anchor: 0/48, 21/48, 1/48.
The answer to your question is clear: initial PCA estimation is accurate on both backgrounds (40/48 and 44/48), and the trained dictionaries usually still contain a well-aligned atom (32/48 and 41/48 with PCA anchors). What fails is the pre-specified usage-based rule for choosing which atom to evaluate, which almost never selects that atom. The aligned atoms are not chance coincidences: random 64-atom dictionaries in this 32-dimensional space reach cosine 0.80 in 0 of 20,000 draws. The bottleneck on LM residuals is therefore primarily atom selection rather than anchor estimation.
Q6 / From the subspace certificate to individual features. Theorem 3 is deliberately objective-agnostic and subspace-level: it certifies any frozen decoder, however trained, from a held-out reconstruction comparison. Atom-level guarantees are the role of Theorems 4–5, which add the anchor-preserving structure that pins the target to a specific atom. The two results are complementary: Theorem 3 audits an arbitrary trained subspace, while Theorems 4–5 guarantee a specific atom when the structure is imposed.
Q7 / Calibrated statistic and equal sample budgets. We ran exactly this comparison. Following a protocol fixed before execution, the correlation statistic and TopK now use the same 20,000 examples per condition, and the detector is calibrated at that sample size (empirical size 4.5–5.1% across four diagonal-Gaussian variance scales). At the primary strength (, 8 seeds):
Detector power on independent draws: 4.4% (142/3200) for the axis-aligned feature vs. 100% (3200/3200) for the dense feature. Detector rejections on the exact matrices used to train TopK: 0/8 (axis-aligned) vs. 8/8 (dense). TopK recovery (decoder cosine ≥ 0.80): 8/8 (axis-aligned) and 8/8 (dense).
On the very same training matrices, the detector never flags the axis-aligned feature, yet TopK recovers it in every seed. The axis-aligned power stays at the nominal level at all three strengths (4.4%, 4.4%, 4.8%), as the theory predicts. The contrast is therefore not a product of calibration or of unequal sample budgets.
Q8 / Robustness, activation-level recovery, chance baselines.
Dictionary width and sparsity. In the independent held-out grid with eight new seeds, axis-aligned features are never recovered less often than matched dense features: 8/8 vs. 8/8, 8/8 vs. 8/8, and 6/8 vs. 5/8. The widest configuration is budget-limited rather than geometry-limited: continuing the same trajectories to 16k steps gives 8/8 (axis) and 7/8 (dense). Objectives and capacities. The anchor-preserving families pass 48/48 rows across three objectives (MSE, MSE+, MSE+) and two capacities each, and all nine dose-response mechanisms recover 8/8. Strength. Across the strength sweep, TopK recovery rises with strength in both geometries, while the independence detector keeps its axis–dense gap. Chance baselines adjusted for dictionary size. For E4-v2 (16 atoms in 8 dimensions), random dictionaries reach mean maximum cosine 0.68; TopK reaches 0.970 (axis) and 0.978 (dense). For the LM experiments, random dictionaries never reach the 0.80 threshold (0/20,000).
Thank you again; these questions helped us sharpen both the theory’s role and the empirical evidence. We would be happy to answer any further questions at any time, and we look forward to your response.
Response
Thank you for the clarifications and additional experiments, which address the calibration concern and strengthen the evidence for atom-level recovery. However, the central concerns remain: the anchor guarantees largely preserve an already estimated direction by construction, and the experiments do not clearly establish recovery of genuinely weak, non-leading features or benefits attributable to the proposed constraints. These leave the novelty and practical significance insufficiently established, so I will maintain my score.
Response to Reviewer D64C (1/3): theoretical novelty and the role of the detector (Q1–Q2)
Thank you for the detailed review and for recognizing the value of separating statistical detection, decoder representability, and recovery through training. Your eight questions map closely onto analyses we have run; we answer them in order and include one new experiment run for this response (Q7).
Q1 / Novelty relative to spiked-covariance and dictionary-learning theory. The paper’s theoretical contribution is a set of results that hold uniformly where existing tools are usually applied to one estimator at a time:
Detector side (Proposition 1, Theorem 1, Corollaries 2–3). We express the evidence available to a detector class as the norm of the feature score after projecting out that class’s nuisance directions. For the frame-diagonal class this yields a closed-form geometric factor, 1−∥B⊤w∥44, which ties evidence loss to the detector frame rather than to the feature. This makes a precise, testable prediction—rotating B at fixed strength changes independence power but not label power—which our rotation experiment confirms (r=0.977 with the predicted coefficient; label power ≡1). Recovery side (Theorems 4–5). The new insight is not a perturbation bound by itself but its uniformity: for anchor-preserving decoder families, the same recovery guarantee holds for every objective, capacity, optimizer, and initialization that respects the constraints. This is exactly what lets the guarantee hold without modeling SAE learning dynamics. Empirically, the exact-anchor family recovers the target in 48/48 rows across three objectives, two capacities, and two geometries (minimum cosine 0.9989; family-wise certified lower bound 0.9405), and the trainable bounded-drift family also passes 48/48 (certified lower bound 0.8543).
Spielman–Wang–Wright and Arora–Ge–Moitra establish identifiability of unknown dictionaries for specific algorithms under sparsity and incoherence assumptions. Our question is complementary: what a specified detector’s failure implies about recoverability by reconstruction-trained SAEs, and which structure makes recovery objective-agnostic. We have added this comparison to the related-work section.
Q2 / What practice the detector corresponds to, and why its blindness matters. The frame-diagonal detector formalizes a common auditing step: standardize or whiten each coordinate, then look for remaining dependence, for example to decide whether activations or SAE residuals still contain an unexplained feature. Its blindness matters because a silent check of this kind is easily read as “no feature here” or “the feature is unrecoverable.” As you note in your review, the distinctive contribution here is the explicit geometric characterization of this blind spot and its juxtaposition with SAE recovery; this is precisely what the paper sets out to provide. Our matched control shows that this inference fails exactly for features aligned with the check’s coordinates: the detector’s power is at the nominal level, yet a reconstruction-trained TopK SAE recovers the same feature in every seed (Q7 below).