acceptodds
Under review as a conference paper at ICLR 2027

Dictionary Size and Dead Latents Handling Jointly Set the Recovery-Optimal Sparsity of Top-K Sparse Autoencoders

Abstract

Sparse autoencoders (SAEs) are trained to reconstruct activations under a sparsity constraint, but what interpretability requires is the recovery of the features underlying them. In a Top- SAE toy model with a known ground-truth dictionary, we judge an SAE by the atoms it actually uses rather than by its nominal width, and partition its live atoms against the ground truth and against a second training run. Against the ground truth, the recovery-optimal sparsity is set jointly by dictionary size and dead-latent handling: below the ground-truth dictionary size it rises with the ratio of dictionary size to feature count, whereas above it the dead-latent handling sets its level, and when dead latents are appropriately handled, every feature at the optimum is carried by a single live atom. Against a second run, the same structure appears at the optimum, agreement and recovery come apart elsewhere—a low shared fraction arises even when every atom is correct—and the overlap between the runs, read as a capture–recapture experiment, estimates the number of features to within its sampling error at every feature count and dictionary size we tested.

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, mechanistic interpretability, feature recovery, superposition, Top-K

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nsQXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents a controllable study of how recovery of ground-truth features by sparse autoencoders is controlled by handling dead latents, and how it corresponds with the optimal sparsity; additional point of scrutiny is what happens when SAE dictionary size is lower than the number of ground-truth features, and therefore cannot represent all of them, and how it connects to the cross-seed instability problem. Authors subsequently present the method to estimate number of ground-truth features from data and trained SAEs. Alongside other findings, authors found that dictionary size is also a component of optimal sparsity choice.

Critical strengths

S1. The result that recovery can be controlled by handling the presence of dead latents in SAE seem novel to me.

S2. Authors test a method to choose sparsity from cross-seed feature agreement and found that it comes apart with the recovery, they drive away from each other outside recovery-optimal conditions. I think this is nice result, but see W2.

Critical weaknesses

W1. Writing is heavily convoluted due to bad practices in using AI for writing assistance. It is very hard to get to the substance of results and contributions due to it and distinguish it from my interpretation. Most empirical sections are just number and formula dumps, and most connections to prior literature establish what was done in it, but not how the current paper differs or build upon; many terms are used before they introduced. This also hurts reproducibility - although there are many details stated, I'm not sure that a reader would be able to follow the procedures.

W2. Main results seem to be too toy; I found conceptual results to be little explained from first principles, and empirical results to be built on narrow setups with unsatisfactory design. Experiments are conducted on one configuration of toy model generator of feature activations, only number of features is controlled, which is currently below standard (see SynthSAEBench for example). Unclear whether any claim breaks when going beyond setup in the paper, which is important for transfer to real applications. It is unclear whether trained SAEs converged and were trained well enough. On related note, the connection to stability problem is strangely handled: authors use only two runs to establish which features reproduce and which are not. This does not tell us much and is also below the standard of experimental design for any seed-related problem (see Paulo & Belrose and recent Unstable Features, Reproducible Subspaces papers).

Questions for authors

Q1. You propose an approach to count the GT features. Have you compared your approach accuracy with the prior literature on that matter?

Q2. Can you extend your study on different data configurations to test generalization of your findings? I'm mostly interested on whether a broad class of toy models supports your conclusions, and how different parts of it change them (the coactivation structure, for example).

Q3. Is there a way to theoretically explain why your conclusions hold, and use it for prediction? How practical are your conclusions, how do you see them extending towards real-world SAEs?

Q4. Can you please state what is the delta between your work and prior work in this field? What exactly your paper introduces that has never existed and could never be derived from prior work, and what you build upon?

Q5. Can you please explain in plain terms what exactly one atom per feature result means and what its consequences, in your view? I have not understood it well enough.

I support this direction of study because it gives interpretability proper ground, but I'm unwilling to support the paper in its current form. I would like to see substantial rewrite for proper communication of results (which I do not expect during rebuttal) and extension of empirical material, with proper handling of any deltas.

Reviewer_zmfbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The subject of the paper is sparse autoencoders (SAEs). The paper reports several computational experiments along with observations to which the experiments lead about the specific setting of these experiments.

Critical strengths

Regretfully, this paper does not have any strengths.

Critical weaknesses
  1. This paper is so poorly written that it is very hard to understand what problem it tries to solve and to what conclusions it leads. It is a report on experiments performed with some vague presentation of their results.
  2. The experiments presented in the paper are based on a single not specified dataset, so they do not lead to any general conclusions.
  3. The paper does not provide any general knowledge or hypotheses.
Questions for authors

What was the motivation for this research? What is its potential real-life usefulness?

Reviewer_JKj3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper builds a toy model which invents the concepts and then trains SAEs on the resulting data and checks it. They argue that you should judge SAE on the slots it uses which are sorted by them into correct ones, duplicates and junk. They argue that the score can look good while junk hides underneath. They find that best sparsity follows a known formula when SAE is small, and is equal to the true concepts per input when its large but only if the unused slots are revived well. If it works, a SAE with 20k slots, ends up using 5000, one per concept. It is also showed by them that two SAEs can disagree by chance if they are too small to hold everything and that low agreement doesnt prove that the features are fake. It also lets them estimate the total number of concepts. The validation/checks are: 4 seeds, several thresholds, three problem sizes and predictions that match what they observe according to my understanding. However, its all a toy model with equally common and independent concepts and nothing tested on a real model.

Critical strengths

The main contribution is a chance baseline for run-to-run agreement. When an SAE has fewer live atoms than features, two fully correct runs share only the overlap expected from random subsets, and maximizing agreement can select a much worse sparsity. This matters because prior work uses consistency both as a proxy for recovery and as evidence against SAE features, without a null model for either reading.

Critical weaknesses

All results come from a toy model with uniform, independent features, a random Gaussian dictionary, and a single (m, s), and the key conclusions depend directly on these assumptions. The reference curve, the hypergeometric agreement baseline, and the Lincoln Petersen estimator all assume equal feature frequencies. under heterogeneous frequencies the baseline understates overlap and the estimator is known to be biased low. The Gaussian dictionary is also a low-coherence regime where sparse recovery is well posed, which real activations are not likely to meet. No experiment tests any finding outside these conditions.

The paper shows what the best sparsity was in its experiments but does not give a way to find it. Above d = n, the best K shifts unpredictably with the AuxK budget, and the authors cannot explain why. Their formula for the best K also needs the true number of active features per input, which is unknown in practice and which the paper does not estimate. The paper does not establish a reliable, ground truth free procedure for selecting K, particularly in the overcomplete regime where the optimum depends non monotonically on AuxK

Questions for authors

Do the main findings hold under non uniform feature frequencies? For example, with power law firing, does the agreement baseline still match observed overlap? Does the capture recapture estimate stay unbiased? Where does K* fall relative to the reference curve?

What does the run to run analysis show on a real SAE? Even one layer of a small language model: does the shared fraction match the chance prediction of equation 6, or exceed it as unequal frequencies would predict?

Appendix A notes that the amplitude range Uniform(0.5, 1) was chosen to keep all active features above the interference floor, and that under Uniform(0, 1) 27% would fall below it. How sensitive are the optimal sparsity and the one to one recovery result to weaker or heavy tailed amplitudes, where some active features fall below this floor?

1 comment

I believe there is no rebuttal from the authors to my review or to any review... Sadly, I have nothing to comment on.. thank you authors for the paper, nonetheless!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.