acceptodds
Withdrawn from ICLR 2027

A Reporting Standard for Sparse Autoencoder Ablation Scores

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Ablation scores for sparse autoencoder latents disagree across dictionaries partly because each dictionary chooses the token at which its own latent is scored. A latent is scored by switching it off at one token and measuring how much the model's output changes, and by convention that token is where the dictionary's latent fires hardest. We isolate the effect with autoencoders trained from a shared initialisation, each differing in a single fitting choice. Across seven base models from four families, the choice of token carries at least 46.6% of the variance in the measured effect, about as much as the choice of latent. We propose a reporting standard: record the measurement position, and when dictionaries are compared, fix it outside all of them. Fixing the tokens makes dictionaries rank latents more consistently, with a bootstrap interval excluding zero on every model at the main corpus size. Both items take a line of evaluation code, and the second needs no more forward passes than letting each dictionary choose. Left to choose, dictionaries rarely agree on where to measure a latent, and agree less as the evaluation corpus grows. Released Qwen-Scope and Gemma Scope dictionaries measure most latents matched across a pair of dictionaries at different tokens, including many that the pair encodes almost identically. Code: https://anonymous.4open.science/r/sae-artifact-8E6D/

Primary area:
interpretability and explainable AI
Keywords:
sparse autoencoders, ablation, causal effect, reporting standard, variance decomposition, dictionary learning, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_sUmyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how token selection affects SAE ablation scores. It compares measurements at each dictionary's preferred positions with measurements at shared positions, then estimates variance components and ranking reliability. Experiments across seven models motivate a reporting standard for position selection and dictionary comparisons.

Critical strengths
  1. Figure 1b directly shows substantial variation across positions within a single dictionary. The released-dictionary comparisons in Figure 3 show that even similar decoder directions often select different tokens. These observations support documenting measurement positions.

  2. The controlled comparisons use the same retained latents in both measurement designs. The alternative selection rule and fixed-latent analyses in Appendices C and D address useful robustness questions, including cases where the result weakens.

Critical weaknesses
  1. The variance analysis is not specified sufficiently to assess the headline reliability gains. Section 3 omits the full moment equations, and the response scale is unclear. Shared positions can induce covariance across arms within a latent. If the estimator treats those observations as independent replicates, subtracting within-cell variance from the interaction mean square can underestimate the interaction, with clipping producing zero estimates. This is a concern about an unspecified estimator, not a demonstrated implementation error. Latent bootstrapping alone does not resolve it. The reported reliability improvement also follows algebraically from the estimated interaction reduction, so it is not independent validation of that reduction.

  2. The comparison population limits the proposed standard. Both shared-position rules require every arm to fire and retain only 22% to 50% of sampled latents. Appendix C changes the ranking rule but preserves this conditioning. Greater agreement on this subset does not establish whether excluded differences reflect measurement noise or meaningful differences in feature support. Shared initialization also does not guarantee that corresponding features remain equivalent after training. For released dictionaries, the paper establishes position disagreement but does not show how shared measurements change a substantive dictionary comparison.

Questions for authors
  1. What response transformation and moment equations produce Table 1? Can an analysis that explicitly models shared-position covariance recover the reported gains and agree with directly measured ranking consistency?

  2. What target population does the standard aim to measure? Can the existing results show a substantive dictionary comparison that changes under the proposed protocol, with feature correspondence checked after training?

Reviewer_5vgtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper claims to propose a reporting standard for sparse autoencoder ablation scores. It is unclear what ablation score they mention, what is the motivation for this reporting standard or how the reporting standard is useful. The reporting checklist is also basically unreadable and is not a real checklist.

Critical strengths

None

Critical weaknesses

The authors claim that LLMs where used to '(write) first drafts of sections' which they 'then revised'. I find this unlikely as the text is very unclear (and sometimes wrong) from start to finish. To report a few of those examples: "To show that a part matters, a practitioner switches a latent off, runs the model again, and reads how much the output changed (Marks et al., 2025; Gao et al., 2025; Cho et al., 2026). The number is reported as a score for the latent and set beside the scores other papers report for other dictionaries." A part? Switches the latent off and then runs the model again? Which of these papers reports a 'score' for the 'latent'? What does "set beside the scores other papers report for other dictionaries" mean?

I don't agree that the reporting checklist is a contribution. It's hardly a checklist, as it seems to mix both the thing that the practicioner should do and the reason for them to do it. I am unsure if I understand the other contributions.

The reported TopK SAEs are trained on 12M tokens, which is an obviously low number for SAEs. Unclear if any of the results reported survive to actually training the SAE for a singificant number of tokens. I also have a hard time understanding what does it mean to train a SAE without a decoder, or what soft frozen means.

Its very hard for me to understand the results of section 5 6 and 7 so I will stop my review here, but it is my understanding that this paper ins not in a state to be accepted at ICLR. Happy to revisit if other reviewers disagree that this is a waste of our time.

Questions for authors

What is the reconstruction of the SAEs trained?

Reviewer_jnsfOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents an issue of selecting the max activation first and then ablating activations to measure the difference in output distributions. Also, this work shows that training an SAE from a similar initialization with different parameters leads features with similar indices to activate on different examples. The authors also show that measuring ablation scores at the same token positions makes feature rankings more consistent across SAE variants in their controlled experiments. They recommend reporting how positions are selected and using the same positions when comparing SAEs.

Critical strengths
  • The paper gives a clear practical recommendation on ablation scores in a setup where a feature is selected first.

  • The result could be interesting for future work on ablation scores.

Critical weaknesses

W1. The paper is hard to follow, starting from the introduction, which references Figure 1. To understand Figure 1, the reader should know what the six branches are (which turn out to be SAE variants, instead of my first guess about untouched and ablated results), and Table 1 requires knowledge of what "cells" are, which are also complex constructions.

W2. Speaking about comparisons across the 6 branches, some of the differences for me could be explained by the fact that their features are different. Even in Table 1, which selects positions where corresponding features co-activate, I still feel like, for example, a feature from one branch could be more general than a feature from another branch, and this leads to analyzing two different features. Shared initialization does not guarantee that corresponding features retain the same meaning. So Figure 1a shows differences in selected positions, but how much of this comes from differences between the learned features?

W3. Figure 1b uses only six probing positions per feature. I am concerned that this small number of examples, or the finite corpus used to select them, may affect the observed spread. How stable are the conclusions when more positions are probed and the selection corpus is enlarged? Is the selected feature the top-1 feature at each token, or do the examples include positions where it ranks 10th or lower among active features?

W4. Also, where are the unnormalized scores corresponding to Figure 1b and the main variance analysis? Section 4 and Appendix A give a raw-versus-normalized comparison, but I would like to see both readouts for the main results. Do we need to normalize when the goal is to measure the effect of removing a feature's actual contribution? Why is KL divided by the perturbation norm the appropriate quantity here? A reversal between raw and normalized scores shows that the choice matters, but does not establish which quantity is appropriate for the intended evaluation.

W5. More serious concerns, however, concern motivation. Different ablation scores in different contexts do not by themselves show that the evaluation pipeline needs to change. How much previous work is affected even if the experimental design is fine? For TPP and SCR, features are selected based on predefined task examples; examples are not selected based on a feature's strongest activations. The paper excludes these evaluation families, but which included results are affected by feature-specific position selection? Can the authors give examples? Can you point to results from previous work that would change if one followed the recommended checklist?

W6. Also, I want to see geometrical metrics for features from the six controlled arms, such as cosine similarity between their decoder vectors. Figure 3 and Appendix B provide this analysis for released dictionaries, but what happens to Table 1 if we filter out feature pairs with cossim < 0.9?

W7. Appendix G does not give enough detail to verify the literature audit. Can the authors identify the exact experiments, passages and code paths supporting the classifications in Table 2?

W8. Table 2 seems to misclassify some evaluation procedures. Bricken et al. describe ablating a feature throughout a context, and Cho et al. average over all positions where a feature is active, rather than using one position per latent. Also, position information can be recovered from released code or artifacts for inspected examples or procedures in at least 3 out of 5 papers (Gao et al., Marks et al., Bricken et al.). Although the table audits the methods text, it should distinguish missing information in that text from information that cannot be recovered from released materials.

Questions for authors

See Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.