acceptodds
Under review as a conference paper at ICLR 2027

Sparse Autoencoders vs. Transcoders: Encoder Direction Use and Single-Feature Discrimination

Abstract

Sparse autoencoders (SAEs) reconstruct MLP outputs, whereas transcoders (TCs) predict them from the corresponding MLP inputs. We compare their single-feature discrimination and examine how encoder direction use and JumpReLU contribute to the difference. Across all 26 layers of Gemma 3 1B, we construct shared MLP-output clusters, select each coder's best feature by postactivation AUROC, and evaluate on held-out documents. TC achieves higher postactivation AUROC in 24 of 26 layers under both sparsity settings, despite SAE having higher mean preactivation AUROC. JumpReLU maps distinct preactivation values to zero, producing a larger discrimination loss for SAE. In an exploratory intervention, reducing TC's reliance on input principal components using an SAE-derived reference narrows its postactivation advantage more than matched random rotations, identifying encoder weighting of high-variance input directions as a partial contributor. Separately, TC retrieves approximately three and four additional target sentences per 200 reviewed under the lower- and higher-activity settings, respectively. Together, these results link single-feature discrimination to both encoder direction use and learned activation thresholds.

Primary area:
interpretability and explainable AI
Keywords:
Sparse autoencoders, Transcoders, Single-feature discrimination, JumpReLU, Encoder direction interventions, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_mv3cOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper compares sparse autoencoders (SAEs) and transcoders (TCs) in terms of single-feature discrimination. Using pretrained Gemma Scope 2 checkpoints on Gemma 3 1B, the authors evaluate how well individual SAE and TC features discriminate shared MLP-output clusters across 26 layers and two sparsity settings. The main finding is that TCs achieve higher postactivation AUROC than SAEs, despite SAEs generally achieving higher preactivation AUROC. The authors investigate this difference through JumpReLU thresholding and encoder-direction interventions. They also conduct entity detection and sentence retrieval experiments on Few-NERD to examine the practical implications of their findings.

Critical strengths
  1. Controlled comparisons. The authors construct shared MLP-output clusters as common evaluation targets and use held-out documents for evaluation. This provides a consistent setting for comparing the discrimination capabilities of SAE and TC features.

  2. Additional practical evaluation. Beyond the analysis of feature discrimination, the paper includes entity detection and sentence retrieval experiments on Few-NERD to examine whether the observed differences translate into practical benefits.

Critical weaknesses
  • W1. Limited experimental scope and lack of generalizability.

All experiments are conducted on a single relatively small language model, Gemma 3 1B, using the corresponding Gemma Scope 2 checkpoints (Section 3.1). Although the authors evaluate all 26 layers and two sparsity settings, these experiments remain restricted to one model family and one model scale.

This raises concerns about whether the observed differences between SAEs and TCs are general properties or specific to the selected model and checkpoints. The paper would be substantially more convincing if the conclusions were validated across multiple model families and scales, such as Qwen3, Qwen3.5, and OLMo 3.

  • W2. Insufficient motivation and unclear significance of the findings.

My main concern is that the paper presents extensive technical analyses without sufficiently establishing why these analyses matter. The paper investigates single-feature discrimination using MLP-output clusters and AUROC. However, the connection between this evaluation setting and the broader objectives of mechanistic interpretability is not adequately motivated. In particular, it remains unclear why improved discrimination of these clusters should be considered an important property of SAEs or TCs, and what practical implications follow from the observed differences.

While the authors distinguish their evaluation from prior work on automated interpretability, the paper does not sufficiently explain why the proposed evaluation perspective addresses an important limitation of existing approaches.

The practical evaluation on Few-NERD also provides relatively modest improvements. As shown in Table 6, TCs retrieve only approximately 2.75 and 4.35 additional target sentences per 200 reviewed under the two sparsity settings. These results do not seem sufficient to establish the broader practical significance of the extensive analyses presented in the paper.

Overall, the paper contains considerable technical detail, but the connections among the research motivation, experimental findings, and broader research significance are not clearly established. This makes it difficult to understand the main contribution and why the findings should matter to the broader research community.

Questions for authors
  • Q1. Incomplete reporting of confidence intervals. In Tables 3, 5, and 6, the authors report 95% confidence intervals only for selected derived quantities, such as additional AUROC reduction, AUROC recovery, and the TC–SAE performance difference. However, confidence intervals are not provided for the underlying measurements, such as the Original, Random, and Targeted AUROC values in Table 3, or the individual SAE and TC results in Table 6. Could the authors clarify why uncertainty estimates are reported only for these selected quantities? Reporting confidence intervals for the other measurements would provide a more complete picture of the statistical reliability of the experimental results.

  • Q2. Clarity and motivation in the abstract. The abstract immediately introduces SAEs and TCs and begins comparing their single-feature discrimination, without first establishing the broader research problem or explaining why this comparison is important. The evaluation task itself is not clearly introduced before the experimental findings are presented. Moreover, the abstract primarily lists experimental observations, making it difficult to identify the main scientific contribution and its significance. Could the authors revise the abstract to more clearly explain the research motivation, the problem being addressed, and the key contribution of the paper?

1 comment
AuthorsOpenReview ↗

Response to Reviewer mv3c

Thank you for asking how far the 1B results generalize and why single-feature discrimination matters for interpretability. We extended the comparison to three more model sizes and revised the paper to explain what this evaluation tells us that a final feature score alone cannot.

W1. Do the findings generalize beyond Gemma 3 1B?

What held across model sizes was that SAE features lost more discrimination through JumpReLU, not that TC features always performed better afterward. We evaluated every layer of Gemma 3 270M, 1B, 4B, and 12B under both sparsity settings. In all eight comparisons, the selected SAE features had higher mean AUROC before JumpReLU. In 270M, 1B, and 4B, SAE lost enough of that initial advantage for TC to finish ahead. In 12B, SAE retained its advantage after JumpReLU (see Table 1).

The input-direction result also held in 12B despite this reversal. For each coder, we removed the contribution of its own first input principal component (PC1) and compared the result with random edits matched for rotation angle and final norm. In the layer averages for all four models and both settings, removing PC1 reduced preactivation AUROC more than the matched edits for TC features and less for SAE features (Figure 4). These experiments extend the comparison across sizes within Gemma 3. We have not tested other model families.

W2. Why does output-cluster discrimination matter, and how does it relate to auto-interpretation and practical use?

For a feature to serve as an indicator of a pattern, it should give higher scores to tokens with that pattern than to tokens without it. If both receive equally high scores, the pattern does not explain why the feature responds. We grouped tokens by MLP output direction because the SAE reconstructs the same MLP output that the TC predicts.

Both auto-interpretation and our evaluation ask whether the relationship between a pattern and a feature’s response holds on new examples. In auto-interpretation, a description inferred from activating examples is used to predict where the feature will activate in new text (Paulo et al., 2024, 2025). We selected one feature from each coder for the same output cluster and tested whether it gave higher scores to tokens in that cluster than to other tokens in held-out documents. Comparing the same features before and after JumpReLU showed something their final scores alone could not. Where TC features distinguished the output patterns better after JumpReLU, it was because they lost less discrimination during sparsification, not because they distinguished the patterns better to begin with.

We agree that the practical gain on Few-NERD was modest. Among 200 retrieved sentences, TC found an average of 2.75 more target sentences than SAE under small and 4.35 more under big (Table 4). TC features also had higher mean AUROC after JumpReLU on human-annotated SemCor noun categories in 1B. Unlike on the output clusters, however, TC was already ahead before JumpReLU in both settings (Table 3).

Q1. Can you give confidence intervals for the individual conditions?

Yes. Table 26 in Appendix B.12 now reports separate estimates and 95% bootstrap confidence intervals for SAE and TC. It covers the original, random, and targeted rotations in Table 2(b), the three leading-16-PC removal conditions in Table 18(a), and Few-NERD detection and retrieval in Table 4. We kept the selected features fixed and resampled held-out documents for AUROC and test sentences for Few-NERD.

Q2. Can the abstract explain the motivation and task before presenting the results?

We revised the abstract to start with why the comparison matters. Researchers use individual SAE and TC features to interpret a model, so it matters which distinctions in the input survive in a feature’s response. The abstract now explains how we select and evaluate features against the same MLP output patterns before presenting the two findings about discrimination lost through JumpReLU and the input directions that support discrimination before it.

Reviewer_sdXWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper compares sparse autoencoders trained on MLP outpus and transcoders trained to predict MLP outputs from MLP inputs on 'single-feature' discrimination, finding that transcoders achieve a larger single feature discrimination than SAEs.

Critical strengths

None

Critical weaknesses

It is not clear from the abstract, and barely from the paper, what is the scientific question behind this work.

There does not seem to be any question about the underlying model where the SAEs or TCs are trained but only a comparison between the two.

Why one should care about single feature discrimination is not motivated.

Single feature discrimination is not defined clear.

The article claims that TCs have higher AUROC than SAEs but does not provide any statistical significance claim for their results in the main figures. How can we know if a 0.03 increase in AUROC is an actual signal?

I did not understand the encoder direction use so I can't really comment on it.

Questions for authors

What is the actual scientific question? Why should we care if TC have better single feature discrimination?

What is the point of looking at pre activation AUC?

What is the motivation of activation restoration?

What is encoder direction use?

What exactly are the features you are trying to discriminate?

1 comment
AuthorsOpenReview ↗

Response to Reviewer sdXW

Thank you for your careful reading. We revised the abstract and introduction to clarify our scientific question and why we study individual features.

W1–W3/Q1. What is the scientific question, and why does single-feature discrimination matter?

We ask how well an individual SAE or TC feature distinguishes a given MLP output pattern and how the encoder and JumpReLU shape that feature’s response. Our question concerns the interpretability tools themselves, rather than how the underlying MLP computes the pattern. We focus on individual features because researchers interpret them one at a time. A researcher might inspect the tokens that strongly activate a feature and conclude that it responds to a particular pattern. If that interpretation is right, the feature should also give higher scores to tokens with the pattern than to those without it in new documents. This is the prediction we test.

Q2. Why measure AUROC before JumpReLU?

A feature may give a target token a higher score than a nontarget token. If neither score exceeds the threshold, JumpReLU maps both to zero. The final values alone cannot tell us whether the encoder distinguished those tokens in the first place. We therefore compared AUROC before and after JumpReLU for the same selected features on the same held-out tokens.

Across all four model sizes and both settings, the selected SAE features had higher mean AUROC before JumpReLU but lost more through JumpReLU. In 270M, 1B, and 4B, that extra loss outweighed SAE’s initial advantage. In 12B, it did not (Table 1). Where TC led after JumpReLU, its features preserved more discrimination rather than starting with better discrimination.

Q3. Why restore the suppressed activations?

The pre-to-post comparison measures the total loss, but JumpReLU maps both nonpositive scores and small positive scores to zero. To test whether suppressing positive scores accounted for TC’s advantage, we restored only those positive scores, leaving nonpositive scores at zero. If suppressing positive scores explained most of the gap, restoring them should nearly close it.

In 1B under small, it did. The TC–SAE gap fell from 0.02505 to 0.00170. Under big, it fell from 0.02951 to 0.02125. Most of the gap under big was already present before any positive scores were suppressed because mapping distinct nonpositive scores to zero cost SAE more discrimination than TC (Appendix B.4).

W6/Q4. What do we mean by encoder direction use?

An encoder uses an input direction for discrimination when differences between tokens along that direction help the feature distinguish target tokens from nontarget tokens. We tested this by removing the contribution of each coder’s own first input principal component (PC1) and comparing the effect with random edits matched for rotation angle and final norm. For TC features, removing PC1 of the MLP input reduced preactivation AUROC more than the matched random edits. For SAE features, removing PC1 of the MLP output reduced it less. This contrast held in the layer averages for every model and setting (Results 4.3, Figure 4).

W4/Q5. What exactly is being discriminated?

We distinguish tokens, not features. At each layer, we group tokens into 32 clusters by the direction of their MLP output vectors. For a given cluster, tokens in that cluster are the targets and tokens in the other 31 are the nontargets. On selection documents, we choose the single feature from each coder with the highest postactivation AUROC for that cluster. On separate documents, AUROC measures how often that feature ranks a target token above a nontarget token, counting ties as half.

W5. How do we know the AUROC gap of about 0.03 is a signal?

Table 7 reports paired 95% confidence intervals for the TC–SAE difference itself. In 1B, the mean postactivation AUROC gap across 26 layers was 0.02505 [0.02478, 0.02533] under small and 0.02951 [0.02924, 0.02977] under big. We computed these intervals by leaving out the same held-out document from both coders and all layers in each jackknife replicate, keeping checkpoints, features, and clusters fixed. Both intervals are above zero, so the estimated TC advantage remains positive after accounting for variation across evaluation documents.

Reviewer_wS7kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper compares SAEs and transcoders based on how well individual features distinguish between different groups of MLP outputs. The authors use GemmaScope-2 checkpoints across all 26 layers of Gemma-3-1B, with two sparsity settings. The main result is that transcoders have higher post-activation AUROC in 24 out of 26 layers, even though SAEs have higher average pre-activation AUROC. The authors investigate why this happens by examining how much information gets lost through ReLU and JumpReLU. They also modify encoder weights to study whether certain input directions contribute to this difference. The results suggest that transcoders lose less discrimination through thresholding and rely more on certain input directions. However, the transcoder advantage reverses when activation rates are matched. The authors also evaluate the features on Few-NERD and find a small improvement in entity detection and sentence retrieval for transcoders.

Critical strengths
  1. I liked the comparison between pre-activation and post-activation AUROC. Interesting to see that you can use the same MLP output clusters for both SAEs and transcoders and select features on separate data from evaluation. The result that SAEs do better before thresholding but worse afterward is interesting. The ReLU and JumpReLU analysis also helps understand where the difference comes from.

  2. I also liked that you tried to present an analysis of why transcoders perform better, rather than just showing that they do. The encoder direction experiments are interesting, especially the comparison against random rotations. I also appreciate the inclusion of some useful controls, like changing clustering seeds and matching activation rates.

Critical weaknesses
  1. I am not sure how much the output cluster results tell us about actual interpretability. Most of the analysis uses clusters formed from MLP outputs. I understand why you do this, since it gives a common target for SAEs and TCs. But these clusters are not necessarily meaningful concepts or features that we would want to interpret. You do have the Few-NERD experiments, which help. But the improvement is fairly small, especially for sentence retrieval. I would have liked to see a stronger connection between the output cluster results and the performance on actual interpretable tasks. I don't think this invalidates the main result, but it does limit what we can conclude from it.

  2. The explanation from encoder directions is still incomplete. I found the encoder intervention experiments interesting, but they explain only part of the difference. In Appendix B.7, the interventions reduce the TC advantage quite a bit in early and middle layers, but a large part of the advantage remains in later layers. I understand that you are not claiming to completely explain the difference. Still, I think some discussion on why the results change so much across layers would help.

  3. Also, all experiments use Gemma-3-1B. I would have liked to see whether at least the main observations hold on another model. This is more of a generalization concern than an issue with the existing experiments.

Questions for authors

Q1. The main evaluation uses MLP output clusters. Do you think the discrimination advantage you observe would also hold for more semantically meaningful features? The Few-NERD results seem to support this to some extent, but the improvement is fairly small.

Q2. In Appendix B.7, the encoder interventions explain much more of the TC advantage in early and middle layers than in later layers. Do you have any intuition for why this happens? Is it mainly because the intervention angles are smaller, or do you think TC features rely on different directions in later layers?

Q3. Appendix B.3 shows that SAEs perform better when activation rates are matched over much of the tested range. Does this suggest that the main TC advantage is mainly due to the learned operating points? Clarifying this would help us understand what the results say about the representations themselves.

1 comment

Thank you for distinguishing the value of a shared output-cluster task from the question of whether the same findings hold for human-defined categories. Your questions prompted us to test that distinction directly, examin the role of intervention size across layers, and extend the comparison beyond 1B model.

W1/Q1, Does the result extend to semantically meaningful targets?

Yes, in Gemma 3 1B, TC features had higher mean AUROC than SAE's after activation on both the output clusters and noun categories derived from SemCor's human annotators. The difference we observed is that on Semcor, TC features already had higher mean AUROC before JumpReLU (see Table 3).

W2/Q2. Why did the original intervention have a smaller effect in later layers?

In the original experiment, we reduced the contribution of the top 16 input PCs to a TC feature only if it exceeded a reference level derived from SAE features. In later layers, many features were already at or below the reference level, so we left them unchanged. Even among features above the reference level in later layers, smaller rotations were needed to bring their contribution down to it.

In the follow-up experiment, we rotated every selected TC feature’s encoder weight vector by 5° to reduce the contribution of those PCs, including vectors we had previously left unchanged. Relative to random rotations of the same angle, these targeted rotations reduced postactivation AUROC more in later layers than in early layers under both settings (Appendix B.7). This supports the explanation that the original intervention had a smaller effect in later layers because it changed fewer features and rotated them less, rather than because later-layer features were less sensitive to those directions.

Q3. Does the TC advantage mainly reflect the learned operating points?

In Gemma 3 1B, TC features had higher mean postactivation AUROC with their learned thresholds, but SAE features did better when we matched activation rates. We kept the selected features fixed and changed only their thresholds so that SAE and TC features were active on the same fraction of tokens. SAE had higher mean AUROC at every rate we tested from 0.1% to 10%, using only the comparisons that could be matched at each rate (Appendix B.3).

To see how much of the original gap came from suppressing small positive scores, we restored those scores but left nonpositive scores at zero. Under small, this nearly closed the TC–SAE gap, from 0.02505 to 0.00170. Under big, the gap narrowed from 0.02951 to 0.02125. Most of the gap was already present before any positive scores were suppressed because mapping distinct nonpositive scores to zero created ties that cost SAE more discrimination than TC (Results 4.1).

W3. Do the 1B findings hold at other model sizes?

Yes, SAE features lost more discrimination than TC features through JumpReLU at all four model sizes, but TC did not always show better discrimination. We extended the 1B analysis to all layers of Gemma 3 270M, 4B, and 12B. In both the small and big settings of every model, the selected SAE features had higher mean AUROC before JumpReLU but also lost more AUROC through JumpReLU. In 270M, 1B, and 4B, that extra loss outweighed SAE’s initial advantage, leaving TC ahead. In 12B, SAE’s initial advantage was large enough to persist (see Table 1).

The PC1 result also held across all four sizes. For each coder, we removed the contribution of the first principal component (PC1) of its own input and compared the effect with random edits matched for rotation angle and final norm. Averaged over layers, removing PC1 reduced preactivation AUROC more than the matched random edits for TC features, but less for SAE features, in both settings of every model (see Figure 4). This contrast persisted even in 12B, where SAE had higher postactivation AUROC. These results extend the comparison across model sizes within the Gemma 3 family.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.