acceptodds
Under review as a conference paper at ICLR 2027

CrossLayerTopK: Dynamically Allocating Sparsity Across Layers in Transcoders

Abstract

Cross-Layer Transcoders (CLTs) decompose the activations of large language models into human-interpretable features and trace how these features interact across layers to form circuits, making them a key tool for mechanistic interpretability. However, training CLTs effectively and efficiently is hard. Beyond the usual trade-off between sparsity and reconstruction quality, a CLT has to model all layers at once, which adds a further challenge. The common JumpReLU recipe controls sparsity only indirectly through a penalty, while the direct extension of TopK selects a fixed number of latents in every layer. Yet reconstruction difficulty differs widely across layers, and intermediate layers are much harder than others, so giving every layer the same budget is suboptimal. We propose \CLTK, which sets a single budget for the whole model and dynamically allocates it across layers during training. In a comparison with JumpReLU and other TopK variants on Qwen3-8B-Base, this gives lower FVU while using fewer active latents than JumpReLU. The better reconstruction carries over to circuit tracing, yielding more complete circuits with higher replacement and completeness scores. The fixed per-token budget of \CLTK also makes it easy to pair with sparse decoding, which cuts both computation and communication and gives a speedup per training step. We further show that \CLTK follows scaling laws in both latent size and sparsity.

Primary area:
interpretability and explainable AI
Keywords:
LLM, interpretability, Cross Layer Transcoders, Transcoders

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vYydOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cross-Layer TopK for training cross-layer transcoders. Instead of selecting a fixed number of features within each layer, it selects features jointly across all layers under a fixed per-token budget. This allows the model to allocate more active features to layers that are harder to reconstruct. Experiments on Qwen3-8B-Base show improved reconstruction and higher circuit replacement and completeness scores compared with JumpReLU and other TopK variants. The paper also combines the method with sparse decoding and a different tensor-parallel layout, reporting a 2.9× reduction in training step time, and studies reconstruction scaling with dictionary size and sparsity.

Critical strengths
  1. The idea of sharing a fixed sparsity budget across layers is simple and well motivated. It combines precise control over the total number of active features with flexibility in where those features are allocated, addressing a practical limitation of per-layer TopK.

  2. The improvement extends beyond reconstruction to circuit-level metrics. At the same budget of 360 active features, Cross-Layer TopK improves the replacement score from 0.577 to 0.617 over Per-Layer TopK.

Critical weaknesses
  1. The conceptual contribution appears relatively incremental. Dynamic budget allocation already exists in BatchTopK, and the paper transfers this idea to the layer axis. The conclusion also acknowledges that TopK variants for CLTs have appeared in earlier implementations, but does not clearly identify what was already available. The most direct comparison isolating cross-layer allocation improves FVU from 27.64% to 27.07%. This is useful, but I am less convinced that it represents a substantial advance beyond a better allocation rule. The 2.9× speedup mainly supports the accompanying implementation improvements rather than the novelty of cross-layer selection itself.

  2. The impact on interpretability is less clear than the impact on reconstruction. Replacement and completeness scores measure how much attribution is carried by features rather than error nodes, but do not directly establish that the features are more understandable or that the resulting circuits better reflect the original model’s causal computation. Since improved circuit interpretation is a central motivation, I would like a clearer account of what these results establish beyond a more accurate sparse approximation. This distinction is important to my overall assessment.

Questions for authors

How should we interpret the learned layer allocation: does it mainly reflect reconstruction difficulty, or the fact that earlier-layer features can contribute to more downstream layers? In particular, how does the decoder-norm-based scoring affect this allocation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer vYyd

We appreciate that the reviewer recognized our strengths that: (1) Our idea of sharing a fixed sparsity budget across layer is simple and well motivated. (b) The improvement extends beyond reconstruction to circuit-level metrics. We have summarized the main contributions and core work of our paper in the global response, and we kindly refer you to the global response for further elaboration. We address the reviewer's concerns here:

W.1. The conceptual contribution appears relatively incremental, and the 2.9× speedup mainly supports the accompanying implementation improvements rather than the novelty of cross-layer selection itself.

Thank you for your comment. We would like to point out that although our improvement is structurally simple, we believe that it addresses an important problem. The intermediate layers of the model encode more abstract and rich semantics, so better reconstruction and interpretation of these layers carry greater significance. We provide a more detailed discussion on the importance of reconstructing intermediate layers in our global response. We kindly refer you to our global response for more details. Existing CLT architectures typically achieve relatively good reconstruction on shallow and deep layers yet perform poorly on intermediate layers. The motivation of this work is that, under a fixed sparsity budget, the model can adaptively allocate more budget to the harder-to-reconstruct intermediate layers to achieve improved reconstruction performance for them.

We agree that the overall gains over Per-Layer TopK are small. However, the main advantage of Cross-Layer TopK is that it gained better performance on the intermediate layers which are the hardest to reconstruct. For example, on all layers whose FVU is above 40% on Per-Layer TopK, i.e. Layer 15~19, the differences of FVU between Per-Layer and Corss-Layer TopK are shown below:

LayerPer-Layer TopK(%)Cross-Layer TopK(%)Delta
1540.4439.131.31
1642.0440.431.61
1740.8939.641.25
1841.8140.331.48
1940.8339.471.36

The average decrease of these layers is 1.402%. The overall FVU is averaged across all layers so the improvement looks slightly. However, The FVUs of the remaining layers are already relatively small compared with these layers (24 out of 36 layers are below 30%). Further improving the reconstruction of these layers yields marginal gains, which are out of scope of our work.

W.2.The impact on interpretability is less clear than the impact on reconstruction.

Thank you for your comment! It is a good idea to evaluate the feature interpretability directly, such as using auto interp score.

We sampled 100 features on each intermediate layer, and used GPT-5.5 to evaluate the auto-interp score. The results are shown below.

Methodscore95% CI
Cross-Layer TopK0.8129[0.8028, 0.8229]
Per-Layer TopK0.8083[0.8982, 0.8184]
Cross-Layer Batch TopK0.7940[0.7837, 0.8044]
JumpReLU0.7911[0.7806, 0.8016]

The result shows that the features interpreted from Cross-Layer TopK gained the highest auto-interp score, meaning a better interpretability.

AuthorsOpenReview ↗

Response to Reviewer vYyd #2

Q.1. How should we interpret the learned layer allocation: does it mainly reflect reconstruction difficulty, or the fact that earlier-layer features can contribute to more downstream layers?

This is a good question! We believe that such allocation primarily reflects reconstruction difficulty. As shown in Table 2, the intermediate layers (8–24), which are the most challenging layers, receive a higher quota under Cross-Layer TopK than under Per-Layer TopK, accompanied by a clear reduction in FVU. We do not attribute this allocation to the fact that shallow layers serve more downstream layers. This is due to the fact that layers 1–5 in fact feed into the largest number of downstream layers, yet they are assigned a budget below k.

Q.2. In particular, how does the decoder-norm-based scoring affect this allocation?

This is a good question! Decoder-norm-based scoring when selecting activations follows common practice, e.g., https://transformer-circuits.pub/2025/january-update/index.html#DL. Multiplying the activation by the decoder norm yields the actual length of the feature decomposed within the model’s hidden space, consistent with the definition of features introduced in https://www.anthropic.com/research/toy-models-of-superposition.

This step is uncommon for SAEs because the decoder is renormalized to unit norm after training, with the encoder scaled by the original decoder norm (Anthropic, https://transformer-circuits.pub/2025/january-update/index.html). Alternatively, the decoder may be normalized at every training step (OpenAI, Appendix C.4, https://arxiv.org/abs/2406.04093). In these cases, the decoder norm equals one at inference time and can therefore be ignored.

By contrast, CLTs cannot renormalize decoders after training. Since each encoder feeds into multiple decoders, there exists no unique scaling factor. For example, suppose a feature from layer 1 writes to the current layer with a decoder norm of 5 and writes to the next layer with a decoder norm of 10. For a single-layer transcoder, we can scale the corresponding encoder entry by 5 and renormalize the decoder to 1 while preserving reconstruction. For a CLT, however, after scaling the encoder by 5, the decoder to the next layer still needs to retain a norm of 2 so that the vector written to the next layer remains unchanged. It is therefore impossible to normalize all decoders to unit norm, and we must rely on decoder norms to score activations.

Thank you for the response. I agree that Cross-Layer TopK improves reconstruction in several layers, particularly the hardest-to-reconstruct middle layers. Given the incremental nature of the methodological contribution, I find the overall FVU improvement still too modest to change my assessment. I therefore maintain my original rating of 2.

Reviewer_96iDOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose an alternative sparsity activation function "Crosslayer TopK" for training Crosslayer Transcoders (CLT). They show that using this loss function improves the pareto frontier of the number of active features and reconstruction loss. The main weakness is that it's a rather small, incremental contribution.

Critical strengths

The paper is written in a straight forward and well-comprehensible way. The experiments and metrics presented are well-suited to benchmark and compare the different models.

Critical weaknesses

The main weakness is that it's a rather small, incremental contribution. The activation function is simple and I and others have previously used this and similar loss functions for PLT and CLT training. In some sense, the proposed activation function is the same as Gao et al's TopK applied over all CLT features. The improvement is also tiny and it is unclear to me whether JumpReLU CLTs are actually worse or whether training wasn't well-optimized for some reason

See Questions.

Questions for authors
  • Can you compare the number of active latents for Per-Layer TopK and JumpRelu with existing publicly available CLTs? Afaik for some (most) CLTs, L0 is highest in early layers and sparse in late layers. Why are the authors models different?
    • An FVU of ~40% in some layers seems exceptionally bad for CLTs. Can the authors compare their FVU with publicly available CLTs for similar and other models? What explains the bad FVU? How can be ruled out that training failure or undertraining was causing this?
    • If CLTs behave differently on other models, it would be great if the authors can train them on other models too, like gpt2-small and gemma/qwen 4B.
    • Why do JumpReLU CLTs have so many dead latents? Was the auxiliary loss correctly applied? What explains the high number of dead latents despite the auxiliary loss which worked well in the past?
    • Regarding this: Is the worse performance of the JumpReLU CLTs only due to the higher number of dead features? If you correct for the number of alive features, would they still perform worse?
    • 360 active features per layer per tokens seems really high, what's the motivation for this? Or is this for all layers in total? In Table 1, L0 is cited as 360 but in Figure 2 it's around 10. Can you clarify?
    • What is meant by "BatchTopK degrades at inference"? The numbers don't look "degraded". Can the authors pick a threshold such that L0 ends up at roughly 360?
    • Since the differences are so small, could the authors add Cis to Table 2?
    • Were activations of different layers normalized by different factors, i.e. to correct for different layers having different norms?
    • Possibly related: The author's motivation letting the CLT freely allocate sparsity across layers is such that "harder to reconstruct" layers or tokens can use more features. Can you provide evidence for this? Figure 2 shows that FVU is roughly mirrored and layers with high FVU don't generally have the highest L0. Why was Crosslayer BatchTopK not investigated further? From table 1, it seems better and the same motivation for doing topk across layers should apply here.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 96iD #1

We appreciate that the reviewer recognized our strengths of our presentation and well-suited metrics. We have summarized the main contributions and core work of our paper in the global response, and we kindly refer you to the global response for further elaboration. We address the reviewer's concerns here:

Q.1 Can you compare the number of active latents for Per-Layer TopK and JumpRelu with existing publicly available CLTs? Afaik for some (most) CLTs, L0 is highest in early layers and sparse in late layers. Why are the authors models different?

Thank you for your question! Since the recovered models differ, we cannot compare our current model with other CLTs. With different recovered models, the sparsity allocation of CLTs may also vary. We have not found discussions related to cross-layer sparsity allocation in well-known CLT studies (such as Anthropic’s circuit tracing and DeepMind’s Gemma Scope2). We only identified one relevant paper by Lange et al.(2026), which reports results solely on GPT2-small, an earlier and less capable model. Compared with GPT2-small, Qwen3-8B-Base adopts more advanced training techniques, has a larger parameter count and more layers. Since CLTs can reflect model properties to some extent, it is explainable that CLTs trained on different models exhibit distinct behaviors. We will open-source our trained Cross-Layer TopK CLT for technical exchange and to verify the correctness of this work.

Existing work (https://transformer-circuits.pub/2024/scaling-monosemanticity) argues that middle layers encode richer semantic meanings and are therefore harder to recover; allocating higher sparsity reduces recovery difficulty, so it is reasonable for CLTs to have larger sparsity on middle layers. By contrast, shallow features are relatively easier to recover and thus do not require a large latent quota.

We welcome further discussion on this topic if the reviewer points out additional relevant literature.

Q.2. An FVU of ~40% in some layers seems exceptionally bad for CLTs. Can the authors compare their FVU with publicly available CLTs for similar and other models? What explains the bad FVU? How can be ruled out that training failure or undertraining was causing this?

Thank you for your question. We observe that CLT recovery becomes more difficult as the target model being recovered grows larger. Most CLT implementations in the open-source community are built on toy models, and we are unaware of any open CLT work with a model size and architecture comparable to Qwen3-8B. We compare our results with Anthropic’s work at https://transformer-circuits.pub/2025/attribution-graphs/methods.html, even though that work does not disclose the exact size of Claude 3.5 Haiku. Figure 24 in their paper reports an average FVU across all layers of approximately 34% at a dictionary width of 1M (similar to our setting), which is comparable to the performance of our trained JumpReLU CLT. Given that shallow and deep layers are relatively easy to recover under this dictionary width, we conjecture that the FVU values for middle layers in their work are also close to ours.

SAEs trained on models of the same size generally achieve substantially lower FVU, since conventionally trained SAEs have a larger dictionary width per layer. For the CLT, although the total dictionary width across all layers is also on the order of millions, the dictionary is evenly partitioned among all layers, leaving each layer with an actual dictionary width of only 32768. We conjecture that this explains the higher FVU of the CLT at intermediate layers.

AuthorsOpenReview ↗

Response to Reviewer 96iD #2

Q.3. If CLTs behave differently on other models, it would be great if the authors can train them on other models too, like gpt2-small and gemma/qwen 4B.

Thank you for your suggestion! Training CLTs on more models would indeed boost the confidence of our findings. However, training a CLT requires constructing cross-layer mappings across multiple Transformer layers. Compared with conventional single-layer SAEs and Transcoders, its training involves a larger activation space, a more complex computation graph, and higher computational and memory overhead. Our CLT fully reconstructs activations from all 36 layers of the Qwen3-8B-Base model, making it the largest CLT in open-source research (we will release our trained CLT weights to the community), with a total of 95B parameters. Constrained by our available compute and the prohibitive training cost (a single CLT training run requires over 2e21 FLOPs), we cannot conduct repeated training across additional models.

We believe that under the same compute budget, thorough and rigorous experiments on a larger model may carry greater scientific value than shallow, underpowered experiments spanning multiple models. Larger, state-of-the-art models possess stronger capabilities and can better advance the practical deployment of interpretability research. This motivates our choice of Qwen3-8B, the largest model feasible within our compute budget.

Q.4. Why do JumpReLU CLTs have so many dead latents? Was the auxiliary loss correctly applied? What explains the high number of dead latents despite the auxiliary loss which worked well in the past?

Thank you for your question! We have tuned the relevant hyperparameters of JumpReLU. Due to the prohibitive cost for full training, we monitored the dead latents trends at about 20K micro steps and terminated the runs with undesirable dead latent fractions early according to the trends. Some records of our tuning are shown below:

Final lambda after warmupWarmup stepsResult at around 20K steps
1010K76.3% dead latent
5100K41.2% dead latent
2100Kfails to converge near the target L0 at the end of training.

Regarding the auxiliary loss, our setup strictly follows Appendix D in https://transformer-circuits.pub/2025/attribution-graphs/methods.html#appendix-ml-details. We also swept over the coefficient of the auxiliary loss, yet we found that the fraction of dead latents is not sensitive to this coefficient.

According to the results above, we chose final settings of training JumpReLU. The table above also shows that the performance of JumpReLU CLT is heavily dependent on relevant hyperparameter configurations, leading to a costly tuning process. CLTs equipped with Cross-Layer TopK require no hyperparameter tuning (all our associated hyperparameters follow the common practice established by prior related work), which substantially reduces the compute demand for experiments. This constitutes one of the notable advantages of our method over JumpReLU.

With respect to dead latents observed in JumpReLU, we view this as a limitation of JumpReLU CLTs at this scale instead of a training failure. Prior SAE work (https://transformer-circuits.pub/2024/scaling-monosemanticity) similarly notes substantial dead latent fractions for large dictionary widths, e.g., 35% dead latents for 4M SAEs.

Q.5. Is the worse performance of the JumpReLU CLTs only due to the higher number of dead features? If you correct for the number of alive features, would they still perform worse?

Thank you for your question! We agree that the subpar performance of JumpReLU partly stems from its high fraction of dead latents. However, as discussed in our previous response, we believe this is an inherent limitation of JumpReLU when applied to large-scale models, rather than a result of incorrect hyperparameter settings on our end. Should future work develop techniques to mitigate this dead-latent-related weakness of JumpReLU, we think JumpReLU has the potential to match or even outperform the approach proposed in this paper.

Q.6. 360 active features per layer per tokens seems really high, what's the motivation for this? Or is this for all layers in total? In Table 1, L0 is cited as 360 but in Figure 2 it's around 10. Can you clarify?

Thank you for your question! We clarify that our setup activates 360 latents in total across all layers. Specifically, Per-Layer TopK activates exactly 10 latents per layer. We kindly refer you to lines 239–241 of our paper:

The three TopK architectures share identical hyperparameters and activate K = 360 latents per token. Per-Layer TopK activates k latents in every layer, with L · k = K so that the total budget matches Cross-Layer TopK.

AuthorsOpenReview ↗

Response to Reviewer 96iD #3

Q.7. What is meant by "BatchTopK degrades at inference"? The numbers don't look "degraded". Can the authors pick a threshold such that L0 ends up at roughly 360?

By "degrades", we refer to the scenario where the inference-time L0 is lower than the training-time L0, alongside an increase in FVU from 26.98% during training to 29.04% during inference, which indicates performance degradation. During CLT training, activations from the full pretraining dataset are shuffled, and a batch is sampled for training. The number of latents activated for one group of activations depends on other activations within the same batch. However, when performing inference with the CLT, the inputs are activations from a single token or a single prompt, rather than a batch matching the training distribution. As a result, we cannot apply the same procedure used during training to determine whether each latent activates. The method from the original work estimates a threshold for each latent based on training data. Therefore, the performance of Cross-Layer Batch TopK should be evaluated under the inference setting, rather than using training-time performance metrics. This constitutes an inherent train-inference discrepancy, and the threshold estimation scheme described in the original paper is biased, leading to degraded performance.

Our threshold estimation method follows the original Batch TopK paper (https://arxiv.org/pdf/2412.06410). This estimator computes thresholds based on the minimum activation value of each feature across a batch and does not incorporate the target L0 at all.

Refining this estimator to align inference-time L0 with the target value is an interesting direction. Nevertheless, it poses a challenging combinatorial optimization problem and falls outside the scope of this work.

Q.8. Since the differences are so small, could the authors add Cis to Table 2?

We agree that the overall gains over Per-Layer TopK are small. However, the main advantage of Cross-Layer TopK is that it gained better performance on the intermediate layers which are the hardest to reconstruct. For example, on all layers whose FVU is above 40% on Per-Layer TopK, i.e. Layer 15~19, the differences of FVU between Per-Layer and Corss-Layer TopK are shown below:

LayerPer-Layer TopK(%)Cross-Layer TopK(%)Delta
1540.4439.131.31
1642.0440.431.61
1740.8939.641.25
1841.8140.331.48
1940.8339.471.36

The average decrease of these layers is 1.402%. The overall FVU is averaged across all layers so the improvement looks slightly. However, The FVUs of the remaining layers are already relatively small compared with these layers (24 out of 36 layers are below 30%). Further improving the reconstruction of these layers yields marginal gains, which are not the target of our work. We agree that training across multiple seeds would strengthen the confidence in our conclusions. However, training a CLT requires constructing cross-layer mappings across multiple Transformer layers. Compared with conventional single-layer SAEs and Transcoders, its training involves a larger activation space, a more complex computation graph, and higher computational and memory overhead. Our CLT fully reconstructs activations from all 36 layers of the Qwen3-8B-Base model, making it the largest CLT in open-source research (we will release our trained CLT weights to the community), which has 95B parameters in total. Constrained by our available compute and the prohibitive training cost (a single CLT training run requires over 2e21 FLOPs), we are unable to perform repeated training across multiple seeds. Furthermore, the primary advantage of Cross-Layer TopK, namely its performance gains on reconstructing the most challenging intermediate layers, is already substantial when compared with Per-Layer TopK.

Reviewer_PR2WOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to improve Cross-Layer Transcoders by using a Cross-Layer TopK activation function, instead of a JumpRelu. This give the opportunity to control sparsity more directly, as well as allowing the active feature budget to be spend in reconstructing the hardest layers. Experiments were run on Qwen3-8B-Base, with favorable comparisons between other baselines. The study also includes evidence of improved circuit tracing, and details the efficiency gains of the method.

Critical strengths

The paper presents a simple, practical sparsity mechanism with exact L0 control. Cross-Layer TopK fixes the per-token budget by construction, so it needs no penalty-coefficient search and no post-hoc thresholds, and it makes sparse decoding straightforward. Training cost and sparsity tuning are major practical barriers to building large CLTs, and a drop-in mechanism that removes both is welcome.

Critical weaknesses

The central claims rest on weak baselines and small gains on a narrow range of settings. JumpReLU appears undertrained (41% dead latents!), and BatchTopK runs at a lower L0, while the margin over Per-Layer TopK (0.57 pt FVU) comes from single runs without confidence intervals. The evaluation is also narrow: there is one model, one dictionary size and one budget, no downstream fidelity metric, and only reconstruction-derived circuit scores, so the paper shows a modest reconstruction gain in one setting rather than a demonstrated interpretability improvement.

Questions for authors
  1. Comparisons are made at a single model / dictionary width / sparsity level. There is no evidence that these results will hold for other hyperparameters choices. Need to include sweeps over these important hyperparameters 2. The gains over Per-Layer TopK are small. Combined with the fact that these results come from single runs without seeds or confidence intervals, makes the comparison problematic as the significance of these results cannot be assessed. Please report results from multiple seeds. 3. JumpReLU doesn't look properly trained. There are 41% dead latents, the target L0 is not reached, and there doesn't seem to be hyperparameter tuning for lambda). Please report a curve over several lambda values and compare against Cross-Layer TopK at a matched L0. 4. A fairer comparison with Cross-Layer BatchTopK would be to evaluate cross-layer TopK at an inference L0 of 289 rather than 360. 5. The paper reports no downstream fidelity metric, such as the increase in cross-entropy loss or KL divergence when the CLT is introduced into the model, which is a standard evaluation practice and would show whether the FVU gains matter for model behavior. 6. The replacement and completeness scores are largely a function of reconstruction quality, so the paper should also evaluate feature interpretability directly, e.g. with auto-interp scores, pruned graph size or qualitative circuit examples. 7. The sparse-decoding speedup depends only on a fixed number of active latents per token, which Per-Layer TopK also guarantees, so it is an advantage over JumpReLU but not of cross-layer selection specifically. 8. Since the last layers receive about one latent each and are reconstructed mainly by early-layer features, the authors should discuss whether this worsens the unfaithful-circuit incentive described by Lange et al. (2026), which they cite
4 comments
AuthorsOpenReview ↗

Response to Reviewer PR2W #1

We appreciate that the reviewer recognized our strengths that: (1) We present a simple and practical sparsity mechanism with exact L0 control. (b) Our method needs no penalty-coefficient search and no post-hoc thresholds, and makes sparse decoding straightforward. (c) We decreased the training cost and sparsity tuning, which are the major practical barriers to building large CLTs. We have summarized the main contributions and core work of our paper in the global response, and we kindly refer you to the global response for further elaboration. We address the reviewer's concerns here:

Q.1. Comparisons are made at a single model / dictionary width / sparsity level. There is no evidence that these results will hold for other hyperparameter choices. Need to include sweeps over these important hyperparameters.

Thank you for your question! We agree that hyperparameter choices are critical to our conclusions. We have tuned the key hyperparameters for each architecture, such as the L0 coefficient, warmup schedule and auxiliary loss coefficients for JumpReLU (please refer to our response to Q.3), as well as hyperparameters including the bandwidth for STE.

However, training a CLT requires constructing cross-layer mappings across multiple Transformer layers. Compared with conventional single-layer SAEs and Transcoders, its training involves a larger activation space, a more complex computation graph, and higher computational and memory overhead. Our CLT fully reconstructs activations from all 36 layers of the Qwen3-8B-Base model, making it the largest CLT in open-source research (we will release our trained CLT weights to the community), with a total of 95B parameters. Constrained by our available compute and the prohibitive training cost (a single CLT training run requires over 2e21 FLOPs), we are unable to perform exhaustive ablations over the full hyperparameter space due to this extreme training expense. To conserve compute resources, we terminate runs that are unlikely to yield favorable results early in training based on observed trends. Owing to this early-stopping practice, we do not report exhaustive ablation studies.

Most of our hyperparameter selections follow common practice, primarily drawing from the publicly disclosed training details from Anthropic, including CLT initialization and STE-related hyperparameters. We also tuned improved coefficients based on their published L0 lambda scheduling strategy. Additionally, we optimized the learning rate schedule to further reduce the loss.

We acknowledge that extensive comparisons across multiple base models, dictionary widths and sparsity levels would strengthen confidence in our results. Nevertheless, the prohibitive training cost prevents us from carrying out such comprehensive comparisons. We believe that training on the largest feasible model (the 8B model) under a fixed compute budget is scientifically more meaningful than conducting experiments on smaller models. Larger models exhibit richer capabilities and are more representative of state-of-the-art models deployed today, and interpretability research on these larger models carries greater practical value.

Q.2. The gains over Per-Layer TopK are small. Combined with the fact that these results come from single runs without seeds or confidence intervals, makes the comparison problematic as the significance of these results cannot be assessed. Please report results from multiple seeds.

We agree that the overall gains over Per-Layer TopK are small. However, the main advantage of Cross-Layer TopK is that it gained better performance on the intermediate layers which are the hardest to reconstruct. For example, on all layers whose FVU is above 40% on Per-Layer TopK, i.e. Layer 15~19, the differences of FVU between Per-Layer and Corss-Layer TopK are shown below:

LayerPer-Layer TopK(%)Cross-Layer TopK(%)Delta
1540.4439.131.31
1642.0440.431.61
1740.8939.641.25
1841.8140.331.48
1940.8339.471.36

The average decrease of these layers is 1.402%. The overall FVU is averaged across all layers so the improvement looks slightly. However, The FVUs of the remaining layers are already relatively small compared with these layers (24 out of 36 layers are below 30%). Further improving the reconstruction of these layers yields marginal gains, which are not the target of our work. We agree that training across multiple seeds would strengthen the confidence in our conclusions. However, due to the extreme training expense(as discussed in the response to Q.1.), we are unable to perform repeated training across multiple seeds. Furthermore, the primary advantage of Cross-Layer TopK, namely its performance gains on reconstructing the most challenging intermediate layers, is already substantial when compared with Per-Layer TopK.

AuthorsOpenReview ↗

Response to Reviewer PR2W #2

Q.3. JumpReLU doesn't look properly trained, with 41% dead latents, and the target L0 is not reached.

We have tuned the relevant hyperparameters of JumpReLU. Due to the prohibitive cost for full training, we monitored the dead latents trends at about 20K micro steps and terminated the runs with undesirable dead latent fractions early according to the trends. Some records of our tuning are shown below:

Final L0 lambda after warmupWarmup stepsResult at around 20K steps
1010K76.3% dead latent
5100K41.2% dead latent
2100Kfails to converge near the target L0 at the end of training.

According to the results above, we chose final settings of training JumpReLU. The table above also shows that the performance of JumpReLU CLT is heavily dependent on relevant hyperparameter configurations, leading to a costly tuning process. CLTs equipped with Cross-Layer TopK require no hyperparameter tuning (all our associated hyperparameters follow the common practice established by prior related work), which substantially reduces the compute demand for experiments. This constitutes one of the notable advantages of our method over JumpReLU.

With respect to dead latents observed in JumpReLU, we view this as a limitation of JumpReLU CLTs at this scale instead of a training failure. Prior SAE work (https://transformer-circuits.pub/2024/scaling-monosemanticity) similarly notes substantial dead latent fractions for large dictionary widths, e.g., 35% dead latents for 4M SAEs.

Regarding the issue that JumpReLU fails to hit the target final L0: we adopt the sparsity loss from Gemma Scope 2 (McDougall et al.), which incorporates the squared difference between the current L0 and the target L0 as a loss term. As L0 approaches the target value, this loss rapidly decays to nearly zero and contributes only marginally to the total loss when compared with the much larger-scale MSE loss. Consequently, the optimizer prioritizes optimizing MSE over sparsity. The sparsity loss is exceedingly small near the target, which differs from the MSE loss by two orders of magnitude in this regime, and prevents L0 from converging to the target. This is an inherent limitation of this loss formulation.

Furthermore, there exists a trade-off between L0 and reconstruction performance: a larger L0 implies greater reconstruction potential. Therefore, when comparing against TopK family CLTs, a larger L0 is a favorable setting for JumpReLU and does not degrade the capabilities of the JumpReLU CLT. Even under this advantageous configuration, JumpReLU still fails to outperform TopK-family CLTs.

Q.4. A fairer comparison with Cross-Layer BatchTopK would be to evaluate cross-layer TopK at an inference L0 of 289 rather than 360.

Thank you for your suggestion. However, Cross-Layer Batch TopK cannot maintain its training sparsity during inference, which is an inherent limitation of this approach. Specifically, during training, Cross-Layer Batch TopK enforces a target average number of active features per token by jointly selecting the highest activations across all tokens and all layers within a batch. Because this introduces dependencies across tokens in the batch, inference must replace the batch-level TopK operator with a static global scalar threshold.

Our method avoids this train-inference discrepancy, constituting one of our key strengths. Setting our inference k to 289 would obscure this advantage and yield an unfair comparison.

AuthorsOpenReview ↗

Response to Reviewer PR2W #3

Q.5. The paper reports no downstream fidelity metric, such as the increase in cross-entropy loss or KL divergence when the CLT is introduced into the model.

Thank you for your suggestion. Evaluating the cross entropy obtained by substituting the original Transformer MLP output with the CLT-approximated MLP output is an appropriate metric for assessing reconstruction performance.

We are unaware of prior work that uses replacement cross entropy for CLT evaluation. Following common practices established for SAEs and Transcoders, we replace the output of the target MLP layer in the LLM with the reconstruction produced by the CLT and measure the resulting increase in the LLM’s cross entropy. The main strength of Cross-Layer-TopK lies in improved reconstruction of intermediate layers. We therefore evaluate the delta CE when performing such replacement on intermediate layers (layers 8–24, which exhibit substantially higher FVU values in Figure 2). The results are presented in the table below.

MethodDelta CE(Lower is better)Fraction of Delta CE(Detla CE / original CE, Lower is Better)
Cross-Layer TopK0.01280.63%
Per-Layer TopK0.01360.67%
Cross-Layer Batch TopK0.01430.70%
JumpReLU0.01820.89%

The result shows that Cross-Layer TopK gains the lowest delta CE on the intermediate layers, meaning that Cross-Layer TopK gives a better approximation of MLP layers among all 4 kinds of activation functions. We also evaluated the delta CE when replacing all MLP layer outputs with CLT approximations. The delta CE is 0.005 for all four models, which makes differences indistinguishable.

Q.6. The replacement and completeness scores are largely a function of reconstruction quality, so the paper should also evaluate feature interpretability directly.

Thank you for your suggestion. It is a good idea to evaluate the feature interpretability directly, such as using auto interp score.

We sampled 100 features on each intermediate layer, and used GPT-5.5 to evaluate the auto-interp score. The results are shown below.

Methodscore95% CI
Cross-Layer TopK0.8129[0.8028, 0.8229]
Per-Layer TopK0.8083[0.8982, 0.8184]
Cross-Layer Batch TopK0.7940[0.7837, 0.8044]
JumpReLU0.7911[0.7806, 0.8016]

The result shows that the features interpreted from Cross-Layer TopK gained the highest auto-interp score, meaning a better interpretability.

Q.7. The sparse-decoding speedup depends only on a fixed number of active latents per token, which Per-Layer TopK also guarantees, so it is an advantage over JumpReLU but not of cross-layer selection specifically.

We agree that the sparse decoding speedup is an advantage over JumpReLU. Since an influential CLT work from Anthropic (https://transformer-circuits.pub/2025/attribution-graphs/methods.html) uses JumpReLU, we include sparse decoding in our discussion to compare the training efficiency of Cross-Layer TopK and JumpReLU.

Q.8. The authors should discuss whether this worsens the unfaithful-circuit incentive described by Lange et al. (2026)

We agree that whether CLT can faithfully recover the internal reasoning process of LLMs is a critical and debatable question. However, this is a general concern for CLT, rather than one introduced by our method. In terms of sparsity allocation, Cross-Layer TopK behaves similarly to JumpReLU. At present, there is no significant evidence showing that Cross-Layer TopK induces greater unfaithfulness compared with JumpReLU. There is also no widely accepted method to evaluate or verify that CLT yields unfaithfulness when recovering real language models. Lange et al. (2026) only conducted experiments on an extremely small neural network with manually set parameters.

Thank you for the revisions and clarifications. Although you explained the reasons behind the critical weaknesses, most of them still hold (a few of them were indeed addressed). To be clear, a CLT with 10x more parameters than the model it explains is not inherently an issue. However, training a 95B-parameter CLT introduces major challenges that, as your responses show, you can describe but have not overcome. I will therefore keep my score, though I appreciate the effort put into the rebuttal.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We thank AC and all the all reviewers for their valuable comments and constructive feedbacks! Our work proposes Cross-Layer TopK as an activation function of Cross-Layer Transcoders. This allows CLT to allocate the sparsity budget across all layers. A key limitation of Per-Layer TopK CLT for LLM interpretation lies in challenging reconstruction of intermediate layers, restricting the ability to understand how intermediate-layer features function within circuits. Intermediate-layer features of the model carry more abstract and diverse semantics than the early and deep layers. Faithful interpretation of intermediate layers is essential for model interpretability, yet reconstruction and interpretation of intermediate layers are typically harder than those of other layers. By paying more sparsity budget to these layers, Cross-Layer TopK gained better performance on these layers, with FVU decreased by an average of 1.56% on the most challenging layers. Meanwhile, our method achieves a reduction in FVU across all layers compared with Per-Layer TopK. We also compared our method with JumpReLU, which also supports adaptive sparsity allocation. We find that JumpReLU is sensitive to hyperparameters, while our method is almost tuning-free, which considerably reduces resource consumption during training. Our method also gains better performance on FVU across all layers compared with JumpReLU.

Furthermore, our paper also discusses training efficiency and the estimation problem of BatchTopK. Compared with JumpReLU, which is currently an influential activation choice for CLT in the community((https://transformer-circuits.pub/2025/attribution-graphs/methods.html)), Cross-Layer TopK enables training acceleration via sparse decoding, delivering a 2.9× efficiency improvement over JumpReLU. Cross-Layer Batch TopK is an extension of Batch TopK built upon the cross-layer idea proposed in this work. We show that it fails to maintain consistency between training and inference, which leads to degraded performance at inference time.

Existing CLTs in the open-source community, as well as the models they reconstruct, are relatively small. We will release our Cross-Layer TopK CLT trained in this work, which is trained to reconstruct activations from all 36 layers of Qwen3-8B-Base. This will be the largest CLT in the open-source community to date, with approximately 95B parameters.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.