acceptodds
Under review as a conference paper at ICLR 2027

Beyond Token Scale: Chunk-Level Sparse Autoencoders for Reliable Semantic Feature Discovery

Abstract

Sparse autoencoders (SAEs) expose features that help us understand and steer language models, but faithful reconstruction does not guarantee informative concepts. Token-level objectives reward lexical and formatting details alongside semantic content, all competing for a limited sparse budget. We introduce a family of chunk-level SAEs that encode mean-pooled activations over chunks, each a contiguous span of tokens: Mean-Chunk reconstructs the observed chunk, Cross-Chunk predicts an independently processed neighbor, and Joint-Chunk combines both targets. These designs separate the effect of a larger observation unit from that of predicting information shared across passages. With matched training data, chunk-level SAEs remain powerful interpretability tools while learning reliable semantic features that capture high-level concepts and respond selectively to relevant content. Their strengths are complementary: Mean-Chunk improves high-level feature discovery, reasoning detection beyond surface cues, and steering; Cross-Chunk leads document retrieval and classification transfer while producing selective, persistent features. Changing what an SAE sees and predicts yields reliable semantic features for more meaningful tasks. We demonstrate their practical value through gains across downstream tasks such as retrieval, reasoning detection, and steering.

Primary area:
interpretability and explainable AI
Keywords:
Sparse autoencoders, Interpretability, Chunk-level

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YvRbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed chunk-level SAEs as an alternative to a known problem with token-level SAEs: reconstructing one token at a time pushed features toward surface characteristics instead of meaning. The authors used mean-pooled activations over different sizes of token chunks and trained 3 variants: Mean-Chunk (reconstruct the chunk mean), Cross-Chunk (predict a same-document neighbor’s mean) and Joint-Chunk (both). The authors evaluated fidelity, LLM-judged interpretability, retrieval, reasoning detection, classification and steering.

Critical strengths
  1. The paper addressed a known limitation in a widely used technique.

  2. The experimental design is sound: Mean- vs. Cross-Chunk separated the effect of pooling from the effect of the prediction target. All SAEs shared the same model, data, width and sparsity, which provided a good global control.

  3. The evaluation included many controls: low word overlap and length-matched candidates for retrieval, frozen features and held-out cue-free/cue-only rewrites for reasoning, a matched-pooling audit, and an α sweep.

Critical weaknesses
  1. The figures were really hard to read. Some examples (but there are many more): a) The two oranges in Fig. 1 look exactly the same under the red-shaded hexagon filter. b) The labels in Fig. 6 almost overlap, and the numbers were hard to read and rotated 90 degrees to the side. c) The traces in Fig. 4 are way too small to follow. d) Fig. 7A has no error bars, while its 0-1 scale made the small differences between methods hard to see. As presented, the results are difficult to verify.

  2. Key hyperparameter and experimental design choices were not justified. Some examples (again, there are more): a) The paper is about changing the SAE's observation unit, but the unit itself was never studied. The chunk lengths {32, ..., 512} were justified only as "to cover different context scales" and mixed into one run, with no results by length. I urge the authors to explain why these lengths were chosen and to back the choice empirically. b) The same applies to the comparison: retrieval ranked by cosine, the dense baseline was raw hidden states with no standardization mentioned, and token codes were simply averaged. But the paper’s own Appendix showed that re-pooling moved BatchTopK’s backtracking results by 24-31 points. It was not clear how much of the gap came from the dictionaries rather than from mean-pooling plus cosine.

  3. The evaluation was mostly built around the method. All evaluations are custom, and there are no standard benchmarks (e.g., SAEBench, which the paper cited). The training objective matched the test: Cross-Chunk was trained on same-document neighbors, then evaluated on same-document retrieval and persistence. The reasoning test was built around the baselines: the cue-only negatives kept the token SAEs’ anchor words. Interpretability rested on one LLM judge with no human check and no chance baseline. Random guessing would score roughly 43-50 on InterpScore, and the explainer prompt asked for topic-style descriptions, which favored chunk features.

  4. All experiments used a single model and a single layer.

Questions for authors

Clarifications:

  1. Why were these chunk lengths chosen? How did the results change as a function of length?
  2. What similarity and normalization did the dense baseline use?
  3. How were the explainer and scoring passages chosen, and were the scoring passages held out?
  4. How many training seeds were used per SAE?

Experiments that could change the assessment: 5. Retrieval with standardized hidden states for the dense baseline, and with max-pooled or IDF-weighted codes for the token-level SAEs. 6. A chance-level and random-model InterpScore baseline, plus a small human check of the judge. 7. Standard benchmarks where applicable (relevant SAEBench metrics, AxBench as-is), or an explanation of why these benchmarks cannot be used.

If the gap survived standardized baselines and the chunk-length choice were given a rationale and empirical support, a higher score would be warranted.

8 comments
AuthorsOpenReview ↗

1, Figure Clarity and Controlled Comparisons(W1、W2/Q2/Q5)

We are grateful for the reviewer’s constructive feedback, which helped us improve clarity! We have added targeted experiments that directly address each of your concerns and provide the evidence you requested.

W1: Improve figure readability and uncertainty reporting.

To address each presentation issue, we revised all four figures. Please see the revised manuscript for the complete figures.

FigureRevision
1Distinct colors and shapes, no red overlay, direct values
4Larger traces, readable feature IDs and explanations
6Horizontal labels and values, separated conditions
7AZoomed axes, separate panels, 95% error bars

The revised figures make the comparisons and their uncertainty directly readable. We have made the revisions in the latest version of the paper as per your request.

W2&Q2&Q5: Separate dictionary gains from context length, normalization and pooling.

To test dictionary quality at equal context, we froze all SAEs and reran the language model on exactly 32 real tokens on both sides of 1,736 documents, using identical 16×32 batches. Every query searches all 1,736 candidates, without variable padding or access to later tokens.

To address normalization and aggregation, dense averages are standardized using training-set means and standard deviations. Token/Temporal use mean or max pooling, optionally weighted by inverse document frequency, IDF, to downweight common features. Cross top128 cosine retains the 128 largest activations. R@5 is the fraction of correct paired passages found in the top five.

We additionally apply CL-SR because it tests the same dictionary with an established concept-based retrieval score (Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval, EMNLP 2025).

Exactly 32 real tokensR@5
Standardized dense46.63%
Token/Temporal mean+IDF45.05/45.48%
Token/Temporal max+IDF35.37/35.74%
Cross top128 cosine49.65%
Cross repository/paper CL-SR53.14/53.28%

Cross exceeds standardized dense by 3.02pp with cosine and 6.65pp with paper CL-SR, both passing correction over 168 tests. Independent 1B training repeats average 2.82/6.24pp gains under the cached 32-token protocol.

To check the original mixed-length comparison under stronger baselines, we added training-fitted centering, per-dimension standardization and feature IDF. The original dense baseline used valid-token means and cosine, comparing L2-normalized vectors.

Original candidate setDenseTokenTemporal
Original dense/mean-pooled cosine70.45%63.71%67.05%
Centered dense71.66%——
Standardized dense78.37%——
Mean+IDF—74.60%74.94%
Max—56.65%57.52%
Max+IDF—65.35%65.26%

Cross cosine reaches 77.02%, exceeding Token/Temporal mean+IDF by 2.42/2.07pp. Applying paper/repository CL-SR to the same Cross dictionary gives 81.68%/81.42%, both above standardized dense after correction. Index statistics use no relevance labels.

To isolate target learning from mean pooling, we also compare Cross with Mean under identical pooling. Cross gains 10.71pp in linking R@5. Complete results retain all 35 scoring configurations and distinguish full pooled codes from historically storage-truncated codes.

Cross's learned dictionary improves retrieval under equal text length, a common full candidate set and strengthened dense/token baselines.

AuthorsOpenReview ↗

2, Public Tasks, Generalization and Evaluation Protocols(W3、W4、Q1、Q3)

W3: Validate on public tasks beyond the custom evaluations.

To test sparse semantic access without generated examples or an LLM judge, we use AG News's public topic labels (Character-level Convolutional Networks for Text Classification, NeurIPS 2015). Following SAEBench sparse probing, four balanced topic-versus-rest tasks use 4,000/1,000 official train/test documents. k denotes training-selected coordinates, and entries are mean accuracy.

Representationk=1k=2k=5
Token74.00%76.90%85.15%
Temporal74.10%79.65%84.70%
Mean74.00%80.60%88.20%
Cross82.80%86.10%91.10%
Joint82.70%85.00%89.75%
Standardized dense80.75%84.10%88.15%

At five coordinates, Cross exceeds standardized dense by 2.95pp after correction over 180 tests. To distinguish dictionary learning from pooling or random expansion, we also compare pooling-only Token (84.75%) and same-width random dictionaries (87.78±0.90%). Unrestricted dense classification reaches 93.60%.

To independently test relations in natural text, we use human-annotated DiscoGeM (DiscoGeM: A Crowdsourced Corpus of Genre-Mixed Implicit Discourse Relations, LREC 2022). With frozen dictionaries and document-disjoint splits, five-coordinate classifiers distinguish reason/result relations from other relations. AUROC measures positive–negative ranking, with 0.5 indicating chance.

MethodTokenTemporalMeanCrossJoint
DiscoGeM AUROC,k=50.56050.55850.60920.59420.5858

All three chunk methods exceed both token-level baselines.

Public classification and human-annotated discourse both demonstrate useful semantic information accessible through very few chunk-feature coordinates. We think these evaluations address dependence on custom tasks and support the paper's sparse-access conclusion.

W4: Replicate across model families and layers.

To test whether the findings depend on one model and layer, we trained all five methods for 1B at another Qwen layer and on Gemma-3-12B. We reuse AG News because the same five-coordinate topic task compares semantic accessibility across representations.

Model/blockToken,k=5TemporalMeanCrossJoint
Qwen-9B/2185.15%84.70%88.20%91.10%89.75%
Qwen-9B/1580.30%86.55%88.50%89.70%89.30%
Gemma-12B/3182.20%86.95%88.45%89.65%89.10%

All nine Chunk-versus-Token gains pass correction. To check Joint's dual-target benefit across these representations, we also compare neighbor reconstruction on common targets: Joint explains 2.96–4.21 percentage points more target variance than Cross.

Sparse-classification gains and Joint's retention of neighboring-passage information extend across layers and model families.

Q1: Why these chunk lengths, and how does performance vary with length?

The 32/64/128/256/512 doubling grid covers a 16-fold context range. To identify each length's contribution, we split retrieval results from the mixed-length checkpoints.

Query tokensToken R@5Cross R@5
3250.00%73.23%
6462.09%77.23%
12866.60%78.86%
25672.30%81.44%
51275.64%86.22%

Cross's gain has a positive confidence interval at every query length.

To isolate added context from candidate difficulty, we fixed the same 249 documents and all 249 candidates.

Real tokensStandardized dense R@5Cross cosineCross paper CL-SR
3258.43%64.06%68.07%
6470.48%76.51%77.11%
12882.53%81.73%85.94%

Cross CL-SR gains 17.87pp from 32 to 128 tokens.

To test training-length effects, we retrained Mean/Cross for 100M tokens using fixed 512-token chunks. We use CSFCube to test fine-grained semantic matching between papers (CSFCube—A Test Collection of Computer Science Research Articles for Faceted Query by Example, NeurIPS Datasets and Benchmarks 2021). Compared with mixed-length training, top-ten ranking quality (nDCG@10) improves by approximately 16.3%/19.3%, respectively.

The length choice has empirical support: gains occur across the grid, and added context improves retrieval even with documents and candidates fixed.

Q3: How are explanation examples selected, and are scoring documents held out?

To rule out reuse of explanation evidence during scoring, we separate generation and scoring documents and verify their hashes globally and per feature. Prompt comparisons reuse the same examples, with method identities and activation labels hidden.

StageExamplesDocument separation
Explanation generation2 highest-activation examples and 2 activation-weighted samples4 development documents
Scoring4 active and 4 inactive contexts, each 128 tokensHeld-out documents, hash-verified
Coverage4,999 features and 9,998 judgments1 unsupported Joint feature marked NA

Lexical explanations are allowed, so descriptions can match the feature's actual level of abstraction.

Explanation development and scoring use separate documents, with verified separation at both global and feature level.

AuthorsOpenReview ↗

3, Reproducibility, Judge Calibration and Standard Benchmarks(Q4、Q6、Q7)

Q4: How many training seeds were used per SAE?

Our original multi-seed evaluation used three seeds per SAE at 100M. To test reproducibility at the main 1B budget, we also report three full-budget seeds, 42/43/44, for Token, Mean and Cross, each with 31,250 updates and validation checkpoint selection.

We reuse AG News because its fixed labels and five-coordinate classifier make training variation directly comparable without judge variation.

MethodSeed 42,k=5Seed 43Seed 44Mean±SD
Token85.15%83.95%84.65%84.58±0.60%
Mean88.20%90.00%89.55%89.25±0.94%
Cross91.10%90.90%89.25%90.42±1.02%

Mean/Cross average 4.67/5.83pp above Token, with positive training-seed confidence intervals. Label-subset sampling seeds are counted separately from SAE training seeds.

The core Mean/Cross sparse-classification gains reproduce across independent full-budget training seeds. This provides the requested seed accounting and evidence of training reproducibility.

Q6: Calibrate InterpScore with chance/random baselines and a human check.

To assess the reliability of automated interpretation, we add chance and random-dictionary baselines and a small human check (100 features, use the same rating panel, and the final result is the average of the assessments made by the seven individuals.). InterpScore averages the per-feature harmonic mean of active-context recall and inactive-context rejection on a 0–100 scale.

For the random-dictionary controls, we evaluate 100 features per method, matching each SAE's activation sparsity using training data and reusing cached contexts. Both explanation generation and scoring use the same LLM as in the paper. The chance baseline is the exact expected score when independently guessing each context's activation with probability 0.5 on the four-active/four-inactive scoring panel.

EvaluationTokenTemporalMeanCrossJoint
Chance-level InterpScore42.942.942.942.942.9
Random-dictionary InterpScore37.835.852.555.853.8
Paper InterpScore53.053.064.067.054.1
Human InterpScore58.460.569.265.062.7

Mean and Cross exceed their matched random-dictionary controls by 11.5pp and 11.2pp. Human assessment also places both above Joint, consistent with the automated evaluation within the chunk-level family. We compare each method against its own controls, without treating scores across different observation units as an interchangeable measure of interpretability.

The human check corroborates the method-level conclusions of the automated evaluation, with Mean and Cross leading the chunk-level family under both assessments.

Q7: Which standard evaluations are appropriate for chunk SAEs?

  1. Token-oriented metrics require care when the observation unit changes. We considered SAEBench's metrics, which were designed around token-level SAE activations. Applying them unchanged to chunk-level activations does not give a like-for-like comparison. For example, an explainer can readily infer “apple” from repeated highly activating tokens, while finding a common pattern across variable-length chunks is a substantially harder inference problem. An unchanged standard autointerp score therefore cannot directly rank the two levels. This is why our sparse-probing adaptation uses a common passage task and coordinate budget, while interpretation evaluates whether chunk features remain faithful tools.

  2. Steering supports a common output-level comparison. Following your suggestion, we use AxBench because all methods can be assessed on the same generated outputs and concept goals (AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders, ICML 2025). We adapt its public concepts and three official scoring templates to Qwen, using common prompts, scorers and scaling rules, with separate strength-selection and test prompts.

Shared scaling factor 1TokenTemporalMeanCrossJointUnsteeredCAAPrompt
Held-out harmonic score,0–20.2880.3240.4180.4240.3980.2930.3161.114

Factor 1 multiplies each concept's predefined intervention magnitude. CAA uses activation differences, while Prompt uses explicit text instructions. Cross achieves the highest score among the activation-intervention methods tested here. Prompt steering remains strongest, consistent with AxBench's finding. Mean/Cross/Joint also exceed the unsteered baseline.

Chunk SAEs remain interpretable tools and provide the strongest activation-steering result in this comparison. This supplementary experiment is consistent with the conclusions drawn from the experiments in our original paper.

Thank you again for your careful review and for reconsidering our paper!

Reviewer_rUfSOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper's chunk-level SAEs encode the mean activation of a text chunk instead of individual tokens. It introduces three variants: Mean-Chunk reconstructs the chunk itself, Cross-Chunk predicts a neighboring chunk from the same document, and Joint-Chunk combines both targets. Experiments on Qwen3.5-9B-Base show that chunk-level SAEs learn more high-level semantic features than token-level SAEs and perform better on retrieval, reasoning detection, classification, and steering.

Critical strengths

The paper evaluates the features on a wide range of downstream tasks, and the cue-free and cue-only controls in the reasoning experiment are a useful way to test whether features rely on surface words.

Critical weaknesses

The idea of training SAEs on sentence-level/chunk-level representations is not new. The authors only cite a few related papers(turn-averaged SAEs, step-level SAEs), but many other works have also explored this direction. The novelty of the method seems limited.

All experiments are conducted on one single model and one layer. The number of models and layers used in the experiments is far from enough, and there are no experiments on models larger than 10B. Also, the paper only compares with two baselines, and both of them are token-level SAEs. As I have mentioned, there are already quite a few prior works on chunk-level SAEs, but the authors do not compare with any of them.

If I understand correctly, the goal of Cross-Chunk is to use chunk A to predict chunk B. In this case, only the information shared by both chunks helps reduce the loss. Content only appears in chunk A does not help predict chunk B, so it is unlikely to be learned. So Cross-Chunk may mostly learn coarse, document-level topics.

The design of Joint-Chunk seems somewhat arbitrary. The partner decoder only reads the first 32768 latents, but the paper does not explain why half of the dictionary is used. The authors also describe α = 0.25 as “a randomly chosen representative” setting. And Joint-Chunk does not perform best on any metric. On every metric, it is outperformed by Mean-Chunk, Cross-Chunk, or both. Its steering score is the lowest among all five methods and is even lower than BatchTopK. Other α values in Appendix G do not change this. So it is unclear why Joint-Chunk is needed as a third method.

The interpretability evaluation may not be fair to token-level SAEs. In Appendix H.1, the prompt asks the judge to describe a general pattern, such as “topic, domain, style, intent, or discourse function”. It also tells the judge “do not search for a trigger token”. The scoring prompt has a similar rule. However, many features of token-level SAEs are triggered by specific tokens. If the judge cannot describe them in this way, these features will likely get lower scores. The authors also say this choice is meant to “align the explanation with the passage-level interpretation task”. This suggests the evaluation is built around chunk-level features, so the InterpScore comparison may not be fair.

Questions for authors
  1. Why do the authors use mean pooling instead of the last-token activation or other pooling methods?
  2. Since mean pooling may dilute token-specific surface information, could the higher HighLevelFraction partly come from pooling itself rather than from better feature learning? For example, how does HighLevelFraction change with chunk length?
  3. How many active features do chunks of different lengths use? Do any features mainly encode chunk length?
  4. The prompts for explanation and scoring tell the judge not to look for trigger tokens. How do token-level SAEs perform under a standard autointerp setup(such as the one in SAEBench), where token-level explanations are allowed?
10 comments
AuthorsOpenReview ↗

1, Contributions, Baselines and Generalization(W1、W2)

We are grateful for the reviewer’s constructive feedback! We have added targeted experiments that address each of your concerns and provide the evidence you requested.

W1: What is added beyond prior span-level SAEs and mean pooling?

Our motivation is to learn compact, interpretable passage features that remain useful across general text, varying input lengths and semantic matching tasks.

DimensionRepresentative prior designsOur designOur value
Training dataTurn-Averaged SAEs(arXiv2026):conversation turns. Base Models Know How to Reason(ICML2026):reasoning traces.Train dictionaries on general text and reuse them across tasks.Features applicable beyond dialogue and reasoning datasets.
Chunk granularityTurn-averaged and sentence-level methods use turn/sentence units.Explicit lengths {32,64,128,256,512}, covering all 25 ordered length pairs.Compact semantic representations for training examples with diverse lengths.
Training targetHypotheSAEs(ICML2025) and Decoding Dense Embeddings(EMNLP2025):self reconstruction. Interpret and Control Dense Retrieval(NAACL2025):self reconstruction plus retrieval-relation preservation.Mean reconstructs the observed mean. Cross predicts an independently processed neighbor mean. Joint retains both targets.Complementary descriptive and predictive features, with selective responses useful for passage matching.
  1. Training on general text makes the dictionaries reusable across retrieval, classification and reasoning analysis, with chunk construction independent of dialogue or reasoning annotations. This supports inspecting diverse training corpora using the same feature dictionaries.

  2. Explicit training across multiple chunk lengths addresses the varying sequence lengths encountered in training-data inspection and semantic deduplication. To quantify this benefit, our length audit finds broadly stable high-level feature fractions and approximately 124–177 active chunk coordinates across 32–512 tokens, with approximately 99% fewer coordinates than pooled Token at 512 tokens.

  3. Predicting an independently processed neighbor encourages Cross to capture meaning shared across passages, while Joint combines this predictive information with Mean's description of the observed chunk. Our activation analyses show interpretable, selective responses to relevant passage patterns. To isolate the target's contribution, we fix pooling and the training budget: Cross improves top-five paired-passage retrieval over Mean by 10.71pp.

Mean/Cross/Joint combine general text training, explicitly tested chunk granularities and complementary prediction targets to provide compact, interpretable features for practical semantic matching.

W2: Can the results extend to other models/layers and existing chunk-level baselines?

To test representation dependence, we trained all five methods for 1B at another Qwen layer and on Gemma-3-12B. We choose AG News because its public topic labels provide a common semantic test independent of our custom evaluations (Character-level Convolutional Networks for Text Classification, NeurIPS 2015).

Following SAEBench sparse probing, four balanced topic-versus-rest tasks use 4,000/1,000 train/test documents and five training-selected coordinates each. Entries are mean accuracy.

Model/blockTokenTemporalMeanCrossJoint
Qwen3.5-9B/2185.15%84.70%88.20%91.10%89.75%
Qwen3.5-9B/1580.30%86.55%88.50%89.70%89.30%
Gemma-3-12B/3182.20%86.95%88.45%89.65%89.10%

All nine Chunk-versus-Token comparisons pass correction over 180 tests. To check training variation, three full-budget seeds at the original Qwen layer give Cross an average 5.83pp gain over Token.

For an architectural baseline, we trained Nested for 1B tokens: its full code reconstructs tokens, while a prefix reconstructs the chunk mean. On the same 10M-token FP32 evaluation, Mean achieves higher variance explained for the observed chunk mean: 0.91498 versus 0.89983, a 1.7% relative gain.

To test fine-grained semantic matching, we use CSFCube, which retrieves papers with relevant backgrounds, methods or results (CSFCube, NeurIPS Datasets and Benchmarks 2021). Joint leads in top-ten ranking quality, measured by nDCG@10, at 0.4015, followed by Mean at 0.3953, Nested token-max at 0.3945 and Cross at 0.3839.

For a retrieval-SAE baseline, we adapted Park's official trainer to Qwen at a matched 100M-token budget and applied the paper's CL-SR scoring to the same CSFCube candidates.

100M dictionaryPark adaptationMeanCross
Paper CL-SR nDCG@100.28710.37670.3636

Mean and Cross improve over the Park adaptation by 31.2%/26.6%, respectively. The language model, token budget and candidates are fixed, while each method retains its training objective and optimizer protocol.

Chunk gains extend across model families and outperform matched-budget retrieval-SAE baselines, while Mean and Joint also compare strongly with Nested.

AuthorsOpenReview ↗

2, Relational Structure, Joint Design and Interpretability(W3、W4、W5/Q4)

W3: Does Cross retain relational information beyond coarse document topics?

To distinguish relational information from topic vocabulary, we tested contrasts containing identical words but different roles, negation scope, temporal order or conditional direction. Entities/templates are held out, and coordinate selection uses training labels only. The table averages three full 1B seeds.

RelationBag-of-words chanceCross,k=16Cross,k=64
Semantic roles50%53.33%68.89%
Negation scope50%63.06%72.50%
Temporal order50%62.50%68.61%
Conditional direction50%68.89%73.61%

At 64 coordinates, every relation scores above the 50% bag-of-words level. Negation and conditional direction exceed chance in every seed at both budgets after correction over 168 tests.

Cross retains readable relational structure even when topic words and word counts are identical.

W4: Why combine Mean/Cross in Joint, and how are the prefix and loss weight chosen?

Joint was motivated by two observations. Cross-Chunk features showed clear response patterns across texts and avoided activation on uninformative or low-quality content. Interpreting Mean-Chunk features revealed the high-level semantics we sought. We therefore asked whether one dictionary could combine the advantages of both SAEs. This combination is itself a contribution of the work.

  1. The prefix is an empirically explored design choice. To test how much of the dictionary should serve neighbor prediction, we trained four prefix fractions at 100M. A prefix is the first specified fraction of coordinates. The full code reconstructs self, while the prefix predicts the neighbor after a single shared BatchTopK selection.
ArchitectureSelf FVE
Mean0.8488
Cross0.7092
Joint,1/4 prefix0.8310
Joint,1/2 prefix0.8306
Joint,3/4 prefix0.8289
Joint,full prefix0.8183
Independent decoders0.8248

The half-prefix retains strong self reconstruction while assigning a substantial shared subspace to neighbor prediction. Its self FVE is 17.1% higher than Cross's and remains close to the quarter- and three-quarter-prefix settings.

  1. The loss weight gives a stable tradeoff. To test sensitivity to α, Appendix I varies it over 0.25/0.5/1/1.5 under the same 1B protocol. Composite reference-normalized reconstruction fidelity stays within 93.50–93.78%. At α=0.25, normalized self/neighbor weights are 0.8/0.2. The downstream experiments show the intended combination of Mean's self-preserving semantics and Cross's predictive structure.

Joint is valuable when an application needs to balance the complementary advantages of Mean-Chunk and Cross-Chunk.

W5&Q4: How should interpretability be assessed across token and chunk features?

  • Standard token-level autointerp scores are not directly comparable across these observation units. When an explainer sees many highly activating instances of a highlighted token such as “apple,” the lexical concept is easy to summarize. When it sees chunks of different lengths, identifying a shared pattern across complete passages is substantially harder. The explanation problems differ even under the same prompt, so an unchanged token-level pipeline does not provide a like-for-like ranking of token and chunk SAEs.

  • Our pipeline tests whether chunk features remain interpretable. We follow the standard autointerp approach, but ask whether features learned through chunk reconstruction or neighbor prediction still admit meaningful explanations, as token features do. The paper repeatedly presents them as faithful interpretability tools. The purpose is to establish their explanatory usefulness at the chunk level, rather than use the scores to claim universal superiority over token-level SAEs.

Our aim is higher-level features with cleaner activation patterns for concrete production and practical problems. Chunk SAEs complement token-level tools by making passage-level patterns accessible.

AuthorsOpenReview ↗

3, Pooling, Feature Learning and Compactness(Q1、Q2、Q3)

Q1: Why mean pooling rather than the last token or max pooling?

To test the pooling choice directly, we retrained Mean/Cross at 100M using mean, last-token and per-dimension max pooling for both input and target, with downstream documents and candidates fixed.

We use paired-passage R@5 to test linking, and reuse CSFCube nDCG@10 to test whether the pooling choice also matters for fine-grained cross-paper relevance.

PoolingMean R@5Cross R@5Mean CSFCube nDCG@10Cross CSFCube nDCG@10
Mean66.39%72.35%0.34660.3263
Last token22.32%18.03%0.19120.2247
Max34.39%44.41%0.22910.2185

Mean pooling wins both tasks for both methods. Cross's linking advantage over max pooling is 27.94pp.

Mean pooling is the strongest of the three tested choices after retraining both Mean-Chunk and Cross-Chunk. So tthis supplies an empirical basis for the pooling design.

Q2: Are high-level features learned, or mainly produced by pooling and longer chunks?

To test whether longer observation units alone explain the result, we compare HighLevelFraction across lengths while matching source/topic. This metric is the fraction of evaluated features judged to express a high-level semantic or functional pattern.

Method32 tokens64128256512
Mean40.63%40.63%34.38%40.63%37.50%
Cross36.12%30.00%34.00%31.00%32.00%
Joint34.24%31.21%39.39%39.39%30.30%

The fractions remain broadly stable across a 16-fold length range, rather than increasing consistently with chunk size.

To isolate dictionary learning from pooling, we freeze Token/Temporal and change only whether averaging occurs before or after encoding. We reuse AG News because a fixed five-coordinate classifier tests whether the resulting features carry useful semantics. The pooling-only controls score 84.75%/86.05%, versus Mean 88.20%, Cross 91.10% and Joint 89.75%. Same-width centered random dictionaries give 87.78±0.90%.

Length stability together with pooling-only and random controls supports a contribution from learned semantic features, rather than longer-span averaging alone.

Q3: How many features activate at each length, and are gains driven by length features?

  1. Compact activity is a practical motivation for chunk SAEs. In training-data deduplication, pooling token-level SAE codes can activate a very large set of features, including incidental lexical details that are not useful for passage matching. We seek a smaller set focused on meaningful passage patterns. To quantify this inspection burden, we measured active-feature counts at fixed inference thresholds. K=128 is the training-batch average, not a fixed count for each example.
TokensToken,per-token/pooledTemporal,per-token/pooledMeanCrossJoint
32117.55/1874.83118.14/1771.95177.36142.72170.73
64123.57/3221.13124.09/2992.82169.53139.66169.83
128125.75/5197.38126.21/4745.92150.64136.71152.04
256129.26/8142.37129.49/7318.81136.47123.63139.15
512129.09/11820.87129.34/10511.76126.90128.37131.66

“Pooled” counts the union of active token-code coordinates before any additional top128 truncation. At 512 tokens, token methods exceed 10,000 coordinates, while Chunk uses about 127–132, approximately 99% fewer than pooled Token. This makes passage-level inspection and matching substantially more compact.

  1. Semantic utility survives direct controls for length. To test whether length coordinates drive classification, we remove 32 length-associated coordinates selected on validation data. Cross retains 89.25% AG News accuracy, versus 91.10% before removal. To eliminate input-length variation, we also test exactly 32 real tokens on 937 common documents. Cross reaches 90.01%, versus Token 84.48% and standardized dense 85.70%, a 4.31pp gain over dense. The dense baseline uses averaged hidden states standardized on training data.

Chunk SAEs combine a compact set of active features with semantic gains that persist at constant text length.

Thank you again for your careful review and for reconsidering our paper!

Reviewer_JDTDOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies whether changing the observation unit and prediction target of sparse autoencoders can improve semantic feature discovery. It introduces three methods operating on mean-pooled token activations: Mean-Chunk reconstructs the observed chunk, Cross-Chunk predicts a neighboring chunk from the same document, and Joint-Chunk combines both objectives. The experiments compare these methods with token-level SAEs using matched training data and layer-21 activations from Qwen3.5-9B-Base. Evaluation covers reconstruction, feature properties, retrieval, reasoning-relation detection, classification transfer, and generation steering. The reported advantages are complementary: Mean-Chunk performs well on reasoning detection and steering, while Cross-Chunk leads retrieval and classification transfer.

Critical strengths

None

Critical weaknesses
  1. The scope of the evidence is limited to one layer of one language model, and I could not find accuracy broken down by chunk length, despite training with lengths from 32 to 512 tokens. This leaves unclear whether the advantages hold across representations and text lengths or are driven mainly by a particular setting.

  2. Several evaluation choices leave uncertainty about semantic generalization and practical utility. Same-document retrieval may exploit writing style or source, and the reasoning tests use generated and rewritten passages. Classifying ArXiv documents from later years does not rule out earlier exposure to ArXiv during SAE training. Steering reports the best result across multiple strengths for each feature. These choices make it difficult to judge performance on natural examples, document sources excluded from SAE training, and a fixed steering strength.

Questions for authors
  1. Can the mean-pooled representation described in lines 171-176 distinguish passages containing similar words but expressing opposite meanings because of word order? What word-order or positional information survives averaging, and how does the resulting representation differ from a bag-of-words representation? Could you provide contrastive examples?

  2. For the experimental setup in lines 258-265, do the advantages of chunk-level SAEs persist when activations come from other layers or another language model? I could not find accuracy broken down by length in the figures or tables. Could you report results separately for chunk lengths 32, 64, 128, 256, and 512? These lengths may contain very different information, so separate results would help show which lengths contribute to the gains.

  3. In the retrieval evaluation in lines 395-405, does the advantage persist when the relevant passage comes from a different document on the same topic, with distractors discussing closely related topics? This would help distinguish semantic matching from clues about writing style or source.

  4. Does the reasoning-relation recognition evaluated in lines 409-425 remain effective on real articles or conversations, including plan revisions without explicit cue words? This would help assess whether the results extend beyond passages generated and rewritten according to instructions.

  5. Given the ArXiv exposure noted in lines 449-455, does classification transfer remain effective on a document source confirmed to be absent from the SAE training corpus?

  6. In the steering evaluation in lines 459-469, does Mean-Chunk retain its advantage when all methods use the same fixed strength, rather than selecting the highest score across strengths separately for each feature? This would help establish whether an effective strength is easy to select in practical use.

  7. For text classification and reasoning recognition, how do the SAE-based methods compare with using Qwen directly, without an SAE?

4 comments
AuthorsOpenReview ↗

1, Robustness across Models, Layers and Lengths (W1/Q2)

We greatly appreciate reviewer’s helpful comments! We have added targeted experiments that directly address each of your concerns and provide the evidence you requested.

W1&Q2: Do the gains persist across models, layers and chunk lengths?

To test whether the gains depend on one representation, we trained all five methods with the same 1B-token-occurrence budget at Qwen block 15 and Gemma-3-12B block 31, alongside the original Qwen block 21.

We use AG News because its existing topic labels test sparse semantic access without a generated task or an LLM judge (Character-level Convolutional Networks for Text Classification, NeurIPS 2015). Following the SAEBench sparse-probing protocol, we use 4,000 training and 1,000 test documents. Four balanced binary tasks each distinguish one topic from the others. Each classifier uses five training-selected coordinates, denoted k=5. Entries are average accuracy.

Model/blockTokenTemporalMeanCrossJoint
Qwen3.5-9B/2185.15%84.70%88.20%91.10%89.75%
Qwen3.5-9B/1580.30%86.55%88.50%89.70%89.30%
Gemma-3-12B/3182.20%86.95%88.45%89.65%89.10%

All nine Chunk-versus-Token comparisons pass correction over 180 tests. Cross leads in all three settings, including the >10B model.

To check training reproducibility, we evaluated Token, Mean and Cross across three independent full-budget seeds, 42/43/44, in the original Qwen setting.

AG News,k=5TokenMeanCross
Three-seed mean±SD84.58±0.60%89.25±0.94%90.42±1.02%
Gain over Token,pp [95% seed CI]—4.67[0.91,8.43]5.83[2.90,8.76]

Here pp denotes percentage points. Both gains have positive training-seed confidence intervals.

To identify which context lengths contribute, we split the mixed-length checkpoints' paired-passage retrieval results by query length. R@5 is the percentage of queries whose paired passage appears in the top five. Each direction uses one side of a pair as the query.

Query tokensDirectionsToken R@5TemporalMeanCrossJointCross−Token,pp [95% document CI]
32147250.00%56.25%61.82%73.23%69.02%23.23[20.66,25.82]
6485262.09%67.14%66.08%77.23%73.71%15.14[12.12,18.52]
12847366.60%69.56%67.23%78.86%74.63%12.26[8.82,16.00]
25636172.30%71.75%74.24%81.44%82.27%9.14[5.32,12.95]
51231275.64%75.96%77.56%86.22%85.90%10.58[6.17,15.26]

Cross exceeds Token at every query length and in 24/25 query-length/target-length combinations.

To separate additional context from changes in candidate difficulty, we fixed the same 249 documents and all 249 candidates while varying real input length. Standardized dense averages hidden states and standardizes dimensions using training data. Cross top128 cosine retains the 128 largest feature activations. CL-SR retrieves through learned concept features (Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval, EMNLP 2025).

Real tokensStandardized dense R@5Cross top128 cosineCross paper CL-SR
3258.43%64.06%68.07%
6470.48%76.51%77.11%
12882.53%81.73%85.94%

Increasing context from 32 to 128 tokens raises Cross CL-SR by 17.87pp. To verify the dictionary advantage at identical text length on the full candidate set, we also reran the language model on exactly 32 real tokens for all 1,736 documents. Cross top128 cosine reaches 49.65%, versus standardized dense 46.63%.

To test training length separately, we retrained Mean/Cross at 100M using fixed 512-token chunks instead of mixed lengths. We use CSFCube because matching another paper's background, method or result tests specific semantic relevance across documents (CSFCube—A Test Collection of Computer Science Research Articles for Faceted Query by Example, NeurIPS Datasets and Benchmarks 2021). Its nDCG@10 rewards placing more relevant papers higher in the top ten. Fixed-length training gives approximately 16.3%/19.3% relative improvements for Mean/Cross.

The doubling grid covers a 16-fold context range. Fixed-document evaluation tests added context, while retraining tests the training unit, including its effect on distinct-text coverage at a fixed token budget.

Mean/Cross gains reproduce across model representations, training seeds and context lengths, with Cross retaining its retrieval advantage under matched text and candidates. So we addresses whether the reported advantages are driven by a single model, layer or length.

AuthorsOpenReview ↗

2, Semantic Generalization and Source Independence (W2/Q3、W2/Q4、W2/Q5)

W2&Q3: Does retrieval capture cross-document semantic relevance beyond source or topic?

The training target determines what is retained: Mean preserves the observed passage, Cross predicts related content, and Joint combines both.

To check source and topic shortcuts, we matched source, topic and length in the same-document analysis. Cross retains a 0.0198pp co-activation increment over matched different-document pairs.

To directly test your cross-document setting, we reuse CSFCube for aspect-specific semantic matching. It provides 50 queries from 34 papers and 211 hard pairs whose distractors match another aspect of the query paper, testing relevance beyond shared topic.

RepresentationCross-document nDCG@10Hard-pair accuracy
Token0.371556.34%
Temporal0.374356.68%
Mean0.395359.75%
Cross0.383958.31%
Joint0.401560.11%

All three Chunk methods exceed both token-level methods on both metrics. Joint leads, improving nDCG@10 by 8.1% relative to Token and hard-pair accuracy by 3.77pp.

To check whether this semantic information remains useful after decoding, we decode each query and compare it with candidate hidden-state means using common training-fitted standardization. Relevance labels fit no parameters.

Decoded-query nDCG@10TokenTemporalMeanCrossJoint
CSFCube0.39090.38600.41750.39720.4151

All three Chunk methods again exceed the token-level methods. Mean leads this readout, improving over Token by 6.8%.

Chunk dictionaries improve cross-document semantic matching, with Joint strongest on direct aspect ranking and hard pairs, and Mean strongest after decoding. These tests address the source/topic-shortcut concern using relevant passages from different documents.

W2&Q4: Do relation features transfer to natural articles and conversations?

To test whether the learned features capture relations in naturally written text, we use DiscoGeM because it provides human annotations of implicit discourse relations (DiscoGeM: A Crowdsourced Corpus of Genre-Mixed Implicit Discourse Relations, LREC 2022). We distinguish reason/result relations from other relations, separating entire documents across 4,645/734/1,113 train/calibration/test pairs.

To test sparse access to this relational information, we freeze each SAE and train a classifier using only five training-selected feature coordinates, denoted k=5. AUROC measures how well the classifier ranks reason/result examples above other relations, with 0.5 indicating chance.

MethodDiscoGeM AUROC,k=5
Token0.5605
Temporal0.5585
Mean0.6092
Cross0.5942
Joint0.5858

All three Chunk methods exceed both token-level methods. Mean/Cross/Joint improve AUROC over Token by 8.7%/6.0%/4.5%, respectively, with Mean leading the five-method comparison.

Mean/Cross/Joint expose human-annotated relations in natural text through only five feature coordinates, extending the paper's relational-information result to an independent public dataset. This directly addresses whether the observed relational information extends beyond generated and rewritten examples.

W2&Q5: Does classification transfer survive excluding the target source from SAE training?

To answer the SAE-training exposure concern, we removed ArXiv by source metadata, screened exact/near duplicates and retrained all five methods at 100M on Qwen blocks 21/27.

We then tested ArXiv subject classification on future-year papers, because this preserves the original transfer task while removing SAE exposure to its document source. The language model, supervision and full-representation readout remain fixed.

BlockTokenTemporalMeanCrossJointStandardized dense
2180.08%81.10%83.50%82.52%82.91%80.22%
2781.49%82.28%85.06%84.18%84.47%81.74%

Mean leads at both layers, exceeding standardized dense by 3.27/3.32pp. Its comparisons with Token and dense pass correction over 288 tests.

Mean's classification-transfer advantage persists after the entire target source is excluded from SAE training. We think this experiment can directly addresses earlier ArXiv exposure as an explanation for the transfer gains.

AuthorsOpenReview ↗

3, Steering, Sparse Classification and Relational Structure (W2/Q6、W2/Q7、Q1)

W2&Q6: Does steering work with one fixed strength shared across methods?

To test practical strength selection without feature-wise maximization, we evaluated 100 features per method at each fixed γ, with no stronger fallback interventions. The common score combines movement toward the intended concept with generation coherence.

Fixed γTokenTemporalMeanCrossJoint
153.9753.4455.1454.9454.64
354.7453.0656.9056.5154.64
556.3753.9357.9457.5656.54
757.4554.2058.2658.9957.68
1058.4554.6160.3061.0559.18

Mean and Cross exceed both token-level methods at every tested strength. Mean leads at γ=1/3/5 and Cross at 7/10. At γ=10, Cross's score is 4.4% higher than Token's.

Mean and Cross improve steering across a shared range of fixed strengths without choosing a separate optimum for each feature. This experiment addresses whether the steering benefit depends on feature-wise strength search.

W2&Q7: What is gained over using Qwen directly for classification and relation recognition?

To isolate the practical value of the SAE, we compare sparse SAE readouts with the same Qwen's averaged hidden states, including training-fitted per-dimension standardization. ArXiv subject classification uses common supervision, 512-token inputs and training-only coordinate selection.

To test compact access and temporal transfer separately, we report small-coordinate same-year classifiers and full-representation classifiers on future-year papers.

ReadoutTokenTemporalMeanCrossJointStandardized dense
16 coordinates,same-year71.44%74.02%77.44%74.97%75.24%73.14%
64 coordinates,same-year79.49%80.57%82.08%81.74%82.08%80.71%
Full representation,future-year81.54%81.20%82.96%83.71%82.62%80.22%

All three Chunk methods exceed dense in every row. Mean leads the 16-coordinate setting by 4.30pp over dense, while Cross leads future-year transfer by 3.49pp.

To verify that the compact-readout advantage extends to an independent labeled task, we reuse AG News for five-coordinate topic classification. Cross reaches 91.10%, versus Token 85.15%, pooling-only Token 84.75% and same-width centered random dictionaries 87.78±0.90%, with supervision and coordinate budgets matched.

To provide direct-Qwen references, label-likelihood classification without a fitted classifier gives 33.54%/31.79% same-year/future-year accuracy. On controlled reasoning relations, a supervised full-dense classifier reaches 0.9944 AUROC and 95.22% balanced accuracy, the average recall of positive and negative examples.

Feature selection searches the complete dictionaries. The practical contribution is reorganizing distributed model information into a few useful coordinates.

Mean provides strong compact classification, Cross provides strong future-year transfer, and Joint combines useful sparse readouts with temporal generalization.

Q1: Can averaging preserve word-order distinctions beyond bag-of-words content?

Mean pooling follows contextualization: in μ(x)=L−1∑tht(x)\mu(x)=L^{-1}\sum_t h_t(x), word order first changes the language-model states, which are then averaged. “Alice thanked Bob” and “Bob thanked Alice” have identical word counts but different contextualized states. “Bob was thanked by Alice” preserves the first meaning.

To test this computational distinction directly, we constructed identical-word relation contrasts, held out entities/templates and selected coordinates using training labels only. Entries average three full 1B training seeds.

RelationBag-of-words chanceCross,k=16Cross,k=64
Semantic roles50%53.33%68.89%
Negation scope50%63.06%72.50%
Temporal order50%62.50%68.61%
Conditional direction50%68.89%73.61%

At 64 coordinates, Cross distinguishes all four relation types despite identical word counts. Negation and conditional direction exceed chance in every seed at both budgets after correction over 168 tests.

A sparse chunk code retains role, negation, temporal and conditional distinctions that bag-of-words cannot express. So we address whether averaging necessarily removes the word-order information needed for these semantic contrasts.

Thank you again for your careful review and for reconsidering our paper! We greatly appreciate your valuable suggestions and will continue to refine the paper in the reviesd revisions.

Dear Reviewer,

As the discussion period draws to a close, we would be grateful if you could review our rebuttal and the additional results. We would be happy to clarify any remaining points. Thank you for your time and for reconsidering our submission.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Summary

We thank the reviewers for their thoughtful feedback and have revised the manuscript in response.

  • To address JDTD’s generalization concern, we tested across models, layers, chunk lengths, and sources, finding that chunk codes retain topic and relational information across the tested settings.
  • To address rUfS’s concerns about novelty and evaluation, we clarified the neighbor-prediction objective, adapted published SAE baselines, and added common-target reconstruction and fixed-strength steering tests.
  • To address YvRb’s concerns about readability and evaluation, we revised the figures and checked the retrieval and interpretation findings with stronger controls and non-author ratings. Both checks support the original findings.

The additional experiments address the reviewers’ core concerns about generalization, the contribution of neighbor prediction, and evaluation rigor while reinforcing our original conclusion that chunk-level SAEs provide reliable semantic feature.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.