acceptodds
Under review as a conference paper at ICLR 2027

ClasSAE: Class-Aligned Sparse Autoencoders via Differentiable Feature-Class Affinity

11 views

Abstract

Sparse Autoencoders (SAEs) began as an unsupervised tool for decomposing neural representations into sparse, interpretable features, and are increasingly used not only for passive analysis but also for active interventions such as unlearning, bias mitigation, and concept editing. A central challenge for these editing and steering methods is reliably matching features to target concepts; most current approaches address this by computing post-hoc scores over an already-trained, frozen dictionary. We instead introduce ClasSAE, a novel method that both automatically assigns classes to features and guides the encoder toward class-separable representations during training. Specifically, we apply a differentiable Top- operator to a trainable feature–class affinity matrix with per-feature budgets, coupling the features selected for each sample to the classes they are trained to represent. Because gradients flow through the selection of active features rather than only through their magnitudes, the encoder and the affinity matrix co-adapt rather than being fit in separate stages. The result is a dictionary that is both class-separable and class-annotated, with no need for post-hoc probing. We propose three variants for enforcing sparsity within this framework, which achieve comparable overall performance with slightly different trade-offs. Using CLIP ViT-L/14 embeddings on ImageNet, we show that the learned affinity matrix agrees closely with an independently estimated post-hoc feature-class matrix computed on held-out data. The model also supports direct class prediction from the encoder and affinity matrix alone, without a separately fitted classifier, and its more class-aligned encoder yields improved separation in Targeted Probe Perturbation evaluations. These results show that known concept annotations can be used not merely to locate relevant features in an SAE dictionary after training, but also to shape the dictionary itself during learning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Sparse Autoencoders, Mechanistic Interpretability, Differentiable Top-K, Feature Attribution / Concept Alignment, Representation Steering, Weak Supervision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WoHcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces ClasSAE that trains a SAE (sparse autoencoder) to jointly learn reconstruction and feature-class affinity matrix. Technically, ClasSAE presents three architectural variants and provides a theoretical analysis of agreement objective. Empirically, ClasSAE demonstrates improved class alignment, dictionary utilization, and reduced reconstruction fidelity. Qualitative example also show meaningful features.

Critical strengths
  1. ClasSAE focus on an important practical problem in SAE i.e., aligning sparse autoencoder features with meaningful classes. This would reduce the need for post-hoc feature annotation and ease usage for interpretation and targeted interventions.

  2. ClasSAE offers a flexible interface between sparse features and concepts. Learning feature-class affinity matrix can reconcile both shared features and multiple features representing single concept.

  3. Empirically, ClasSAE show promising results on dictionary utilization, class prediction, and interventions.

Critical weaknesses
  1. Reviewer's biggest concern is on fairness. Current comparison does not isolate the effect of supervision. Other works like G-SAE [1], AlignSAE [2] also provide supervision while there is no comparison between other methods who are also using supervison.

  2. Also, the quantitative evaluation only happens on CLIP embeddings for ImageNet-1K, with only a single dictionary size and association budget.

[1] Measuring and Guiding Monosemanticity

[2] AlignSAE: Concept-Aligned Sparse Autoencoders

Questions for authors
  1. How does ClasSAE compare with a BatchTopK SAE trained with an ordinary supervised classification head and a G-SAE-style guided objective, using the same labels?

  2. Two metric definitions seem to be inconsistent with the reported values. If both predicted and reference class sets contain the top k classes, why do Table 2 precision and recall differ? Also, how can Table 3’s stated “gap between the top two class scores” be negative? Please clarify the implemented definitions and whether any results require correction.

4 comments
AuthorsOpenReview ↗

Response to Reviewer WoHc: Summary of Changes and Point-by-Point Answers (1/2)

We thank the reviewer for the careful reading and for recognizing the practical value of aligning SAE features with meaningful classes, the flexible interface between sparse features and concepts, and the results on dictionary utilization, class prediction and interventions. Both main concerns, that the comparison does not isolate the effect of supervision and that the evaluation covers a single setting, led to changes: three supervised baselines (Sup BatchTopK, ClassifSAE [2], G-SAE [1]) now appear in all main experiments (Tables 1–9), a reduced benchmark at a smaller dictionary size and budget is reported in Appendix H, and the metric definitions were clarified (Appendix C, Table 3 caption). Tables here are subsets of the paper's tables. Numbers are 5-seed means unless stated. Table numbers refer to the revised paper (Tables 5–7 and 10–14 are new).

W1 / Q1. Comparison with supervised methods

We now include Sup BatchTopK (BatchTopK + ordinary linear classification head), G-SAE [1] (the guided objective of Härle et al.) and ClassifSAE [2], with the same ImageNet labels, L0 ≈ 380 and protocol (Section 4 "Setup", Appendix D). Baselines are read out with the post-hoc matrix. "Emp." is the same empirical-PMI edit rule for all models, "native" is ClasSAE's own rule.

Table A: subset of Tables 1 (FVE), 2 (F1), 3 (Acc), 4 (TPP PMI) and 6 (off-diagonal, PMI).

ModelMatrix / ruleFVEF1AccTPP PMIOff-diag
Sup BatchTopKpost-hoc / emp.0.8810.5370.3380.2000.071
ClassifSAEpost-hoc / emp.0.7550.6510.7000.4040.077
G-SAEpost-hoc / emp.0.6990.9900.7230.2020.282
ClasSAE Hardtrained / native0.9100.8510.7890.4690.012
ClasSAE Hardpost-hoc / emp.0.9100.9190.6680.4720.010
ClasSAE L1trained / native0.8640.7960.7990.4750.003
  • Classification (Table 3). With trained M, all ClasSAE variants exceed all three baselines on accuracy and margin (best baseline: G-SAE, 0.723 vs. 0.750–0.799). This compares ClasSAE's trained M with the baselines' post-hoc matrix. With the post-hoc matrix for every model, only ClasSAE L1 (0.766) stays ahead of G-SAE and ClassifSAE; ClasSAE (0.555) and ClasSAE Hard (0.668) fall below them. Sup BatchTopK reaches only 0.338.
  • Intervention (Tables 4–6). Under the same PMI rule, ClasSAE variants score 0.465–0.472 on TPP PMI vs. 0.404 (ClassifSAE), 0.202 (G-SAE) and 0.200 (Sup BatchTopK), with lower collateral damage. We concede that ClassifSAE leads standard TPP at k ≥ 200, but Appendix F describes this as overshoot: its on-diagonal drop of 0.761 at k=600 (Table 5) means target-probe accuracy of about 0.23, below the 0.5 chance level, with off-diagonal damage 0.212 vs. 0.019 for ClasSAE Hard (Table 6).
  • Dictionary use and reconstruction (Table 1). G-SAE keeps only 9.7% of latents alive, and its high F1 reflects its preassigned one-feature-per-class design. ClasSAE Hard uses 100% of the dictionary.
  • Shuffle control. Training with permuted labels collapses on annotation, classification and native-rule editing (Tables 2–4), so the coupling alone, without true labels, gives no class structure.

On AlignSAE [3]: we investigated it, but its loss includes a value loss that depends on the structure of the LLM it is trained in, which we cannot mimic directly for CLIP vision embeddings. Without that term, its objective is very close to the supervised baselines we already include, so we did not add it as a separate baseline.

AuthorsOpenReview ↗

Response to Reviewer WoHc: Summary of Changes and Point-by-Point Answers (2/2)

W2. Single dataset, dictionary size and association budget

We agree that the quantitative evaluation is limited to CLIP ViT-L/14 on ImageNet-1k. To check that the findings do not hinge on one dictionary size and budget, Appendix H repeats a reduced benchmark at d=4096, ρ=20 (target L0=64), comparing Sup BatchTopK, ClassifSAE and ClasSAE Hard (single seed; Tables 10–14).

Table B: subset of Tables 10–14 (single seed). "Native" and "emp." rows are the same ClasSAE Hard model evaluated with trained M or with the post-hoc matrix.

ModelMatrix / ruleFVEF1AccTPP 600TPP PMIOff-diag PMI
Sup BatchTopKpost-hoc / emp.0.7100.4500.6530.4570.4400.043
ClassifSAEpost-hoc / emp.0.6690.6820.5840.3640.3180.104
ClasSAE Hardtrained / native0.7800.8940.7830.4500.4710.003
ClasSAE Hardpost-hoc / emp.0.7800.9330.7420.4500.4660.011

The main conclusions carry over: ClasSAE Hard has the best FVE, annotation faithfulness, classification accuracy, TPP PMI and collateral damage. As in the main experiments it is not the best on standard fixed-k TPP, where Sup BatchTopK is highest (0.484, 0.468, 0.457 at k=20, 200, 600 vs. 0.335, 0.466, 0.450; Table 13). Since these results are single-seed, we make no ranking claim there. Beyond this setting we have no quantitative results on other datasets or backbones, so we do not claim that the numbers generalize.

Q2. Metric definitions

We thank the reviewer for catching these two inconsistencies. In both cases the descriptions in the paper were imprecise, and the reported values are correct.

Precision vs. recall (Table 2). The sets are not "top-k classes" for a common k, which was the source of the confusion. Appendix C (Eq. 8–9) now gives the full procedure:

  • Each feature has its own non-integer budget, derived from the post-hoc firing-rate matrix with a threshold (same procedure for baselines). Its claimed set is the top classes of its row, as many as its budget.
  • The prediction set comes from the train-side matrix (trained M for ClasSAE) and the reference set from the test-side matrix, each with its own budget.

Precision and recall are therefore equal only when the two sets have equal size. This holds approximately for post-hoc rows (ClasSAE 0.910 vs. 0.906), but not for trained M, whose learned budgets are not tied to firing rates (0.766 vs. 0.889). Table 2 now reports both variants for every ClasSAE model.

Margin (Table 3). The caption was wrong, not the results. It now defines Margin as the mean difference between the true-class score and the highest-scoring incorrect-class score. It is signed, hence negative when the true class is not ranked first (BatchTopK: accuracy 0.002, margin -0.148). No results require correction.

We thank the reviewer again for the constructive feedback.

References

  1. Härle et al. Measuring and guiding monosemanticity. NeurIPS 2025 (Spotlight).
  2. Le Bail et al. Unveiling decision-making in LLMs for text classification: Extraction of influential and interpretable concepts with sparse autoencoders. arXiv:2506.23951, 2025.
  3. Yang et al. AlignSAE: Concept-Aligned Sparse Autoencoders. Transactions on Machine Learning Research, 2026. https://openreview.net/forum?id=I9UjKxW4nq

Thank you for the detailed response and additional experiments. The supervised baselines strengthen the evaluation, and the revised definitions resolve my concerns about precision/recall and the signed margin.

My main remaining concern is the classification comparison. Section 4 explicitly evaluates ClasSAE using its trained MM, while supervised baselines use a post-hoc matrix rather than their own trained classification heads. Could you also report Sup BatchTopK and ClassifSAE using those heads? The current comparison demonstrates the usefulness of ClasSAE’s annotation interface, but does not fully establish its classification advantage over ordinary supervised SAE training.

You raised an important point that helps clarify the purpose of the classification experiment. We do not claim that ClasSAE is superior to dedicated models trained for classification. The goal of the experiment is to validate the matrix M as a meaningful addition: if M captures real class structure, the same matrix that was built for annotation and editing should also work as a classifier with no extra training. The question is whether one model can offer reconstruction, classification and intervention quality at the same time, which in our experiments the supervised baselines do not provide. As requested, we report the baselines read out with their own trained classification heads (5 seeds, same data and settings as Table 3). We include these results to give a complete picture of the classification experiment, not to claim an advantage.

Table D: classification with each model's own readout (baselines: own trained head; ClasSAE: trained matrix M). ClasSAE rows are a subset of Table 3.

ModelReadoutAccAcc Top 5Acc Top 10F1Margin
Sup BatchTopKown head0.841±0.0000.972±0.0000.986±0.0000.839±0.00062.783±0.458
ClassifSAEown head0.800±0.0180.958±0.0070.977±0.0040.797±0.01841.723±8.325
ClasSAEtrained M0.750±0.0020.914±0.0030.941±0.0010.748±0.0030.096±0.001
ClasSAE Hardtrained M0.789±0.0010.940±0.0000.962±0.0000.787±0.0010.131±0.001
ClasSAE L1trained M0.799±0.0010.955±0.0000.976±0.0000.796±0.0010.168±0.000

With their own heads, the supervised baselines reach equal or higher accuracy (Sup BatchTopK 0.841, ClassifSAE 0.800 vs. 0.750–0.799 for ClasSAE), and we make no claim of a classification advantage. Margins are not comparable across the two readouts: for the baselines it is a gap between unconstrained head logits, for ClasSAE a gap between bounded matrix scores. We have added these results, with the clarification, as a new appendix (Appendix J, Table 15), and revised Section 4 and the Conclusions so that our classification claims refer to the matrix readout rather than to classification in general.

We read the result as supporting the matrix, for two reasons:

  • M is a strongly constrained readout and is still close. It is sparse and non-negative, so its decision depends only on which features are present, while the heads of the baselines are unconstrained real-valued weights that can also use negative evidence between features (and ClassifSAE classifies from a 64-dimensional bottleneck, Appendix D). ClasSAE Hard (0.789) and ClasSAE L1 (0.799) are within 0.011 and 0.001 of ClassifSAE (0.800), and within 0.052 and 0.042 of Sup BatchTopK. We take this as evidence that M reflects high-quality class structure, although we did not isolate which constraint matters most.
  • Basic supervision does not give all qualities at once. The supervised baselines' accuracy comes with lower reconstruction (FVE 0.881 and 0.755 vs. 0.910 for ClasSAE Hard, Table 1), lower annotation faithfulness (F1 0.537 and 0.651 vs. 0.851, Table 2) and weaker editing (TPP PMI 0.200 and 0.404 vs. 0.469, Table 4, with more collateral damage, Table 6). ClasSAE Hard reaches competitive classification while keeping the other three.
Reviewer_pKAFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ClasSAE, a sparse autoencoder that incorporates class information during training to learn more class-aligned features. It jointly learns feature-class associations with the encoder and decoder and explores three approaches to sparsity. The authors also provide a theoretical analysis linking these associations to pointwise mutual information. Experiments on CLIP embeddings from ImageNet-1k show improved class alignment and targeted interventions, but at the cost of reduced reconstruction quality.

Critical strengths
  1. Joint learning of features and class associations. The main strength of the paper is that class information is incorporated directly into SAE training, avoiding the need to identify class-related features afterwards. The learnable budgets are also interesting, as they allow features to be shared across classes. The theoretical analysis provides useful support for the approach.

  2. Promising experimental results. The evaluation goes beyond reconstruction and shows clear improvements in class alignment. ClasSAE Hard achieves an F1 score of 0.851 compared with 0.682 for the strongest baseline. The intervention results are also encouraging, suggesting that the learned features are more useful for class-specific interventions.

Critical weaknesses
  1. Lack of comparison with simpler supervised baselines. My main concern is that ClasSAE uses class labels during training, while the baselines do not. It is therefore unclear whether the improvements come from the proposed mechanism or simply from supervision. A standard SAE trained with an additional classification loss would be an important baseline, particularly given the drop in reconstruction quality (FVE of 0.934 versus 0.973 for BatchTopK).

  2. Limitations of the intervention experiments. The intervention results are promising but vary considerably with the number of selected features. ClasSAE Hard performs poorly at k=20 but much better at k=600, raising questions about its usefulness for sparse interventions. Also, ClasSAE uses its learned affinities for feature selection, while the baselines use estimated PMI scores. Applying the same selection method across models would make the comparison fairer and the conclusions more convincing.

Questions for authors

Could the authors compare ClasSAE with a standard SAE using an additional classification loss under similar training conditions? This would help clarify whether the gains come from the proposed method or simply from using class labels.

Could the authors use the same feature-selection procedure and comparable intervention strengths across all models? This would help establish whether ClasSAE's improvements hold under a fairer comparison.

3 comments
AuthorsOpenReview ↗

Response to Reviewer pKAF: Summary of Changes and Point-by-Point Answers (1/2)

We thank the reviewer for the careful reading and for recognizing the joint learning of features and class associations, the learnable budgets that let features be shared across classes, and the theoretical analysis. Both main concerns, the missing simple supervised baselines and the intervention comparison, led to changes: three supervised baselines (Sup BatchTopK, ClassifSAE [2], G-SAE [1]) now appear in all main experiments (Tables 1–9), and the intervention rules are specified in Appendix E. Tables here are subsets of the paper's tables (supervised baselines and ClasSAE variants). Numbers are 5-seed means unless stated. Table numbers refer to the revised paper (Tables 5–7 and 10–14 are new).

W1 / Q1. Added value over a standard SAE with a classification loss

We added exactly this baseline, Sup BatchTopK (BatchTopK + linear classification head), plus ClassifSAE [2] and G-SAE [1], with the same labels, L0 ≈ 380 and protocol (Section 4 "Setup", Appendix D). Baselines are read out with the post-hoc matrix. "Emp." is the same empirical-PMI edit rule for all models, "native" is ClasSAE's own rule.

Table A: subset of Tables 1 (FVE), 2 (F1), 3 (Acc), 4 (TPP PMI) and 6 (off-diagonal, PMI).

ModelMatrix / ruleFVEF1AccTPP PMIOff-diag
Sup BatchTopKpost-hoc / emp.0.8810.5370.3380.2000.071
ClassifSAEpost-hoc / emp.0.7550.6510.7000.4040.077
G-SAEpost-hoc / emp.0.6990.9900.7230.2020.282
ClasSAE Hardtrained / native0.9100.8510.7890.4690.012
ClasSAE Hardpost-hoc / emp.0.9100.9190.6680.4720.010
ClasSAE L1trained / native0.8640.7960.7990.4750.003
  • Labels alone do not explain the gains. Sup BatchTopK is far behind on classification and annotation. ClasSAE Hard's F1 of 0.851 (the reviewer compared it with an unsupervised L1L^1 SAE 0.682) also exceeds ClassifSAE (0.651) and Sup BatchTopK (0.537); G-SAE's 0.990 reflects its preassigned design (9.7% of latents alive, Table 1). With trained M, all ClasSAE variants beat all three baselines on accuracy. The Shuffle control collapses (Tables 2–4).
  • Reconstruction. The FVE drop vs. BatchTopK (0.934 vs. 0.973, Table 1) is real and comes from the class signal (Shuffle: 0.977). ClasSAE and ClasSAE Hard retain more FVE than all three supervised baselines.

W2 / Q2 (a). Same feature-selection procedure for all models

TPP PMI already applies the same empirical-PMI rule to every architecture, and ClasSAE is additionally shown with its native rule. Appendix E now specifies both, and Table 4 reports ClasSAE under both, so the comparison does not depend on the learned affinities:

  • Common rule (Appendix E, Eq. 10), all architectures: each feature's activation is scaled by its 0.9 quantile and multiplied by its normalized empirical PMI with the target class (counted on the train split), times a strength multiplier (5 for all models). The edit is $z' = z \cdot (1 − strength) $.
  • Native ClasSAE rule (Appendix E, Eq. 11): soft selection weight times the trained matrix column of the target class, times the same multiplier.
  • Fixed-k TPP (TPP 20/200/600) does not use the affinity matrix at all.

Under the common rule (the "emp." rows of Table A) all ClasSAE variants still lead (TPP PMI 0.465–0.472 vs. 0.404 for ClassifSAE). At the shared multiplier the target effect is similar (on-diagonal 0.482 for ClasSAE Hard, 0.481 ClassifSAE, 0.484 G-SAE; Table 5), so the advantage comes from lower collateral damage and smaller edits (L2 of the edit: 2.54 vs. 22.85 for ClassifSAE and 15.19 for G-SAE; Table 7). Section (b) examines strength.

AuthorsOpenReview ↗

Response to Reviewer pKAF: Summary of Changes and Point-by-Point Answers (2/2)

W2 / Q2 (b). Comparable intervention strength

We agree that a matched-strength comparison would be valuable, and we tried. Appendix F states that we make no matched-strength claim. Below we explain why matching is hard and report our attempts to answer the question. We do not find them significant enough to include in the paper, but we are open to including them if the reviewer thinks otherwise.

Why matching is hard. The paper uses one untuned multiplier (5) for all models.

  • ClasSAE is stable across multipliers: for ClasSAE Hard, going from 5 to 1000 moves TPP only from 0.472 to 0.466 (L2 of the edit 2.54 to 4.86), with on-diagonal fixed at 0.482 (probe at chance).
  • The baselines are very sensitive and noisy across seeds: for ClassifSAE, multiplier 1.3 vs. 5 gives L2 3.55 vs. 22.85 and on-diagonal 0.413 vs. 0.481. For Sup BatchTopK, 2.3 vs. 5 gives on-diagonal 0.032 vs. 0.271 (single-seed vs. 5-seed mean). The L2 standard deviations across seeds are large (Table C), so one multiplier gives different edit sizes on different seeds and a strict match would need per-seed tuning.

We ran two limited experiments with the common empirical rule (G-SAE excluded for lack of rebuttal time). In the 5-seed setting, ClasSAE Hard is at multiplier 1000 (its maximum, since strength is clamped at 1) and the baselines are hand-tuned. In the single-seed setting, baselines are tuned to approximately match the L2 of ClasSAE Hard at multiplier 10.

Table C: additional rebuttal experiments, not in the paper.

SettingModelMult.L2On-diagOff-diagTPP
5 seedsClasSAE Hard10004.86±0.790.482±0.0000.016±0.0020.466±0.003
5 seedsClassifSAE2.412.12±5.640.461±0.0200.022±0.0070.439±0.026
5 seedsSup BatchTopK3.86.12±0.630.228±0.0620.043±0.0110.186±0.054
1 seedClasSAE Hard103.220.4820.0110.471
1 seedClassifSAE1.33.550.4130.0050.408
1 seedSup BatchTopK2.33.670.0320.0050.027

Taken with caution (limited evidence: hand-tuned multipliers, and a single seed in the second setting), this suggests the following. At similar edit strength, ClassifSAE either overshoots (Appendix F, fixed-k selection at k ≥ 200) or slightly under-edits: its on-diagonal effect is 0.461 at L2 12.1 (5 seeds) and 0.413 at L2 3.6 (single seed), vs. 0.482 for ClasSAE Hard in both settings. ClasSAE Hard stays reliable over a much wider multiplier range (TPP 0.466–0.472 from 5 to 1000), while the baselines swing widely (ClassifSAE 0.404–0.439, Sup BatchTopK 0.027–0.200 across the multipliers we tried). In the single-seed setting ClassifSAE has lower off-diagonal damage (0.005 vs. 0.011), at the cost of this weaker target effect, so ClasSAE's collateral advantage at multiplier 5 should not be read as holding at matched strength. Because edits are not strictly matched, we make no matched-strength claim.

W2 (c). Sparse interventions (small k)

We acknowledge that ClasSAE is weak when only a few latents can be edited (TPP 20 of 0.064 for Hard vs. 0.409 for ClassifSAE and 0.429 for Sup BatchTopK, Table 4). Its classes are spread over more, finer-grained features, so a handful cannot erase a class. This concerns the number of edited latents, not the size of the edit: under TPP PMI the L2 change is 2.5–2.8 with on-diagonal 0.48 and off-diagonal 0.010–0.014 (Tables 5–7). Conversely, a small k is not a small edit: at k=20 ClassifSAE's L2 is 29.1 vs. 3.9 for ClasSAE Hard. At k=20 the low collateral damage of ClasSAE reflects a mild edit rather than precision (Appendix F). At d=4096 the weakness is smaller (0.335, Table 13, single seed), though Sup BatchTopK (0.484) is still higher.

For k ≥ 200, ClassifSAE leads standard TPP, but Appendix F describes this as overshoot: its on-diagonal drop of 0.686 and 0.761 (k=200, 600) means probe accuracy of about 0.30 and 0.23, below the 0.5 chance level, with far more damage to untargeted probes.

ClasSAE is best suited to settings where avoiding collateral damage matters more than minimizing the number of edited latents (Section 4, Appendix F).

We thank the reviewer again for the constructive feedback.

References

  1. Härle et al. Measuring and guiding monosemanticity. NeurIPS 2025 (Spotlight).
  2. Le Bail et al. Unveiling decision-making in LLMs for text classification: Extraction of influential and interpretable concepts with sparse autoencoders. arXiv:2506.23951, 2025.

Dear Reviewer pKAF,

We would like to thank you once again for your valuable feedback. We hope that our previous responses have adequately addressed your questions. However, should you have any further questions or require additional clarification, please do not hesitate to contact us. We would be more than happy to address any remaining concerns.

Reviewer_JbdWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ClasSAE jointly learns a sparse autoencoder and a feature-class affinity matrix by coupling differentiable feature selection with globally budgeted class associations. It introduces soft-gradient BatchTopK, hard BatchTopK, and L1 variants, together with a fixed-encoder allocation analysis and a conditional correspondence with PMI thresholding. Evaluation includes five-seed CLIP ViT-L/14 experiments on ImageNet-1k covering reconstruction, annotation agreement, classification, and interventions, supplemented by joint-classifier unlearning and a qualitative DINOv3 patch-token example.

Critical strengths
  1. The method combines explicit class annotation with flexible feature sharing through a learned affinity matrix coupled to differentiable support selection. Compared with G-SAE’s preassigned single feature per concept, this permits multiple features per class and shared features across classes within SAE training.
  2. The five-seed ImageNet results demonstrate practical value for direct classification and interventions involving larger feature sets. ClasSAE Hard achieves native top-1 accuracy of 0.789 and TPP of 0.517 at k=600k=600, compared with 0.432 for the strongest external TPP baseline, Standard, although reconstruction fidelity and small-kk performance are weaker.
Critical weaknesses
  1. The experiments do not establish the added value of the budgeted affinity-selection coupling over ordinary supervised SAE training. ClassifSAE already jointly trains a TopK SAE and a classification head using LLM-predicted labels. Because the external baselines are unsupervised and Shuffle removes meaningful class supervision, the comparisons cannot attribute the gains specifically to ClasSAE’s new coupling, leaving its incremental practical contribution unresolved despite the reconstruction cost.
  2. TPP PMI compares different intervention rules across architectures, while Table 5 measures the advantage over a shuffled intervention rather than absolute forgetting. Consequently, these results do not isolate improved encoder separation or establish superior selective unlearning across methods, limiting the claimed low-strength editing value despite the positive fixed-kk TPP results.

REF:

  1. ClassifSAE: Le Bail, M., Dentan, J., Buscaldi, D., & Sonia, V. (2026, March). Unveiling decision-making in llms for text classification: Extraction of influential and interpretable concepts with sparse autoencoders. In Findings of the Association for Computational Linguistics: EACL 2026 (pp. 2477-2504).
Questions for authors
  1. Under identical ImageNet labels and matched sparsity, how does ClasSAE compare with an SAE trained jointly with a linear classification head, following ClassifSAE, on annotation alignment and intervention performance?
  2. What exact procedure constructs M^\hat M and the predicted and reference class sets in Table 2, including the ki/dk_i/d threshold, noninteger budgets, and budgets for the baselines? In particular, why does ClasSAE obtain precision of 0.766 and recall of 0.889 when the stated top-kik_i protocol should produce equal values if both sets use identical budgets and averaging?
  3. Table 3 defines Margin as the difference between the highest and second-highest class scores, but reports -0.148 for BatchTopK. What formula was actually used to compute Margin, and how does it produce negative values?
  4. Do the TPP PMI gains and low-strength conclusion persist when the same empirical PMI estimator and intervention formula are applied to every encoder? Please specify how the current ClasSAE intervention converts MM and native selection weights into editing strength so that this comparison can be reproduced.
  5. What are the baseline, graded-intervention, and shuffled-intervention forget and retain accuracies underlying Table 5 for each architecture? These values are needed to distinguish absolute unlearning effectiveness from sensitivity to the shuffled control.
4 comments
AuthorsOpenReview ↗

Response to Reviewer JbdW: Summary of Changes and Point-by-Point Answers (1/2)

We thank the reviewer for the careful reading and for recognizing the flexible feature sharing of the learned affinity matrix (vs. G-SAE's [1] single preassigned feature per concept). Both main concerns, the missing supervised baselines and the non-uniform intervention comparison, led to changes: three supervised baselines (Sup BatchTopK, ClassifSAE [2], G-SAE [1]) now appear in all main experiments (Tables 1–9), and the metric definitions and intervention rules are specified in Appendices C and E. Tables here are subsets of the paper's tables (supervised baselines and ClasSAE variants). Numbers are 5-seed means. Table numbers refer to the revised paper (Tables 5–7 and 9 are new; submitted Table 5 is now Table 8).

W1 / Q1. Added value over ordinary supervised SAE training

We added exactly this baseline, Sup BatchTopK (BatchTopK + linear classification head), plus ClassifSAE [2] and G-SAE [1], with the same labels, L0 ≈ 380 and protocol (Section 4 "Setup", Appendix D). Baselines are read out with the post-hoc matrix. "Emp." is the same empirical-PMI edit rule for all models, "native" is ClasSAE's own rule.

Table A: subset of Tables 1 (FVE), 2 (F1), 3 (Acc), 4 (TPP PMI) and 6 (off-diagonal, PMI).

ModelMatrix / ruleFVEF1AccTPP PMIOff-diag
Sup BatchTopKpost-hoc / emp.0.8810.5370.3380.2000.071
ClassifSAEpost-hoc / emp.0.7550.6510.7000.4040.077
G-SAEpost-hoc / emp.0.6990.9900.7230.2020.282
ClasSAE Hardtrained / native0.9100.8510.7890.4690.012
ClasSAE Hardpost-hoc / emp.0.9100.9190.6680.4720.010
ClasSAE L1trained / native0.8640.7960.7990.4750.003
  • Labels alone do not explain the gains. Sup BatchTopK is far behind on classification and annotation. With trained M, all ClasSAE variants beat all three baselines on accuracy, and beat ClassifSAE and Sup BatchTopK on F1. G-SAE's F1 reflects its preassigned design (9.7% of latents alive, Table 1). The Shuffle control collapses (Tables 2–4).
  • Reconstruction. The FVE cost vs. BatchTopK (0.934 vs. 0.973, Table 1) is real and comes from the class signal (Shuffle: 0.977). ClasSAE and ClasSAE Hard retain more FVE than all three supervised baselines.
  • ClassifSAE at k ≥ 200. We concede its higher standard TPP, but Appendix F describes this as overshoot: on-diagonal drop 0.761 at k=600 (Table 5) means target-probe accuracy of about 0.23, below the 0.5 chance level, with off-diagonal damage 0.212 vs. 0.019 for ClasSAE Hard (Table 6).

Q2. Construction of the post-hoc matrix, class sets and budgets (Table 2)

We appreciate the reviewer's caution and agree our previous explanation was misleading. Appendix C (Eq. 8–9) now gives the exact procedure. In short:

  • The post-hoc matrix is each feature's per-class firing rate, computed on the train (train side) and validation (test side) splits. Per-feature non-integer budgets are derived from it with a threshold; baselines use the same procedure.
  • A feature's claimed set is the top classes of its row (as many as its budget). The prediction set comes from the train-side matrix (trained M for ClasSAE) and the reference set from the test-side matrix, each with its own budget.

Precision and recall are therefore equal only when the two sets have equal size. This holds approximately for post-hoc rows (ClasSAE 0.910 vs. 0.906) but not for trained M, whose learned budgets are not tied to firing rates (0.766 vs. 0.889). Table 2 now reports both variants.

Q3. Definition of Margin (Table 3)

We thank the reviewer for catching this. The caption was wrong, not the results. The Table 3 caption now defines Margin as the mean difference between the true-class score and the highest-scoring incorrect-class score. It is signed, hence negative when the true class is not ranked first (BatchTopK: accuracy 0.002, margin -0.148). No results change.

AuthorsOpenReview ↗

Response to Reviewer JbdW: Summary of Changes and Point-by-Point Answers (2/2)

W2 / Q4. Same intervention rule for all encoders

Yes, the TPP PMI gains persist when the same estimator and rule are applied to every encoder (the "emp." rows of Table A). Appendix E now specifies both rules, and Table 4 reports ClasSAE under both:

  • Common rule (Appendix E, Eq. 10), all architectures: each feature's activation is scaled by its 0.9 quantile and multiplied by its normalized empirical PMI with the target class (counted on the train split), times a strength multiplier (5 for all models). The edit is z' = z × (1 − strength).
  • Native ClasSAE rule (Appendix E, Eq. 11): the strength is the soft selection weight of each feature times the trained matrix column of the target class, times the same multiplier.

At the shared multiplier the target effect is similar (on-diagonal 0.482 for ClasSAE Hard, 0.481 ClassifSAE, 0.484 G-SAE; Table 5), so the advantage in Table A comes from lower collateral damage and smaller edits (L2 of the edit: 2.54 vs. 22.85 and 15.19; Table 7). Edit magnitudes are not matched across architectures, so, as stated in Appendix F, we do not claim superiority at matched strength.

Q5. Underlying forget and retain accuracies (Tables 8-9)

Table 9 (Appendix G) now reports baseline, graded and shuffled forget/retain accuracies for every architecture, and Table 8 is their difference.

Table B: subset of Table 9 (accuracy drops; baseline accuracy is about 0.82 for all models).

ModelRuleForget drop (graded / shuffled)Retain drop (graded / shuffled)
Sup BatchTopKemp.0.348 / 0.8320.103 / 0.789
ClassifSAEemp.0.819 / 0.7930.313 / 0.391
ClasSAE Hardnative0.820 / 0.0040.011 / 0.006
ClasSAE Hardemp.0.814 / 0.5870.007 / 0.015

As the reviewer suspected, Forget vs. Shuffled measures the specificity of the edit, not absolute forgetting: most models forget the target almost completely under the graded edit. The retain drop is more informative: small for ClasSAE, large for ClassifSAE and G-SAE (0.326). Appendix G notes that this benchmark reuses the TPP PMI intervention, so we rely on TPP for cross-architecture claims.

We thank the reviewer again for the constructive feedback.

References

  1. Härle et al. Measuring and guiding monosemanticity. NeurIPS 2025 (Spotlight).
  2. Le Bail et al. Unveiling decision-making in LLMs for text classification: Extraction of influential and interpretable concepts with sparse autoencoders. arXiv:2506.23951, 2025.

The additional experiments address my main concerns about the fairness of the comparisons and the interpretation of the intervention results. The authors clarify their claims and acknowledge the remaining limitations. I have no further questions and maintain my score of 3.

We are glad to hear that your concerns have been addressed. Thank you very much for your constructive feedback and valuable input throughout the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.