acceptodds
Under review as a conference paper at ICLR 2027

NAMED FROM THE START: CONCEPT-ANCHORED SPARSE AUTOENCODERS FOR INTERPRETABILITY OF VISION-LANGUAGE MODELS

Abstract

Sparse autoencoders (SAEs) decompose model activations into interpretable features, but learn those features without semantic identities. Names are assigned only afterward, through inspection or similarity matching, and this step often fails even when the desired concept is present in the learned dictionary. We hypothesize that the problem lies not in feature discovery but in semantic assignment: post-hoc methods must align an already learned dictionary with language, although nothing during training required their geometries to agree. We introduce NAMED-SAE, a concept-anchored SAE that makes semantic identity part of dictionary learning. Text-derived concept embeddings define the named latents, a shared orthogonal adapter aligns them with the activation space, and bounded concept-specific residuals permit adaptation to visual evidence without unrestricted semantic drift. A separately budgeted bank of unnamed latents captures information outside the vocabulary. The adapter is estimated once and frozen; it can be fitted using limited labelled examples or inferred without annotations from unsupervised pseudo-means. Across CLIP and SigLIP, \name provides complete coverage of the target concepts, produces names that are more faithful to activating images and more predictive of intervention effects than post-hoc assignments, and retains most of the reconstruction quality of an unconstrained SAE. These identities also persist under distribution shift without retraining or renaming, while changes in latent activity describe the semantic content of the shift itself. Our results show that semantic identity is more reliably imposed as a constraint during feature learning than recovered through analysis afterward.

Primary area:
interpretability and explainable AI
Keywords:
Sparse autoencoders, mechanistic interpretability, vision–language models, concept-based interpretability, dictionary learning, semantic grounding, concept bottlenecks, representation learning.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LMd8OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Instead of traditionally labelling SAE concepts after training, authors assign labels to the decoder directions before training. This allows to learn specific pre-defined concepts directions while also leaving space for unsupervised concepts learning like in traditional SAEs. They evaluate their method on ImageNet with CLIP and SigLIP models, comparing against unconstrained, pretrained and SpLiCE SAEs.

Critical strengths

I appreciate the honesty of authors about for example the 15 class latents with zero precision and about the sibling classes confusion.

Critical weaknesses
  1. The whole evaluation is centered on ImageNet and a better coverage on ImageNet of the author’s method could be due to this. Authors do not test distribution shifts or transfer to other datasets. Authors should show results on different datasets from ImageNet and also OOD concepts performance.
  2. Authors target only post-projection embedding while normally SAEs are trained also on intermediate layers. So if their goal is also to understand internal computations of models they should also target intermediate layers in their analysis.
  3. Concept-SAE (Ding et al., 2026) and CB-SAE (Kulkarni et al., 2026) are the closest prior work. Can you run and compare them to your method? Right now it's hard to tell how much of the gain over plain SAEs is unique to anchoring.
  4. R squared seems to suffer a lot as compared to unconstrained SAEs (Table 1). As the primary purpose of SAEs is to faithfully reconstruct the activations, this drop is not a good sign. It would be interesting to see if this drop comes from named or unnamed concepts? This could potentially be problematic for generalisation too.
Questions for authors

See Weaknesses

7 comments
AuthorsOpenReview ↗

Response to Reviewer 3 Part 1

We thank the reviewer for the careful assessment and for recognizing our transparent discussion of the method's failure modes. The concerns highlight results we insufficiently emphasized and comparisons that required additional evidence. We therefore clarify the transfer and reconstruction analyses, and add intermediate-layer experiments and matched controls for the mechanisms underlying Concept-SAE and CB-SAE. We address each concern in detail below.


W1: Distribution shift, cross-dataset transfer, and OOD concepts. We thank the reviewer for raising this concern. The submission evaluates all three settings, but compresses the discussion into lines 449–459 and places most evidence in Appendix E.

Distribution shift. We evaluate ten ImageNet-derived shifts without retraining or renaming: ImageNet-V2, ImageNet-Sketch, ImageNet-R, ImageNet-A, and three ImageNet-C corruptions at severities 2 and 4. Table 11 reports the complete results.

NAMED-SAE outperforms post-hoc naming on all ten shifts. Precision@16 remains 0.670–0.979 even as R2R^2 falls to 0.507–0.674, indicating that reconstruction degrades before latent identity. This ordering is stable across three seeds, with top-1 varying by at most 0.007 and R2R^2 by at most 0.008, and is reproduced with SigLIP.

The code also identifies the shift itself. The latents illustration and monochrome fire on 79% and 42% of ImageNet-Sketch, while fog and smoke fire on 89% and 55% under fog corruption. All four activate on less than 0.1% of clean ImageNet. In contrast, on four of six examined shifts, the most frequent post-hoc latent receives a semantically unsupported name.

Cross-dataset transfer. Transfer to a different label space is more difficult. On CIFAR-100, NAMED-SAE reaches class-latent accuracy of 0.485, compared with 0.403 for post-hoc naming, while precision@16 remains 0.942. A 32×32 ImageNet control shows that the decline is not explained by resolution alone. Instead, the chosen vocabulary and 20-sparse code do not capture every distinction required by the new label space.

OOD concept evaluation. We evaluate 43 vocabulary concepts on iNaturalist taxa. Specific order- and family-level concepts achieve mean AUROC 0.833, including cactus (0.994), moss (0.996), and owl (0.950). Broad kingdom- and class-level concepts average 0.514.

This failure appears to arise from sparse competition with more specific atoms. Aggregating nearby anchors at readout raises AUROC from 0.51 to 0.98 for bird, 0.50 to 0.98 for insect, and 0.55 to 0.99 for fish. Because this readout applies to any named representation, we present it as evidence that the hypernym information survives transfer, not as an advantage over post-hoc naming. It also motivates a hierarchical NAMED-SAE in which activation from specific concepts propagates to their hypernyms.

Thus, the learned identities remain stable across substantial distribution shifts and transfer to specific concepts on a different dataset. Transfer to a new label space and broad hypernyms remains limited. We will make both the positive results and these limitations more prominent in the main text.

W2: Intermediate-layer activations. Applying NAMED-SAE inside the network requires aligning the 768-dimensional text space with an activation space of different dimension. We therefore trained it on the 1,024-dimensional layer-22 post-MLP residual stream of CLIP ViT-L/14-336 and evaluated on the full ImageNet validation set, with the text anchors entering through a learned semi-orthogonal embedding. Causal removal is unavailable because our saved layer-22 representation holds only the CLS token and cannot be propagated through the remaining vision layers.

ModelR2R^2DeadActiveP@16Top-1
NAMED-SAE, output embedding0.71415%9940.8630.717
NAMED-SAE, layer 220.70235%9890.8200.626
NAMED-SAE, layer 22, second seed0.702†35%9900.8150.622

†Second-seed R2R^2 is from the training held-out split. The layer-22 model uses 2,000 free atoms and r=0.5r=0.5, against 500 and r=0.3r=0.3 at the output, so R2R^2 is not comparable across layers.

At layer 22, precision@16 remains high at 0.820 against 0.863 at the output, while class top-1 falls from 0.717 to 0.626. Of the 1,000 class slots, 989 remain active against 994, though the dead fraction over the full vocabulary rises from 15% to 35%. Moving inward therefore affects vocabulary utilization and classification more than the semantic precision of active class latents. The second seed agrees within 0.005 precision@16 and 0.004 top-1.

Thus NAMED-SAE extends to an intermediate representation: nearly all class slots stay active and their semantic precision stays high. This establishes feasibility inside the network, though not a complete layerwise or spatial-token analysis. The full layer-22 ladder and alignment ablations will be added to Appendix B.

AuthorsOpenReview ↗

Response to Reviewer 3 Part 2

W3: Comparison with Concept-SAE and CB-SAE. We agree that citing these methods without an empirical comparison leaves the distinction unclear. The central difference is when and how a latent acquires its semantic identity.

NAMED-SAEConcept-SAECB-SAE
Identity assignedbefore trainingduring trainingafter training
Dictionaryanchored, adaptivesupervisedpretrained, pruned
Supervisiontext anchorsimage scores and masksconcept set
Per-image concept targetsnoyesno

NAMED-SAE fixes identities from text before dictionary learning. Concept-SAE supervises designated concepts during training using per-image concept scores and localization masks. CB-SAE modifies and names a pretrained SAE after its dictionary has formed.

CB-SAE. CB-SAE has no released implementation, so we cannot reproduce the complete method. We instead evaluate its central post-hoc mechanism using the unconstrained-SAE control in Table 1. The backbone, training data, vocabulary, and alignment are the same as for NAMED-SAE; the principal difference is whether identities are assigned after dictionary learning or constrained during it.

Post-hoc assignment reaches 0.559 top-1, 0.461 precision@16, and 0.36 causal removal, compared with 0.713, 0.853, and 0.89 for NAMED-SAE, despite reconstructing better (R2=0.807R^2=0.807 versus 0.710). This control does not reproduce CB-SAE's pruning stage, but directly tests the part relevant to our claim: applying the same semantic alignment after training does not reproduce the precision or causal control obtained by anchoring identities during training.

Concept-SAE. Concept-SAE cannot be directly ported because it operates on spatial token maps using per-image concept scores and CLIPSeg masks, whereas our representation is a pooled embedding. We therefore construct a pooled Concept-SAE-style control that isolates the source of the named directions.

For each of the 1,000 ImageNet classes, the Concept-SAE-style control estimates a supervised class-presence direction and freezes it as a named decoder atom. The matched text-anchor control instead freezes each atom to the aligned text embedding of its class name. Both use 1,000 named atoms, 500 free atoms, and the same encoder, sparsity, training data, reconstruction objective, and freezing. Only the source of the named directions differs.

Named directionsFreeR2R^2Class hitTop-1Named share
Supervised class presence5000.5940.9440.7570.274
Aligned text anchors5000.6380.9600.7320.578

Named share is the fraction of the combined reconstruction magnitude carried by named atoms.

Text anchors improve R2R^2 by 0.044, class hit by 0.016, and named share from 0.274 to 0.578. Supervised presence directions improve top-1 by 0.025, as expected because they are estimated specifically for class discrimination.

For this controlled comparison, both arms receive the full ImageNet labels, but use them differently. The supervised control estimates a separate direction for every class, whereas the text-anchor control uses the labels only to estimate one shared alignment. Supervised directions also require labelled examples for every named concept, limiting this control to the 1,000 classes. Text anchors require only a concept name and extend to all 9,866 vocabulary concepts, including 8,866 without ImageNet labels. The standard NAMED-SAE configuration estimates its shared alignment from only ten images per class.

Thus, the post-hoc control provides the direct evidence that the gain over plain SAEs comes from anchoring identities during dictionary learning: applying the same alignment afterward does not recover NAMED-SAE's precision or causal control. The Concept-SAE-style control addresses a complementary question, showing that text anchors preserve more reconstruction and named-bank utilization than class-specific supervised directions, while the latter provide moderately higher class discrimination.

AuthorsOpenReview ↗

Response to Reviewer 3 Part 3

W4: Whether the reconstruction cost falls on named or unnamed concepts. We decompose reconstruction by decoding with only the named bank or only the free bank. Following Appendix C, variant B fixes each named decoder row to its text anchor, whereas variant C adds a bounded residual that allows controlled adaptation. Our reported configuration is variant C with r=0.3r=0.3.

Every model has the same 9,866 named slots and permits 16 active named atoms per image. We vary the named residual, free-bank width, and free-bank sparsity budget. The first two rows isolate the residual at fixed free capacity; the remaining rows show how additional free capacity changes the allocation of reconstruction. All models are evaluated on the full ImageNet validation set.

Named bankFreeSparsityTotal R2R^2Named R2R^2Free R2R^2Named share
No residual (B)50016+40.6470.3570.2280.554
Bounded residual (C)50016+40.7140.4780.1770.611
Bounded residual (C)2,00016+80.7620.3620.3940.494
No residual (B)4,00016+160.7960.2290.5730.386

Sparsity gives the named and free per-image budgets. Named share is the fraction of combined reconstruction magnitude carried by named atoms. Per-bank R2R^2 is measured by decoding with that bank alone and is not additive.

The exact-width control in our response to Reviewer 1, Q3 establishes that the reconstruction gap is the genuine cost of anchoring. At the same width (10,366 atoms) and sparsity (16+4), the unconstrained SAE reaches R2=0.8052R^2=0.8052, compared with 0.7135 for NAMED-SAE. Anchoring therefore costs 0.0917 in R2R^2 at fixed capacity, close to the 0.097 gap in Table 1.

The bank decomposition clarifies where reconstruction is carried. At fixed free capacity and sparsity, adding the bounded residual raises named-bank R2R^2 from 0.357 to 0.478, named share from 0.554 to 0.611, and total R2R^2 from 0.647 to 0.714. The residual therefore recovers reconstruction within the named bank by adapting its concept directions to the activation distribution.

Additional free capacity recovers further reconstruction by moving information into unnamed features. Increasing the bounded model's free bank from 500 to 2,000 atoms raises total R2R^2 to 0.762, while named-bank R2R^2 falls to 0.362 and free-bank R2R^2 rises to 0.394. Similarly, the no-residual model reaches R2=0.796R^2=0.796 with 4,000 free atoms, but its named share falls to 0.386. This larger model is not capacity-matched to the baseline; it shows that additional width and support can buy back reconstruction by delegating more information to the free bank. Thus, the free bank does not cause the reconstruction deficit: it compensates for the constraint imposed on the named bank.

Generalization. The transfer results in W1 show that reduced reconstruction does not necessarily destroy semantic identity. Across ten ImageNet-derived shifts, R2R^2 falls to 0.507–0.674, while precision@16 remains 0.670–0.979 and NAMED-SAE continues to outperform post-hoc naming. However, transfer to a different label space is limited: on CIFAR-100, class-latent accuracy falls to 0.485, although precision@16 remains 0.942. We therefore conclude that anchored identities remain stable under the shifts tested, not that reconstruction loss is irrelevant to every form of generalization.

Thus, the reconstruction deficit is the genuine cost of constraining the named bank at fixed capacity. Bounded residuals recover part of that cost within the named bank, while additional free capacity recovers more reconstruction by moving information into unnamed features. The identities remain stable across the evaluated shifts, although transfer to substantially different label spaces remains limited.

Reviewer_M4DXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a concept-anchored sparse autoencoder for improving the interpretability of vision-language models by assigning semantic identities to latent features during training. The method employs text-derived concept embeddings, orthogonal alignment, and bounded residuals to constrain named decoder features, while incorporating unconstrained features to preserve reconstruction capability. Experiments on CLIP and SigLIP demonstrate improvements in concept coverage, naming accuracy, and causal intervention metrics over conventional SAEs and post-hoc naming methods, at the cost of reduced reconstruction quality.

Critical strengths
  1. The paper addresses a relevant limitation of conventional SAEs by incorporating predefined semantic identities directly into feature learning rather than relying on post-hoc naming.
  2. The combination of concept anchoring, bounded residuals, and unconstrained features provides a straightforward approach to balancing semantic interpretability and reconstruction quality.
  3. The authors conduct relatively comprehensive experiments on CLIP and SigLIP, including comparisons with existing baselines, ablation studies, and distribution shift evaluations, demonstrating improvements in several interpretability-related metrics.
Critical weaknesses
  1. The experimental evaluation lacks comparisons with closely related concept-aware SAE methods, such as Concept-SAE and interpretable concept-bottleneck SAEs. Comparing NAMED-SAE with these methods in terms of concept precision, causal removal, and reconstruction quality, while accounting for differences in supervision requirements, would provide stronger evidence of its effectiveness and advantages over existing approaches.
  2. Given that the label probe reveals highly class-selective features in unconstrained SAEs, could the authors compare against supervised concept-to-latent assignment methods under matched annotation budgets? This would help clarify whether the improvements primarily arise from concept anchoring during training or from more effective semantic assignment.
  3. The causal removal experiment primarily measures the influence of named latents on classification predictions, which does not necessarily establish their semantic faithfulness. A latent may influence the prediction of a target class while responding to correlated contextual features rather than the intended concept itself. Could the authors additionally compare interventions on semantically related or contextually correlated latents (e.g., dog versus wolf, fur, or grass), while controlling for intervention strength?
  4. The proposed method incurs a noticeable reconstruction cost, reducing R² from 0.807 to 0.710 compared with the matched unconstrained SAE. However, it remains unclear whether the improvements in interpretability justify this loss of reconstruction quality. Could the authors compare NAMED-SAE with unconstrained SAEs at comparable reconstruction quality, and examine whether the proposed method still achieves better concept precision and causal faithfulness under similar R² values?
Questions for authors

Please refer to my concerns raised in the weaknesses section.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 2 Part 1

We thank the reviewer for the thoughtful assessment and for recognizing the paper’s central strengths: addressing the limitations of post-hoc SAE naming, combining concept anchoring, bounded adaptation, and free features to balance semantic identity with reconstruction, and providing extensive evaluation across CLIP, SigLIP, ablations, baselines, and distribution shifts. We address the remaining concerns below with matched controls and additional experiments.

W1: Comparison with concept-aware SAE methods.

Concept-bottleneck SAEs. At the time of rebuttal, CB-SAE has no released implementation. Table 1 nevertheless includes two baselines that bracket its central design choices. The unconstrained SAE followed by concept assignment represents post-hoc naming: the dictionary is learned before identities are assigned. SpLiCE represents the fixed-dictionary alternative: every coordinate is named by construction, but the dictionary cannot adapt to the activation distribution. NAMED-SAE fixes identities before training while permitting bounded adaptation.

Using the same vocabulary and alignment, post-hoc assignment reconstructs better than NAMED-SAE (R2=0.807R^2=0.807 versus 0.714) but has lower coverage (970 versus 1,000), precision@16 (0.461 versus 0.863), and causal removal (0.36 versus 0.90). SpLiCE guarantees full coverage but reaches only R2=0.237R^2=0.237. Although the post-hoc control does not reproduce CB-SAE's pruning stage, it isolates the distinction central to our claim: assigning identities after dictionary learning versus constraining them during training.

Concept-SAE. Concept-SAE cannot be directly ported to our setting because it learns spatial concept features using per-image concept scores and CLIPSeg masks, whereas we model one pooled embedding per image. We therefore construct a pooled control that isolates the relevant difference: whether named decoder atoms are obtained from supervised concept presence or text embeddings.

For each of the 1,000 ImageNet classes, the Concept-SAE-style control estimates a direction that discriminates that class from the remaining images and freezes it as a named decoder atom. The text-anchor control instead freezes each atom to the aligned text embedding of its class name. Both controls use 1,000 named atoms, 500 free atoms, and the same encoder, sparsity, training data, and reconstruction objective. Only the source of the named directions differs.

ControlNamed directionsFreeR2R^2 ↑Class hit ↑Top-1 ↑
Concept-SAE-stylesupervised class presence5000.5940.9440.757
NAMED-SAE (ours)aligned text anchors5000.6380.9600.732

Text anchors improve R2R^2 by 0.044 and class hit by 0.016. Supervised presence directions improve top-1 by 0.025, as expected because they are fitted specifically for class discrimination. However, they require labelled examples for every concept and therefore limit this control to the 1,000 ImageNet classes. Text anchors require only a concept name and extend the dictionary to all 9,866 vocabulary concepts, including 8,866 without ImageNet labels.

For this controlled comparison, both models receive the full ImageNet labels. The supervised control uses them to estimate 1,000 class-specific directions, whereas the text-anchor control uses them only to estimate one shared alignment applied to every concept. The standard NAMED-SAE configuration estimates this alignment from ten images per class. In its original spatial setting, Concept-SAE additionally requires per-image concept scores and localization masks.

Thus, NAMED-SAE differs from both approaches in when and how identity is imposed. CB-SAE assigns concepts after an unconstrained dictionary has been learned, whereas NAMED-SAE fixes identities before training and preserves them through bounded adaptation. Concept-SAE supervises identities during training using per-image concept scores and spatial masks, whereas NAMED-SAE derives them from text anchors and one shared alignment, allowing it to scale to concepts without labelled examples. Our controls reflect these distinctions: post-hoc assignment does not recover NAMED-SAE's precision or causal control, while class-specific supervised directions provide slightly higher discrimination but lower reconstruction and class-slot utilization than text anchors. We are adding these comparisons and distinctions to the revision.

AuthorsOpenReview ↗

Response to Reviewer 2 Part 2

W2: Supervised concept-to-latent assignment under a matched annotation budget. We agree this comparison is needed to separate anchoring during training from better semantic assignment. The label probe motivates the question but is not an assignment method: it uses validation labels to pick the most class-selective latent per class. Its precision of 0.87–0.89 shows unconstrained dictionaries often contain suitable class features even when post-hoc naming fails to find them. Because it needs labels for every queried concept and names nothing beyond the selected classes, we treat it as an oracle for dictionary content.

Our supervised assignment baseline is the + our alignment (proc) row of Table 1. We train an unconstrained SAE, fit the orthogonal alignment on all 1.28M labelled training images, and assign each decoder atom its nearest vocabulary concept under that alignment. The matched NAMED-SAE comparison uses the same backbone, vocabulary, labelled images, and fitted alignment. The controlled difference is when identity enters: after dictionary learning, or during it.

MethodLabelsR2R^2CoverageP@16HitRemoval
Label-probe oracleval——0.87–0.89——
Supervised post-hoc assignment1.28M0.8079700.4610.8580.36
NAMED-SAE, anchored during training1.28M0.7141,0000.8630.8810.90

At the same annotation budget, NAMED-SAE raises precision@16 from 0.461 to 0.863 and causal removal from 0.36 to 0.90, and covers all 1,000 classes rather than 970. Since both use the same alignment, vocabulary, and backbone, the gap cannot be attributed to annotation volume or alignment quality; it arises from constraining identities during dictionary learning rather than matching concepts to a learned dictionary.

The label-budget ladder adds a separate robustness result. NAMED-SAE retains precision@16/removal of 0.853/0.89 with ten images per class, 0.785/0.80 with one, and 0.836/0.88 with none. These are not matched comparisons at each budget; they show the anchoring result does not depend on the full-label alignment.

Thus, under an exactly matched 1.28M-image budget, anchoring substantially outperforms supervised post-hoc assignment using the same semantic alignment, and the budget ladder shows the advantage persists when that supervision is greatly reduced or removed.


W3: Whether causal removal establishes semantic faithfulness. We agree that changing a prediction does not by itself establish semantic faithfulness. Precision@16 tests the activation side—whether a latent's top images contain its named class—but need not separate the class from correlated context. We therefore add the matched-strength intervention test the reviewer requested, which probes the decoder side.

We add tt to one named latent, decode, and measure the effect on every other named concept; because decoder rows have unit norm, the perturbation has matched strength, ∥Δx^∥2=t\|\Delta\hat{x}\|_2=t. Re-encoding shows which other latents become active. Over 3,662 named slots and 2,048 images, we bin affected concepts by text-anchor similarity to the intervened concept.

The effect grows with similarity but stays small: at unit strength the highest-similarity bin moves by about 2% of the perturbation and under 0.6% of latents become newly active, scaling linearly in tt.

The reviewer's dog/grass case illustrates this. Intervening on dog affects its taxonomic neighborhood almost uniformly—otterhound (+0.0217+0.0217), canine (+0.0216+0.0216), Tibetan terrier (+0.0214+0.0214), pup, hound, dalmatian, puppy, dogs and further breeds, all within 0.0020.002—while grass moves only +0.0006+0.0006. Intervening on wheel affects car wheel (+0.0246+0.0246), tire (+0.0152+0.0152) and rim (+0.0110+0.0110), while road and street change by +0.0006+0.0006 and −0.0030-0.0030.

Across the eighty most-affected pairs for eight probed concepts the lowest text similarity is 0.315; most are synonyms, morphological variants or taxonomic neighbors, some close associations (fur with mink and Arctic fox, snow with winter). The reviewer's contextual correlates sit far lower—grass at 0.060 from dog, road and street at 0.088 and −0.029-0.029 from wheel—and none is positively affected by more than +0.0006+0.0006.

Most coupling follows the text vocabulary's geometry: a no-residual model, whose decoder rows equal the anchors exactly, stays within 0.004 of NAMED-SAE in every similarity bin. Near-synonyms are coupled at the granularity of the anchor space, not because a named direction drifted toward context.

Thus precision@16 gives activation-side evidence and this test gives decoder-side evidence: manipulating a named latent affects that concept and its semantic neighborhood, with little effect on contextual correlates. We will present this as evidence supporting semantic faithfulness rather than proof that causal removal alone establishes it.

AuthorsOpenReview ↗

Response to Reviewer 2 Part 3

W4: Whether the interpretability gain justifies the reconstruction cost. We first isolate the reconstruction cost of anchoring at fixed capacity, then match reconstruction in both directions.

Anchoring has a genuine reconstruction cost at fixed capacity. An unconstrained control with the same 10,366 atoms, sparsity allocation (16+4), encoder, optimizer, schedule, and training steps reaches R2=0.8052R^2=0.8052, against 0.7135 for NAMED-SAE. Anchoring therefore costs 0.0917 in R2R^2 at equal width and sparsity, close to the 0.097 gap the review cites (0.807 to 0.710), so the small width difference in the submitted comparison does not explain the result.

We first raise NAMED-SAE to the baselines' reconstruction. Enlarging the free bank to 4,000 atoms brings NAMED-SAE to R2=0.8052R^2=0.8052, comparable to the unconstrained and pretrained SAEs. This comparison matches reconstruction, not capacity: it has 13,866 atoms and total sparsity K=32K=32 (16 named and 16 free), against 6,144–12,288 atoms and K≈20K\approx20 for the baselines. The pretrained rows are Pach et al.'s released BatchTopK dictionaries for the same CLIP backbone at 8× and 16× expansion, evaluated unchanged; ×16 measures L0=20.4L_0=20.4 because BatchTopK spends its budget per batch, not per image. The extra support is necessary: with the same 4,000-atom free bank but K=20K=20 the model reaches only R2=0.7365R^2=0.7365, with 53% of its free atoms dead.

ModelWidthKKR2R^2Cov. ↑P@16 ↑Removal ↑
Unconstrained SAE9,866200.8077820.5890.36
↳ + our alignment (proc)9,866200.8079700.4610.36
Unconstrained, matched architecture10,366200.8057850.5680.43
Pretrained ×16\times1612,28820.40.8058200.4940.31
Pretrained ×8\times86,144200.8017560.6300.37
NAMED-SAE, 4,000 free13,866320.8051,0000.7380.56
NAMED-SAE, 500 free (full-data alignment)10,366200.7141,0000.8630.90

At matched reconstruction, NAMED-SAE reaches precision@16 of 0.738, compared with 0.461–0.630 across the post-hoc baselines, and causal removal of 0.56, compared with 0.31–0.43. ×8 is the strongest post-hoc baseline on precision@16 despite being the narrowest, so the gap is not an artifact of dictionary size. Coverage remains complete, against 756–970 classes for post-hoc naming. An unrelated-slot intervention preserves the prediction in 0.9998 of cases, confirming removal remains specific. This gain narrows the semantic advantage and cuts class top-1 from 0.717 (500 free) to 0.547 (4,000 free). The reported configuration retains that advantage; the matched model trades it for R2R^2.

Matching reconstruction in the other direction gives the same conclusion. Reducing the unconstrained SAE to K=7K=7 brings it to R2=0.709R^2=0.709, close to NAMED-SAE at 0.714. At this sparsity, however, 56.5% of its dictionary is dead, post-hoc naming covers only 664 classes, and precision@16 reaches 0.746 over that subset, against 0.863 over all 1,000 classes for NAMED-SAE. Causal removal reaches 0.70, compared with 0.90 for NAMED-SAE, with unrelated-slot controls preserving the prediction in 0.9902 and 1.0000 of cases; W3 above adds the specificity evidence removal alone cannot give. A supervised probe reaches 0.862, showing that the sparse dictionary still contains class-selective latents when labels are used to locate them. The gap is again in assigning stable identities, not discovering selective directions.

The sparsified baseline achieves a higher positive-steering hit rate (0.893 versus 0.647), but the same intervention changes the highest-scoring non-target class in 81.4% of cases, compared with 33.1% for NAMED-SAE. Its stronger insertion therefore behaves largely as a generic perturbation rather than selective control.

The trade-off is measurable. Along the same capacity ladder, total sparsity increases from K=20K=20 to 24 to 32 as the free bank grows from 500 to 2,000 to 4,000 atoms. Precision@16 falls from 0.863 to 0.811 to 0.738. With 8,000 free atoms, R2R^2 reaches 0.813, class top-1 is 0.555, and the number of nameable concepts falls by 34%, from 1,321 to 866. Here, a concept is nameable when its latent activates on at least sixteen images and its top-16 images cohere above a permutation null.

Thus, anchoring costs about 0.09 R2R^2 at fixed width and sparsity. Matching reconstruction in either direction preserves NAMED-SAE's precision, removal, and coverage advantages: the sparsified SAE has a mostly dead dictionary, covers only 664 classes, and produces less selective interventions, while added free capacity raises NAMED-SAE to R2=0.805R^2=0.805 and remains stronger than every post-hoc baseline on all three metrics. This recovery costs class top-1 and narrows the semantic margin.

Reviewer_DEXPOpenReview ↗
Rating
2(+1)
Confidence
3(+1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NAMED-SAE, an SAE training method that anchors a subset of decoder neurons to a defined text vocabulary before training, then trains SAE with a constraint to preserve the anchored (named) decoder neurons. Trained on VLM representations, NAMED-SAE achieves the highest top-1 accuracy among SAE baselines and SpLiCE.

Critical strengths

(S1) A new initialization technique for SAE using an orthogonal map AA addresses the difficulty of post-hoc labeling of SAE neurons and the issue of SAE neurons being often non-monosemantic. In Cross-modal alignment (L. 187), the authors ablate how AA is derived (with supporting results in Table 1) and how training is constrained to preserve the named decoder neurons.

(S2) The abstract, introduction, and related work sections are well-written and richly referenced, presenting an easy-to-understand motivation grounded in the interpretability problem of SAEs.

Critical weaknesses

(W1) the 4.1 Results and Discussion is hard to follow. The section reads as a wall of text. Pages 6–7 are dense with text and numbers containing a single table, and pages 8–9 a single figure. Most paragraphs point back to Table 1 or the appendix, often repeating values already shown there. I suggest reviewers:

  • split the section into subsections, with an ablation subsection included.
  • make each paragraph self-contained, with explicit references to figures and tables in the main text, rather than the appendix.
  • replace number-heavy wording with a concise discussion of what the results show and how the reader should interpret them.

(W2) NAMED-SAE, with the evaluation performed in the paper, reads as a method closer to a concept bottleneck than to an SAE. Only 500 of 10,366 (Appendix, Table 2) neurons are free for the model to learn, and the rest are nearly fixed. The architecture therefore resembles CBM with a residual (e.g., PCBM-h [1], Res-CBM [2]) or a SpLiCE decomposition with a residual. The evaluation reflects this resemblance. Table 1 reports mainly classification performance with only R2R^2 from SAE metrics (without even evaluating steering), and on R2R^2 NAMED-SAE is substantially worse than the SAE baselines.

Minor Weaknesses

  • (W3) Several relevant works are missing or are under-discussed. G-SAE [5] also anchors neurons during training, so it should be compared against or at least discussed. Related Work should also include an early work that trains SAEs on CLIP [3].
  • (W4) The metrics from G-SAE [5] and SAE-Bench [4] should be adopted or at least discussed.
  • (W5) The paper lacks an overview figure of the NAMED-SAE architecture and training procedure. The residual Δi\Delta_i (L. 165) is hard to understand, and the 28-step pseudo-code in the appendix A.1 does not clarify it.
  • (W6) The post-hoc naming techniques should be used from prior works in Baselines (L. 240). The custom hai resembles [6], and the probe naming resembles [7]. These connections should be stated explicitly and credited.
  • (W7) FF (L. 225) is used before it is defined.
  • (W8) ImageNet-R (L. 453) is never cited in the main text, although it appears in the bibliography.

References

[1] Yuksekgonul, Mert, Maggie Wang, and James Zou. "Post-hoc concept bottleneck models." arXiv preprint arXiv:2205.15480 (2022). (ICLR 2023)

[2] Shang, Chenming, et al. "Incremental residual concept bottleneck models." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

[3] Zaigrajew, Vladimir, Hubert Baniecki, and Przemysław Biecek. "Interpreting CLIP with hierarchical sparse autoencoders." arXiv preprint arXiv:2502.20578 (2025). (ICML 2024)

[4] Karvonen, Adam, et al. "SAEBench: A comprehensive benchmark for sparse autoencoders in language model interpretability." arXiv preprint arXiv:2503.09532 (2025). (ICML 2024)

[5] Härle, Ruben, et al. "Measuring and guiding monosemanticity." Advances in Neural Information Processing Systems 38 (2026): 167468-167508.

[6] Dreyer, Maximilian, et al. "From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance." arXiv preprint arXiv:2505.20229 (2025).

[7] Rao, Sukrut, et al. "Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

Questions for authors

(Q1) Following up on (W2): can the authors justify why NAMED-SAE should still be considered an SAE variant? The method is optimized toward, and evaluated as, a CBM or SpLiCE. Its reconstruction quality is also notably lower (R2≈0.7R^2 \approx 0.7 vs. ≈0.8\approx 0.8 for the SAE baselines, Table 1).

(Q2) Why is the model trained with the fraction of variance unexplained in equation 7 (L. 203) as the loss instead of the MSE loss that is standard for SAEs?

(Q3) Do the existing baselines match NAMED-SAE in number of neurons?

9 comments
AuthorsOpenReview ↗

Response to Reviewer 1 Part 1

We thank the reviewer for the careful and specific assessment. We appreciate that the central contribution came through clearly: anchoring named decoder atoms through a shared orthogonal alignment addresses the limitations of post-hoc naming while promoting stable semantic identities, with the alignment ablations supporting this claim. We address each concern below and would be happy to clarify any remaining questions.


W1, W7, W8 We will address this and the reviewer's remaining presentation comments and upload a revised version within the next two days, in time for the reviewer to inspect it during the discussion period.


W5: Architecture and implementation. We release the complete implementation at \url{https://anonymous.4open.science/r/NAMED-SAE-C680/}, including training, evaluation, intervention, and table-reproduction code. The revision will also add an architecture and training overview figure clarifying the bounded residual.


W2: Whether NAMED-SAE is closer to a concept bottleneck than to an SAE. We agree that the large anchored bank gives NAMED-SAE a concept-bottleneck-like interface. However, its objective, architecture, and inference procedure remain those of an SAE.

The named decoder rows are deliberately constrained. The named decoder rows are deliberately constrained. Of 10366 atoms, 9866 are anchored and 500 are free. With r=0.3r=0.3, each named row can move at most 17.5∘17.5^\circ from its anchor, and the mean residual norm uses 31% of its permitted budget. This controlled adaptation fits the activation distribution while preserving each direction's semantic identity.

The sparse encoder remains learned from reconstruction. The encoder Wenc∈R768×10,366W_{\mathrm{enc}}\in\mathbb{R}^{768\times10{,}366} contains 7.967.96M unconstrained parameters and determines when each latent fires. It is trained through sparse reconstruction and receives no concept or task labels during SAE training. Labels are used only before training to estimate one shared alignment; the label-free variant removes even that step.

Three properties distinguish NAMED-SAE from PCBM-h [1] and Res-CBM [2]. First, NAMED-SAE has no downstream classifier and is never optimized for a prediction task. Its class top-1 metric is read directly from the sparse code as a diagnostic, whereas concept-bottleneck methods fit a task predictor on concept activations. Second, NAMED-SAE activates only 20 of 10366 coordinates per image. Third, it learns an overcomplete reconstructive dictionary rather than a task-specific concept representation. SpLiCE is the closest fixed-dictionary comparison, but it has no learned encoder, optimizes a new code for each image, and reaches R2=0.237R^2=0.237 versus 0.7100.710 for NAMED-SAE.

Positive steering. The submission reports removal but not positive steering. We therefore clamp a class latent to α\alpha times its mean active value, decode, and apply the VLM's zero-shot head. Hit is the fraction of interventions making the target class top-1 (↑\uparrow), while collateral is the fraction changing the highest-scoring non-target class (↓\downarrow). ZS top-1 measures behavioral preservation before intervention and differs from the class-latent top-1 in Table~1. Parentheses give dictionary width.

ModelPoolZS top-1α=2\alpha=2: hit / coll.α=4\alpha=4: hit / coll.
Unconstrained SAE (9866), post hoc7820.7170.082 / 0.1200.416 / 0.341
Unconstrained SAE (10366), width-matched, post hoc7680.7170.138 / 0.1410.506 / 0.406
Pretrained ×16\times16 (12288), post hoc8190.7130.081 / 0.1230.384 / 0.322
SpLiCE (9866)1,0000.6780.163 / 0.1370.664 / 0.398
NAMED-SAE (10366)1,0000.7310.097 / 0.1060.647 / 0.331

Each method is evaluated on 200200 classes sampled from those it covers, so the results should be read together with coverage. At α=2\alpha=2, the methods are not clearly separated: NAMED-SAE has lower hit than SpLiCE and the width-matched SAE, but the lowest collateral. At α=4\alpha=4, NAMED-SAE reaches 0.6470.647 hit, compared with 0.3840.384--0.5060.506 for post-hoc SAEs, with lower collateral than two of three. SpLiCE reaches 0.6640.664 hit but has higher collateral and substantially lower reconstruction. \emph{Thus, NAMED-SAE provides stronger target control than post-hoc SAEs at sufficient intervention strength.}

At matched reconstruction (R2≈0.805R^2\approx0.805), NAMED-SAE retains higher precision@16 (0.7380.738 versus 0.4610.461--0.6300.630) and causal removal (0.560.56 versus 0.310.31--0.430.43), although it uses additional width and sparsity. Reviewer~2, W4 gives the full comparison, while W4 below reports the applicable SAE-Bench metrics.

Thus, NAMED-SAE provides a concept-addressable interface while remaining an overcomplete sparse autoencoder trained through activation reconstruction, without a task head or prediction objective.

AuthorsOpenReview ↗

Response to Reviewer 1 Part 2

W3: Related work and positioning. We thank the reviewer for these helpful pointers. G-SAE [5] is particularly relevant because it also introduces semantic guidance during SAE training, while Zaigrajew et al. [3] provides important early work on hierarchical SAEs for CLIP. We will expand Section~2 to clarify how NAMED-SAE relates to both.

G-SAE (H"{a}rle et al. [5]). G-SAE uses labelled concept examples to guide where and how specified concepts are represented in the SAE code, and introduces the Feature Monosemanticity Score (FMS) to evaluate their localization and disentanglement. Like NAMED-SAE, it acts on semantic identity during dictionary learning rather than assigning names afterward.

The main difference is the source and scale of identity. G-SAE requires labelled examples for every guided concept and is evaluated on compact concept sets such as toxicity, writing style, and privacy attributes in language models. NAMED-SAE obtains each identity from its text embedding, allowing a bank of 9,8669{,}866 concepts, including 8,8668{,}866 without ImageNet labels. Labels in our main configuration estimate only one shared cross-modal alignment rather than supervising each concept independently.

The Concept-SAE-style control in our response to Reviewer~2, W1 provides related evidence about this distinction: at matched width and capacity, supervised class-presence directions improve class discrimination slightly, while text anchors preserve substantially more reconstruction. We do not present this as a direct reproduction of G-SAE, whose guidance objective differs. The revision will discuss G-SAE explicitly and include FMS in the expanded SAE-oriented evaluation described under W4.

Hierarchical SAEs on CLIP (Zaigrajew et al. [3]). Zaigrajew et al. provides an important early application of hierarchical SAEs to CLIP. Our baselines include later pretrained BatchTopK and Matryoshka VLM SAEs, and we cite the Matryoshka architecture through Bussmann et al. We will additionally cite and discuss the earlier CLIP-specific study to make this lineage explicit.

These additions clarify the paper's position: G-SAE establishes semantic identity from concept-specific labels, whereas NAMED-SAE derives identities from text anchors at vocabulary scale; Zaigrajew et al. establishes important prior work on hierarchical SAEs for CLIP.


W6: Attribution of post-hoc naming baselines. We thank the reviewer for these pointers. The revision will make the provenance of each baseline explicit.

BaselinePrior workRelation
decRao et al. (2024) [7]Nearest text embedding to each decoder row
procRao et al. (2024) [7]Same rule after our frozen alignment
haiDreyer et al. (2025) [6]Label inferred from the top-activating images
label probeSupervised oracleBest latent per class selected using labels

The dec baseline uses the naming rule of Discover-then-Name [7], while proc applies it after our frozen alignment. The hai baseline follows the image-grounded labelling approach of Dreyer et al. [6], without adopting their attribution-patching framework. The label probe is not a naming method: it uses evaluation labels to measure whether a class-selective latent exists in the dictionary.

Q3: Do the baselines match NAMED-SAE in number of neurons? No, not exactly. NAMED-SAE has 10366 atoms: 9866 named atoms, one for each vocabulary concept, and 500 free atoms. The unconstrained SAE and SpLiCE use 9866 atoms to match the named-bank width, while the pretrained SAEs use their released widths. All methods are matched at approximately 2020 active atoms per image.

ModelAtomsMean active (L0L_0)
Unconstrained SAE (Table 1)9,86620.0
Pretrained ×8\times 86,14420.0
Pretrained ×16\times 1612,28820.4
SpLiCE dictionary9,86620.0
NAMED-SAE10,36620.0
Unconstrained, width-matched (new)10,36620.0

To test whether the width difference affects the comparison, we trained an unconstrained control with the same 10366 atoms, 16+4 sparsity allocation, optimizer, schedule, and training steps as NAMED-SAE. It reaches R2=0.8052R^2=0.8052, essentially identical to the 9866-atom unconstrained SAE at 0.80620.8062. The additional 500 atoms therefore have negligible effect on reconstruction. The corresponding NAMED-SAE reaches R2=0.7135R^2=0.7135, so anchoring costs 0.0917 in R2R^2 at equal width and sparsity.

All values in this comparison were recomputed on the same 50000 validation images. The small difference from above Table (0.80620.8062 versus 0.8070.807) is due to evaluation precision. The NAMED-SAE value here uses the full-data alignment model, whereas Table reports the ten-image alignment model.

Thus, the submitted baselines do not all have exactly the same width, but they are matched in sparsity. An exact-width control confirms that the additional 500500 atoms do not explain the reconstruction gap.

AuthorsOpenReview ↗

Response to Reviewer 1 Part 3

W4: Adopting metrics from G-SAE and SAE-Bench. We agree that reconstruction and naming metrics alone do not provide a complete SAE evaluation. In response, we computed the SAE-Bench-style metrics that transfer directly to a VLM embedding SAE. Other metrics depend on language-model behavior or task-specific datasets and do not have direct analogues in our setting.

Applicable SAE metrics. The following results use the same CLIP post-projection activations and approximately matched sparsity, K≈20K\approx20 (the pretrained dictionary runs at L0=20.4L_0=20.4). The unconstrained column is the 9,866-atom SAE of Table 1; the 10,366-atom width-matched control appears in W2 and Q3.

MetricNAMED-SAE (10,366)Unconstrained (9,866)Pretrained, post hoc (12,288)
Sparse probing, AUC above null ↑0.3920.3680.368
Feature splitting, latents per class ↓134145163
Feature absorption rate ↓0.8220.5710.519
↳ eligible neighbors within cos⁡0.4\cos 0.425.09.9—
Monosemanticity, all alive slots ↑0.0970.1920.163
↳ ImageNet class slots only0.248——

NAMED-SAE performs better on sparse probing and feature splitting. Sparse probing measures the gap between the best single-latent AUC and a width-matched permuted-label null. Feature splitting measures the effective number of latents carrying each class. These results indicate that anchoring makes class information more localized.

NAMED-SAE performs worse on absorption. Absorption asks whether, when a class's primary latent is silent, a nearby latent fires instead. The anchored vocabulary contains many synonyms, hypernyms, attributes, and related concepts: each named atom has 25.0 eligible neighbors within cosine 0.4, compared with 9.9 for the unconstrained dictionary. This larger candidate set partly explains the higher absorption rate, although we do not treat it as a complete explanation. Importantly, the absorbing alternatives remain semantically identifiable. As shown in our response to Reviewer 2, W3, dog is coupled to concepts such as canine, puppy, and specific breeds, while the contextual correlate grass remains largely unaffected. Similarly, an absorbed goldfish feature may appear under the named hypernym fish, whereas absorption in an unconstrained dictionary occurs among anonymous latents. We will additionally report which named concepts absorb each target, making the type of absorption directly inspectable.

Aggregate visual monosemanticity is lower for NAMED-SAE: 0.097 across all alive slots versus 0.192 for the unconstrained SAE. We validate the implementation by reproducing the published score of 0.16 for the pretrained ×16\times16 SAE. The NAMED-SAE aggregate includes many abstract, relational, and rarely activated vocabulary entries for which visual coherence is difficult to estimate. Among its 994 active ImageNet class slots, the score rises to 0.248; because the unconstrained dictionary has no corresponding named subset, we report this only as a within-model diagnostic. We will report G-SAE's FMS separately because it measures supervised feature localization and disentanglement rather than visual coherence.

Metrics without direct analogues. Some SAE-Bench metrics are specific to language-model tasks and do not transfer directly to pooled VLM embeddings, including RAVEL, unlearning, and next-token loss recovered. We therefore adopt the applicable representation-level metrics above and use R2R^2 and decoded zero-shot behavior as the corresponding reconstruction and behavioral measures.

Thus, the revision adds sparse probing, feature splitting, absorption, and representation-based monosemanticity as a standard SAE-evaluation block, and discusses the relationship to FMS. We report the mixed outcome directly: NAMED-SAE improves localization and splitting, but performs worse on absorption and aggregate monosemanticity.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revised Manuscript Uploaded Addressing Reviewer Comments and Concerns. Summary of Edits below.

Summary of Revisions in the updated Manuscript Uploaded Above.

Edits are marked in blue for ease of reading. The current version doesn't have the results added today.

  1. Comparison to Concept-aware SAE methods added to Related Work, including G-SAE, Concept-SAE, CB-SAE, and hierarchical CLIP SAEs. Reviewer DEXP & M4DX. Section 2.

  2. Concept-SAE-style controls and fully supervised concept-to-latent assignment added, with causal removal measured on the label-based assignment. Reviewer M4DX & LMd8. Appendix D.3.

  3. Positive steering and matched-strength intervention-specificity experiments added. Reviewer DEXP & M4DX. Definitions in Appendix A.3, results in Appendices C.7 and C.8.

  4. SAE-Bench metrics added: sparse probing, feature splitting, absorption, and monosemanticity. Reviewer DEXP. Definitions in Appendix A.3, results in Appendix C.6.

  5. Width-matched and reconstruction-matched SAE comparisons added, with the anchoring cost reported explicitly. Reviewer M4DX. Section 4.1 and Appendix C.9.

  6. Named- and free-bank reconstruction contributions analyzed separately. Reviewer LMd8. Appendix C.10.

  7. Intermediate-layer CLIP experiments added, with deeper and non-VLM architectures acknowledged as limitations. Reviewers DEXP & LMd8. Appendix B.2.

  8. FVU/MSE controls and a clearer explanation of the normalized reconstruction loss added. Reviewer DEXP. Section 4.1 and Appendix C.5

  9. Post-hoc naming baselines properly attributed, and the label probe clarified as a supervised oracle. Reviewer DEXP. Section 4, Baselines paragraph and Table 1 caption.

  10. Results reorganized and implementation released. Reviewer DEXP. Section 4.1 and Appendix A.1.

We are working on adding the new results posted today to the paper as well. We will add those and update the manuscript again.

AuthorsOpenReview ↗

Summary of the Discussion Phase For all Reviewers and Area Chairs.

As the discussion period closes, we briefly summarize the reviewers' assessments and our revisions.

Strengths recognized by reviewers

  • Motivation: Anchoring semantic identities during training, rather than naming features post hoc, addresses a relevant limitation of conventional SAEs (Reviwers DEXP, M4DX).
  • Method: The orthogonal-map initialization addresses both post-hoc labeling difficulty and neuron non-monosemanticity (DEXP). Combining concept anchoring, bounded residuals, and free features is a straightforward way to balance interpretability and reconstruction (Reviewer M4DX).
  • Evaluation: Experiments are comprehensive across CLIP and SigLIP, baselines, ablations, and distribution shifts (Reviewer M4DX).
  • Transparency: We report failure cases candidly, including zero-precision class latents and sibling-class confusion (Reviewer LMd8).
  • Writing: The abstract, introduction, and related work are well written and richly referenced (Reviewer DEXP).

Key new evidence

  • A label-oracle assignment matches our precision@16 (0.89 vs. 0.86) but not causal removal (0.43 vs. 0.90), so the gain comes from anchoring rather than better assignment.
  • At matched reconstruction (R² = 0.805), NAMED-SAE retains higher precision@16 (0.74 vs. 0.46–0.63) and removal (0.56 vs. 0.31–0.43).
  • We also added SAE-Bench metrics, steering and intervention specificity, layer-22 anchoring, a named/free split of reconstruction, and mechanism-level controls for Concept-SAE and CB-SAE.
  • Transfer to new label spaces without retraining trails SpLiCE (Reviewer LMd8), due to encoder shift and with few shot(3-shot) fine-tuning outperforms SPLICE; we report this as a limitation of our method.

Most of the reviewers' concerns were addressed with new experiments and analyses. We hope the reviewers, AC, and SAC will consider these in their final evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.