acceptodds
Under review as a conference paper at ICLR 2027

Concept-Conditioned Scores Can Rank Sparse Dictionaries Without Using the Concepts

Abstract

Sparse dictionaries are increasingly compared by scores computed from concept labels: how compactly a labeled concept is represented, how well it matches a feature, how much information a feature carries about it. Concept labels give such a score an apparent semantic grounding. Whether the ranking of dictionaries it induces depends on the labels is a separate question, and existing sanity checks, which randomize the dictionary, do not ask it. On dictionaries we train for chest radiographs and on public sparse autoencoders for language models, under three label systems, replacing each concept's support with a size-matched random set of documents leaves the ranking under most scores unchanged. The invariance reaches model selection: on a controlled sweep, compactness scores rank collapsed dictionaries first and give their best value to a dictionary trained on permuted concept targets, whereas probes favor real targets on every one of 20 paired seeds. We therefore propose TWIN (two-way identity null), a protocol for auditing a concept-conditioned score before it is used to compare dictionaries: it destroys concept identity once at evaluation and once at training, controls occupancy, and anchors every score to probing. Applied to fourteen scores, TWIN separates failures of different kinds: rankings that do not move when the concepts are randomized, preferences for real targets that vanish once the feature budget is matched, and a matching score whose ranking follows the live count. The mutual-information gap is the most label-sensitive score, and its sensitivity to discretization keeps it a diagnostic. For compactness the failure can be explained: the score depends only on the sorted shape of a concept's activation profile, that shape stays close to the label-free marginal one on all our pools, and for the primary score we prove that this closeness confines its value to an interval around the value it takes with no labels at all.

Primary area:
interpretability and explainable AI
Keywords:
Sparse Autoencoders, Dictionary Learning, Mechanistic Interpretability, Evaluation, Sanity Checks, Concept-based Interpretability

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_U83ROpenReview ↗
Rating
4(+1)
Confidence
3(+1)
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper investigates whether concept-conditioned scores genuinely depend on concept identity when ranking sparse dictionaries. The authors propose TWIN, combining evaluation-time label randomization, training-time target permutation, occupancy control, and probing anchors. Across 65 dictionaries and three label systems, many scores preserve nearly identical rankings under randomized labels, sometimes leading to poor model selection. A theoretical bound explains this failure for mass coverage.

Critical strengths
  1. Important and actionable methodological contribution.

    TWIN addresses an overlooked gap in existing SAE sanity checks by testing whether dictionary rankings, rather than merely score values, depend on concept identity. Its occupancy control further addresses a confound that survives label randomization.

  2. Strong empirical validation and theoretical insight.

    The study combines 65 dictionaries, three label systems, paired training experiments, and controlled synthetic tests. The label-free bound explains the compactness failure, while pre-registration and transparent reporting strengthen the evidence.

Critical weaknesses
  1. Limited evidence of practical model-selection harm.

    The strongest selection failures involve collapsed dictionaries, while selection regret is measured by probing recoverability rather than concept compactness itself. The practical consequences among healthy dictionaries remain unclear.

  2. Limited generalization to other evaluation regimes.

    Training-time and model-selection conclusions rely on one medical dictionary family, while public SAEs use document-level pooling with saturated probing anchors. Directly concept-bound supervised dictionaries are also not tested.

Questions for authors
  1. Can the authors report held-out selection regret after excluding collapsed dictionaries (e.g., live@k32>100\mathrm{live@k32} > 100), ideally comparing full-probe recoverability with sparse-probe or concept-selectivity criteria?​

  2. Can the authors evaluate TWIN on directly concept-bound supervised dictionaries, or report the profile discrepancy and null-test results for the existing per-feature-aligned configurations?

Reviewer_JNWbOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether concept-conditioned evaluation metrics genuinely reflect concept information when ranking sparse dictionaries. The authors demonstrate that many existing metrics, particularly compactness-based scores, produce nearly identical dictionary rankings even when concept labels are replaced with random supports. To systematically assess this issue, the paper proposes TWIN, an evaluation protocol combining evaluation-time label randomization, training-time target permutation, occupancy control, and probing-based anchors. Experiments on medical-image dictionaries and public Gemma Scope SAEs reveal that several commonly used metrics are insensitive to concept identity, can favor collapsed dictionaries, or are confounded by the number of active features. The paper further provides a theoretical explanation for the failure of compactness scores through a label-free bound and identifies the mutual-information gap as a relatively label-sensitive but estimator-dependent diagnostic.

Critical strengths
  1. The proposed TWIN protocol offers a systematic and interpretable evaluation framework. By combining evaluation-time randomization, training-time target permutation, occupancy control, and probing-based positive controls, TWIN distinguishes several sources of evaluation failure. In particular, the separation between sensitivity to concept identity and sensitivity to the number of active features provides useful insights into the behavior of different metric families.

  2. The experimental analysis is relatively comprehensive and supports several important findings. The authors examine fourteen evaluation scores across three label systems and two application domains, supplemented by twenty paired training seeds, synthetic planted-concept experiments, and alternative label-randomization strategies.

  3. The paper addresses an important yet underexplored problem in sparse dictionary evaluation, where the authors go beyond testing whether a metric responds to concept labels by examining whether those labels meaningfully affect dictionary rankings. This distinction is particularly relevant when concept-conditioned scores guide model selection and offers a valuable perspective on evaluation reliability.

Critical weaknesses
  1. The model-selection conclusions rely heavily on severely collapsed dictionaries, limiting their relevance to well-trained models. For example, Section 4.3 reports that the five dictionaries ranked highest by mass coverage have only 9–16 live features. Appendix C attributes this collapse to configurations where 96–99.99% of retained entries are negative and removed by post-hoc truncation. Although ranking correlations are analyzed for dictionaries with over 100 live features, held-out selection regret is not reassessed on this subset. Evaluating selection regret among healthy dictionaries with comparable reconstruction quality and sparsity, and comparing selected models against probing anchors and label-free baselines, would clarify whether compactness drives poor selections in realistic model pools or mainly fails to reject pathological configurations.

  2. Section 3.2 restricts each dictionary to its top 115 features according to activation mass, but this operation may disproportionately remove low-frequency features that encode meaningful concepts, rather than merely equalizing dictionary capacity. The authors should examine sensitivity to different feature budgets and alternative occupancy-matching strategies, such as comparing dictionaries with naturally similar live-feature counts or training dictionaries under matched occupancy constraints. Such experiments would help distinguish genuine occupancy confounding from performance degradation induced by the feature-selection procedure itself.

  3. While evaluation-time label randomization is tested on both medical dictionaries and public Gemma Scope SAEs, the training-time null and its occupancy-controlled comparison rely entirely on the report-aligned medical dictionary trained with real or permuted concept targets. Consequently, important findings concerning max-over-features scores and their loss of preference for real targets after occupancy control have not been independently validated on a different supervised dictionary family. An additional experiment involving concept-supervised language-model SAEs or another supervised dictionary architecture, with non-saturated probing anchors, would substantially strengthen the generality of TWIN's conclusions.

  4. Proposition 1 bounds mass coverage using the deviation δv\delta_v between sorted concept-specific and marginal activation profiles, but the bound does not establish conditions under which δv\delta_vshould become small. Therefore, the proposition alone provides limited predictive guidance about which concepts or dictionary configurations will exhibit label-insensitive rankings. A more informative analysis would characterize how δv\delta_v varies with concept prevalence, activation sparsity, and concept–feature alignment, and examine whether these quantities predict actual ranking disagreement under label randomization.

Questions for authors

Please carefully checking the weaknesses.

Reviewer_W7bYOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors target the problem that some concept-conditioned scores used to rank sparse dictionaries may not actually use these concepts to determine the rankings. They tested this with 65 dictionaries and three different label systems and found that when some parts are randomly changed, the rankings remain mostly unchanged. The compactness metrics can even use collapsed dictionaries, since some concepts activate the same concept, which may actually be activated by all concepts, leading to a shortcut. To find good metrics, the authors proposed TWIN, which utilizes evaluation-time label randomization, training-time permutation, occupancy control, and anchor probing. The experiments show that TWIN can successfully expose these failures.

Critical strengths
  • The authors point out a problem that may affect obtaining sparse dictionaries.
  • The authors propose a validation protocol that can be used to figure out which metrics are wrong.
  • The experiments demonstrate the proposed metric’s success in exposing failure cases.
Critical weaknesses
  • I appreciate the authors’ contribution in identifying this issue and their extensive empirical evaluation of concept-conditioned scores for sparse dictionaries. While TWIN provides a useful validation protocol for metric, the paper would be strengthened by more actionable guidance on model selection or a reliable alternative metric informed by the findings. MIG appears more sensitive to concept labels than most of the evaluated metrics, but its sensitivity to discretization limits its suitability as a robust model-selection criterion. Will it be helpful to have some multiple metrics weightly combined, where different metrics can check different aspects?

  • Many of the tests appear to depend heavily on threshold choices, yet these choices are often presented without sufficient justification. Were the thresholds chosen heuristically or empirically? Would different label systems require different thresholds, and how robust are the conclusions to variations in these thresholds?

Questions for authors

Check the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.