- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Natural Language Dictionary Learning (NLD). NLD decomposes broad LLM behaviors, such as sycophancy, deception, and emotional expression, into more specific, interpretable, measurable, and intervenable behavioral concepts. Unlike conventional methods such as PCA and SAE, NLD first learns behavioral measurement profiles and subsequently identifies the corresponding neural activation directions.
The main contributions can be summarized in three aspects: 1.Paradigm innovation: The paper introduces a concept-first framework for behavioral dictionary learning, incorporating natural-language interpretability directly into the representation learning process.
2.Methodological innovation: NLD combines differentiable soft concepts, an objective for explaining behavioral variation, linear readability of unique concept components, and natural-language cycle consistency to jointly learn behavioral descriptions and their corresponding activation directions.
3.Evaluation framework: The paper evaluates behavioral dictionaries along three dimensions: name–direction fidelity, behavioral intervention selectivity, and average intervention effect on the broader behavioral domain (AIE), rather than relying solely on the plausibility of generated explanations.
Regarding experimental validation, NLD generally identifies representation directions that can be more faithfully described in natural language and more selectively intervened upon than those obtained from the selected PCA and SAE baselines, particularly for smaller dictionaries. However, these advantages diminish as dictionary size increases. Although NLD can effectively influence broad behavioral outcomes, it does not consistently outperform PCA in overall intervention strength. In terms of explanation fidelity, NLD achieves the highest point estimates in all 12 main multi-concept comparisons at matched dictionary sizes. The paper also provides some evidence of generalization through evaluations on held-out datasets and additional models.
Critical strengths
Methodologically, this paper challenges the conventional assumption that complex behaviors such as sycophancy and deception can be represented as single, coherent constructs. NLD proposes a concept-first paradigm for behavioral dictionary learning. Compared with the post-hoc interpretation approaches used in PCA and SAE, this methodological shift is valuable.
Critical weaknesses
NLD optimizes the ability to explain behavior-related variations in model activations, which does not necessarily lead to the identification of the genuine constituent factors underlying these behaviors. Some of the discovered concepts appear to reflect linguistic styles or dataset-specific features rather than meaningful components of sycophancy or deception. Consequently, Faithful Decompositions of LLM Behavior in the title, suggests a stronger scientific claim than is currently supported by the evidence.
Questions for authors
How does NLD establish that the discovered concepts are genuine constituents of the target behaviors, rather than linguistic styles or dataset-specific correlates? For example, some concepts in Table 4 appear only indirectly related to sycophancy or deception. Is there independent evidence supporting behavioral relevance beyond activation correlations and intervention selectivity? This bears directly on whether NLD is merely a method for learning behavior-related features or a method for genuinely decomposing behavior.
W1: Yes, we agree this is a core limitation in this method. However, this is also a limitation of every other current interpretability method that relies on contrastive datasets. Regardless, we are confident in the overall relevance of our results, both due to the causal intervention results in the main table, but also due to the performance of the probes and names in the external validity results in section 7.
We also renamed the paper to replace "faithful" with "interpretable". So the updated title is, "Natural Language Dictionaries: Learning Interpretable Decompositions of LLM Behavior".