acceptodds
Under review as a conference paper at ICLR 2027

From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability

Abstract

We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception. Existing unsupervised methods like sparse autoencoders attempt to recover the whole feature space, often at the cost of feature quality and interpretability. We take a more pragmatic route: focusing on concepts relevant to AI safety and alignment, we generate synthetic concept-antagonist story pairs and use the difference of their mean model activations as a feature vector. We scale this approach to 1,036 alignment-relevant concepts using 2M synthetic stories across multiple languages and genres, which we publicly release. Across six models from three model families, our feature vectors reliably generalize to detecting their corresponding concepts in chat dialogues and web text. They form an interpretable geometry shared across models and can causally steer model behavior when injected into activations. We further use these features to analyze model representations during jailbreak attacks, mathematical and agentic reasoning. Our targeted feature construction thus offers a scalable route to studying concepts in language models, potentially extending to millions of features across arbitrary domains.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, concept vectors, steering vectors, linear probing, activation steering, logit lens, cross-model transfer, jailbreaks, LLM reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_78sHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper approaches the task of improving and scaling interpretability of Large Language Models (LLMs). The authors use tools already known in literature (contrastive pairs to generate concept vectors) and find a way to make the pipeline cheaper and scalable by having LLMs generating story pairs for 1036 concepts. They then evaluate the quality and generalization of the feature vectors they obtain on several settings. Finally, the authors argue that their pipeline produces reliable and general results.

Critical strengths

The contrastive pairs large scale dataset they generate has the potential to be a useful and adopted tool in the field. The interpretability problem in LLMs is very relevant and curated datasets/benchmarks that allow to conduct reproducible and systematic experiments and comparisons are absolutely useful.

Critical weaknesses

The main concern regards the novelty of the method. The procedures contain many similarities with Axbench by Wu et al. (2025). While this work cites Axbench, it understates the comparison with a work that produces a large synthetic dataset for LM steering, extracts difference in means vectors and releases dictionaries for 16K concepts. I still think that the paper has selling points, for example in the concept antagonist pairs strategy for the large dataset, but the differences should be discussed better.

I have concerns regarding the experiments, in particular:

The Probing Real text experiment in Section 3.1 does not have a null experiment. It is not clear whether the result is better than having a random vector select text.

The Steering experiment in Section 3.2 also does not report random directions results. Furthermore, it reports only the average over all the concept pairs. It is not clear whether this result is caused by consistent steering for all the concepts or by spike in steering for just some of them.

I lean towards weak rejection of the paper but I am open to change my score if the authors address my novelty and validation concerns.

Questions for authors

What are the advantages of this method and dataset over the ones produced in the Axbench paper?

How do the results change in the probing experiment in Section 3.1 by substituting the concept vector with a random vector?

What are the results for the steering experiments in Section 3.2 at a finer granularity? a histogram of the steering effects would help in proving the point.

2 comments

We thank the reviewer for the thoughtful comments and for taking the time to engage with our work. Below we address each point.

Comparison with AxBench. AxBench was designed to compare SAE steering with other methods, so its evaluated vocabulary comes from SAE feature interpretations. These often describe simple lexical or topical patterns (such as rental services, UI layout, or C/C++ syntax). The authors themselves acknowledge that their labeling pipeline is “often skewed towards token-level concepts and misses high-level abstractions.” If a behavior such as reward hacking is absent from those interpretations, it will also be missing from the resulting vocabulary. Such surface-level features alone have limited practical value for monitoring complex alignment-relevant behaviors. Our work addresses this gap.

Second, we improve the vector-construction pipeline, using multiple languages and genres, shared story settings, and controls for confounds. As shown in our response to reviewers rmQq and RPsC, these controls improve vector stability and reduce sensitivity to irrelevant story outcomes.

Third, we investigate practical case studies, ablations, and limitations to establish how these vectors can be used. While AxBench’s framework accepts new concept descriptions, our focus is on constructing and validating useful alignment features and identifying effective ways to obtain them.

Probing with random vectors. We repeated §3.1 with random directions across all three model families and both corpora. Our vectors retrieved relevant spans 5.3x more often: 50.8% versus 9.6%. They outperformed random directions in all 48 model–corpus–pair comparisons. Below are the Llama results:

CorpusOur vectors: relevant spansRandom directions: relevant spans
LMSYS63.0%13.7%
OpenWebText64.8%11.0%

We will include the full table for all models and experimental details in the appendix of the revised paper.

Steering with random vectors. We repeated §3.2 with three random directions, matching the original injection norms, prompts, and evaluation. Across eight pairs per model spanning the steering ranking, our vectors substantially improved steering scores, while random directions did not reproduce this improvement.

The table reports changes from the unsteered baseline at the settings selected in the original paper. Positive values indicate improved steering.

ModelChange with our vectorsChange with random directions
Qwen3-8B+0.324−0.009
Llama-3.1-8B-Instruct+0.387−0.088
Gemma-2-9B-IT+0.397+0.020

Distribution of steering effects. More than 80% of the 1,034 evaluated pairs per model have positive mean steering effects. The improvement therefore extends across most concepts rather than coming from a few spikes. Below are changes from baseline at representative positions in each model’s full ranking:

Approximate rank percentileQwenLlamaGemma
6th−0.06−0.06−0.06
19th+0.09+0.13+0.13
56th+0.38+0.44+0.44
94th+0.72+0.81+0.86

We will add all new experiments and a detailed comparison with prior work to the revised paper. We hope this addresses your concerns and would be grateful if you would consider raising your score. We are happy to answer any further questions.

I thank the authors for the rebuttals and for running the new experiments. This rebuttal addresses my concerns about validation but I find that the novelty with respect to Axbench is still limited. This work introduces improvements to Axbench by adding vocabulary, new genres and languages. I think this makes it a useful improvement to the scope of the dataset but still an incremental contribution over Axbench. That said, while maintaining my concerns about novelty I will raise my score to weak accept as I think that the practical utility and breadth of this contribution are aligned with the important goal of developing systematic and standardized tools for interpretability.

Reviewer_RPsCOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a pipeline for generating and using “concept vectors”, linear directions in LLM activation space corresponding to human-relevant concepts. Specifically, the authors isolate safety and alignment-relevant concept vectors by employing 1,036 concept-pairs to generate a corpus (called Feature Stories) of 2M+ matched synthetic narratives across 148 concept classes. These safety-relevant concept vectors are constructed through a difference-in-means procedure, and are useful for behavior detection and steering in natural settings such as jailbreaks, chat, and chain-of-thought reasoning.

Critical strengths

This paper is very well-written and easy to follow. The pipeline decisions and subsequent tests and experiments are well-motivated and conclusive overall. The large corpus generated by the authors will surely be useful for safety and alignment experiments. Explicit antagonist pairing appears to work well.

Critical weaknesses

While the paper is very well-written, the significance of the work is doubtful to me. It is unclear what, if any, novel methodological or descriptive innovations are present in this work. As detailed in the paper, using difference-in-means to extract concept vectors is not new (Turner et al., 2023), and using synthetic stories to extract these concept vectors has been explored before (Sofroniew et. al, 2026). Therefore, it is hard for me to identify a genuine novel contribution put forward by this paper. The paper offers a potentially useful resource, but as a standalone research paper the contributions are somewhat lacking.

W1. There is limited demonstrated novelty and significance. Difference-in-means and synthetic story generation for feature extraction are established.

W2. There are not enough evaluation comparisons, for instance against unpaired synthetic stories, previous concept-vector methods, or SAE features. The pipeline appears to work well based on the experiments, but it is unclear which design choices are most effective.

W3. The authors somewhat over-intepret the feature readouts. For instance, on the 100 benign Alpace prompts, the authors state that their pipeline “does not degrade general behavior”, which seems to be a qualitative statement. The model could have low benign refusal an degraded performance, for instance.

Questions for authors

Q1. What do the authors view as the principal research contribution of their work, beyond the construction of the synthetic dataset?

Q2. Does explicitly pairing concept-antagonist stories improve detection or steering versus independently generated contrastive examples? Versus SAE features?

3 comments

We thank the reviewer for the thoughtful comments and for taking the time to engage with our work. Below we address each point.

Comparison with Persona Vectors. In Persona Vectors, the authors put a persona instruction into an instruct model’s system prompt (for example, “be evil”), ask N neutral questions, obtain the model’s responses to these questions (with an evil inclination), and then extract a difference-in-means vector for this persona by averaging over all N examples. We see two limitations of this approach. First, the method is on-policy: if we obtain evil-persona responses from Llama, it is unclear how well these same responses can be used to extract vectors for Qwen, since their styles and formats may differ. Moreover, it is unclear how to extract such features for base models when the entire pipeline relies on instruct models. We address this by using pretraining-like stories that can be reused across different models and their base/instruct versions.

Second, this pipeline limits the concepts and features that can be extracted. For example, it is unclear how to obtain a feature for eval-oblivious behavior vs. eval-aware behavior, or faithful chain-of-thought vs. post-hoc rationalization, in their setup. Our pipeline allows us to extract any feature that can be expressed in a pretraining-like style, thanks to the variety of formats. Moreover, as the authors themselves note, an instruct model may simply refuse to answer questions under a system prompt telling it to be evil. In such a pipeline, it would therefore not be possible to obtain a refusal feature, whereas our dictionary includes such features.

Comparison with Emotion Vectors. In Emotions, the authors average story activations for each emotion and subtract the mean across all emotions. At the very beginning of our experiments, we tried this approach too. However, when working with political ideologies, we noticed something unexpected: the vectors for socialism and right-wing radicalism were very similar (cos-sim >0.8, with an average cos-sim around 0). We attributed this to the main axis captured by our features being the degree of radicalism. As a result, radical left-wing and right-wing ideologies became very similar. This was not what we wanted, since we were interested specifically in the differences between them.

We therefore started explicitly specifying an antagonist: it is important to define not only the concept, but also what we want to distinguish it from. When subtracting the overall mean, this contrast depends on the other concepts in the set. In our approach, we specify it directly. For example, we are interested in the distinction between genuinely completing a task and manipulating the reward metric. This is precisely the distinction we build into the stories.

We also use a shared setting for both poles. If we independently ask a model to write about honesty and deception, it might write about a scientist reporting experimental results and a salesperson concealing a product defect. Then the profession, topic, and circumstances change along with the behavior. We instead keep the same scientist in the same situation: in one version, they report the failed experiment; in the other, they conceal it. We also ask for stories to end at the decision, before describing any subsequent reward or punishment.

New experiments. We ran two tests to measure what our story-construction choices contribute, keeping the generator and vector-extraction method the same.

First, we compared stories generated with shared settings against stories generated with independent settings, using the same concepts. We then extracted vectors from two separate, non-overlapping samples of stories and checked how closely the vectors agreed. We repeated this with different numbers of stories to see how much data each approach needed to produce a stable direction.

Table 1. Mean cosine similarity at different story counts per pole. Both conditions stop at the decision.

ModelSettings4 stories8 stories16 stories32 stories64 stories
Qwen3-8BIndependent0.4370.6160.7620.8650.925
Qwen3-8BShared0.6740.8080.8960.9450.972
Gemma-2-9B baseIndependent0.4220.5960.7510.8570.920
Gemma-2-9B baseShared0.6750.8090.8960.9460.972

Shared settings improve agreement at all five story counts in both models. With 32 stories per pole, they already outperform independent settings using 64 stories per pole.

Second, we tested why stories should stop before rewards or punishments. Reporting a failed experiment is honest whether the scientist is rewarded or punished. We want the vector to recognize the action in both cases. We built two sets of vectors using shared settings: one from training stories that could describe consequences, the other from stories that stopped at the decision. We then tested both on new stories, giving the exact same action either an approval or a rejection ending. We measured how much the ending changed the feature score, relative to the score gap between the two behaviors. Stopping training stories at the decision reduced this outcome sensitivity by 38% for Qwen and 35% for Gemma. These choices make vectors more stable and less sensitive to story outcomes.

Comparison with SAE. Our goal is to start with a behavior of interest and obtain a vector for it, for example, for the distinction between genuinely completing a task and reward hacking. A direct comparison with SAE first requires finding features corresponding to this exact contrast, which is itself nontrivial. AxBench has already systematically compared methods on concepts from an SAE dictionary: difference-in-means outperformed SAE on average in both concept detection and steering. We build on these results, while our work investigates the construction and application of alignment features.

Response quality under safety steering. You are right here: a model may rarely refuse benign requests while still producing worse answers. We therefore added direct evaluations of accuracy, helpfulness, and correctness.

Using the safety-steering vectors on all 285 MMLU dev questions, we found that mean accuracy differs from baseline by no more than 1.3 percentage points across all six tested steering strengths:

ModelAccuracy without steeringAccuracy with steering
Qwen3-8B67.02%67.23–68.14%
Llama-3.1-8B-Instruct67.02%65.75–67.16%
Gemma-2-9B-IT72.63%71.89–72.67%

We also evaluated 18,300 responses to the same 100 Alpaca prompts. GPT-OSS-20B separately rated helpfulness and correctness/completeness without knowing the model or steering condition:

ModelHelpfulness: unsteered → steeredCorrectness: unsteered → steered
Qwen3-8B3.690 → 3.762–3.8563.610 → 3.686–3.786
Llama-3.1-8B-Instruct3.540 → 3.750–3.7943.420 → 3.684–3.736
Gemma-2-9B-IT3.680 → 3.693–3.7453.600 → 3.625–3.689

Scores are on a 0–4 scale; the ranges cover all six steering strengths. Full tables, intervals, and details will be included in the appendix of the revised paper.

We will add all new experiments and a detailed comparison with prior work to the revised paper. We hope this addresses your concerns and would be grateful if you would consider raising your score. We are happy to answer any further questions.

I thank the authors for responding to most of my concerns and for running additional experiments.

However, I will maintain my score. I still do not see that there is sufficient novelty to warrant an acceptance...the core pipeline is still a prompt-refinement of Sofroniew et al. (2026) and AxBench, with some added scaling. I do not see a substantial methodological or even descriptive advance here. As well, I think direct head-to-head comparisons against prior methods (SAEs, persona vectors) are necessary to show the downstream utility of this approach. I understand that it might be non-trivial to find exact features for testing, but these comparisons are necessary to identify if the presented method is worthwhile compared to more established approaches. Testing cosine agreement between splits only proves that there is internal consistency, it does not show how useful these vectors are for downstream tasks, above other methods.

Reviewer_rmQqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies feature vectors that correspond to concepts relevant to AI safety and alignment, by generating synthetic concept-antagonist story pairs and calculating the difference of their mean model activations. On over 1036 concepts using 2M synthetic stories across different languages and genres, the feature vectors are shown to reliably represent their corresponding concepts. These concepts are further utilized to analyze model representation during jailbreak attacks, mathematical and agent reasoning.

Critical strengths
  • The main strength of this paper is that its claims are well supported by a broad set of validation experiments. The paper covers 1,036 concept pairs across multiple languages and genres and studies three model families. Table 8 evaluates transfer to held-out languages, genres and generators, including training-size-matched controls. Natural-text retrieval in Section 3.1 and steering comparisons against unsteered responses to the same prompts in Appendix D.3 and Figure 12 further broaden the evaluation. This provides evidence of practical utility within the tested settings and makes the resource a potentially useful basis for subsequent probing and steering studies, which is the main positive consideration in my assessment.
Critical weaknesses

The incremental methodological contribution over direct predecessors is insufficiently established. Persona Vectors (Chen et al., 2025, Section 2) already automates contrastive data construction from a trait description and extracts difference-in-means activation directions for monitoring and steering. Sofroniew et al. (2026) already extracts concept directions from synthetic stories, using across-emotion centering and neutral-context components to mitigate confounds. The present pipeline substantially follows these established ideas. Explicit concept–antagonist pairs, shared story settings, and broader multilingual alignment coverage are identifiable differences, but the paper does not isolate what these changes contribute relative to the closest constructions under comparable conditions. The expanded ontology and the analyses of stability and intervention layers add empirical content; however, the submission does not sufficiently establish why these additions constitute a substantial advance over the existing pipelines. This is a central concern about the significance of the contribution, beyond whether the proposed vectors work within the reported evaluation.

Questions for authors

What evidence establishes a substantial benefit of the explicit antagonist and shared-setting construction over Persona Vectors or the story-based construction of Sofroniew et al., beyond the expanded ontology? Are comparisons under matched concepts and comparable data budgets available?

3 comments

We thank the reviewer for the thoughtful comments and for taking the time to engage with our work. Below we address each point.

Comparison with Persona Vectors. In Persona Vectors, the authors put a persona instruction into an instruct model’s system prompt (for example, “be evil”), ask N neutral questions, obtain the model’s responses to these questions (with an evil inclination), and then extract a difference-in-means vector for this persona by averaging over all N examples. We see two limitations of this approach. First, the method is on-policy: if we obtain evil-persona responses from Llama, it is unclear how well these same responses can be used to extract vectors for Qwen, since their styles and formats may differ. Moreover, it is unclear how to extract such features for base models when the entire pipeline relies on instruct models. We address this by using pretraining-like stories that can be reused across different models and their base/instruct versions.

Second, this pipeline limits the concepts and features that can be extracted. For example, it is unclear how to obtain a feature for eval-oblivious behavior vs. eval-aware behavior, or faithful chain-of-thought vs. post-hoc rationalization, in their setup. Our pipeline allows us to extract any feature that can be expressed in a pretraining-like style, thanks to the variety of formats. Moreover, as the authors themselves note, an instruct model may simply refuse to answer questions under a system prompt telling it to be evil. In such a pipeline, it would therefore not be possible to obtain a refusal feature, whereas our dictionary includes such features.

Comparison with Emotion Vectors. In Emotions, the authors average story activations for each emotion and subtract the mean across all emotions. At the very beginning of our experiments, we tried this approach too. However, when working with political ideologies, we noticed something unexpected: the vectors for socialism and right-wing radicalism were very similar (cos-sim >0.8, with an average cos-sim around 0). We attributed this to the main axis captured by our features being the degree of radicalism. As a result, radical left-wing and right-wing ideologies became very similar. This was not what we wanted, since we were interested specifically in the differences between them.

We therefore started explicitly specifying an antagonist: it is important to define not only the concept, but also what we want to distinguish it from. When subtracting the overall mean, this contrast depends on the other concepts in the set. In our approach, we specify it directly. For example, we are interested in the distinction between genuinely completing a task and manipulating the reward metric. This is precisely the distinction we build into the stories.

We also use a shared setting for both poles. If we independently ask a model to write about honesty and deception, it might write about a scientist reporting experimental results and a salesperson concealing a product defect. Then the profession, topic, and circumstances change along with the behavior. We instead keep the same scientist in the same situation: in one version, they report the failed experiment; in the other, they conceal it. We also ask for stories to end at the decision, before describing any subsequent reward or punishment.

New experiments. We ran two tests to measure what our story-construction choices contribute, keeping the generator and vector-extraction method the same.

First, we compared stories generated with shared settings against stories generated with independent settings, using the same concepts. We then extracted vectors from two separate, non-overlapping samples of stories and checked how closely the vectors agreed. We repeated this with different numbers of stories to see how much data each approach needed to produce a stable direction.

Table 1. Mean cosine similarity at different story counts per pole. Both conditions stop at the decision.

ModelSettings4 stories8 stories16 stories32 stories64 stories
Qwen3-8BIndependent0.4370.6160.7620.8650.925
Qwen3-8BShared0.6740.8080.8960.9450.972
Gemma-2-9B baseIndependent0.4220.5960.7510.8570.920
Gemma-2-9B baseShared0.6750.8090.8960.9460.972

Shared settings improve agreement at all five story counts in both models. With 32 stories per pole, they already outperform independent settings using 64 stories per pole.

Second, we tested why stories should stop before rewards or punishments. Reporting a failed experiment is honest whether the scientist is rewarded or punished. We want the vector to recognize the action in both cases. We built two sets of vectors using shared settings: one from training stories that could describe consequences, the other from stories that stopped at the decision. We then tested both on new stories, giving the exact same action either an approval or a rejection ending. We measured how much the ending changed the feature score, relative to the score gap between the two behaviors. Stopping training stories at the decision reduced this outcome sensitivity by 38% for Qwen and 35% for Gemma. These choices make vectors more stable and less sensitive to story outcomes.

We will add all new experiments and a detailed comparison with prior work to the revised paper. We hope this addresses your concerns and would be grateful if you would consider raising your score. We are happy to answer any further questions.

Thanks very much for your reply. While the authors have clarified the methodological difference between this work and previous works, it does not convince me that the difference leads to significant novelty and contributions to the research area. I would still maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.