- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper approaches the task of improving and scaling interpretability of Large Language Models (LLMs). The authors use tools already known in literature (contrastive pairs to generate concept vectors) and find a way to make the pipeline cheaper and scalable by having LLMs generating story pairs for 1036 concepts. They then evaluate the quality and generalization of the feature vectors they obtain on several settings. Finally, the authors argue that their pipeline produces reliable and general results.
Critical strengths
The contrastive pairs large scale dataset they generate has the potential to be a useful and adopted tool in the field. The interpretability problem in LLMs is very relevant and curated datasets/benchmarks that allow to conduct reproducible and systematic experiments and comparisons are absolutely useful.
Critical weaknesses
The main concern regards the novelty of the method. The procedures contain many similarities with Axbench by Wu et al. (2025). While this work cites Axbench, it understates the comparison with a work that produces a large synthetic dataset for LM steering, extracts difference in means vectors and releases dictionaries for 16K concepts. I still think that the paper has selling points, for example in the concept antagonist pairs strategy for the large dataset, but the differences should be discussed better.
I have concerns regarding the experiments, in particular:
The Probing Real text experiment in Section 3.1 does not have a null experiment. It is not clear whether the result is better than having a random vector select text.
The Steering experiment in Section 3.2 also does not report random directions results. Furthermore, it reports only the average over all the concept pairs. It is not clear whether this result is caused by consistent steering for all the concepts or by spike in steering for just some of them.
I lean towards weak rejection of the paper but I am open to change my score if the authors address my novelty and validation concerns.
Questions for authors
What are the advantages of this method and dataset over the ones produced in the Axbench paper?
How do the results change in the probing experiment in Section 3.1 by substituting the concept vector with a random vector?
What are the results for the steering experiments in Section 3.2 at a finer granularity? a histogram of the steering effects would help in proving the point.
2 comments
I thank the authors for the rebuttals and for running the new experiments. This rebuttal addresses my concerns about validation but I find that the novelty with respect to Axbench is still limited. This work introduces improvements to Axbench by adding vocabulary, new genres and languages. I think this makes it a useful improvement to the scope of the dataset but still an incremental contribution over Axbench. That said, while maintaining my concerns about novelty I will raise my score to weak accept as I think that the practical utility and breadth of this contribution are aligned with the important goal of developing systematic and standardized tools for interpretability.
We thank the reviewer for the thoughtful comments and for taking the time to engage with our work. Below we address each point.
Comparison with AxBench. AxBench was designed to compare SAE steering with other methods, so its evaluated vocabulary comes from SAE feature interpretations. These often describe simple lexical or topical patterns (such as rental services, UI layout, or C/C++ syntax). The authors themselves acknowledge that their labeling pipeline is “often skewed towards token-level concepts and misses high-level abstractions.” If a behavior such as reward hacking is absent from those interpretations, it will also be missing from the resulting vocabulary. Such surface-level features alone have limited practical value for monitoring complex alignment-relevant behaviors. Our work addresses this gap.
Second, we improve the vector-construction pipeline, using multiple languages and genres, shared story settings, and controls for confounds. As shown in our response to reviewers rmQq and RPsC, these controls improve vector stability and reduce sensitivity to irrelevant story outcomes.
Third, we investigate practical case studies, ablations, and limitations to establish how these vectors can be used. While AxBench’s framework accepts new concept descriptions, our focus is on constructing and validating useful alignment features and identifying effective ways to obtain them.
Probing with random vectors. We repeated §3.1 with random directions across all three model families and both corpora. Our vectors retrieved relevant spans 5.3x more often: 50.8% versus 9.6%. They outperformed random directions in all 48 model–corpus–pair comparisons. Below are the Llama results:
We will include the full table for all models and experimental details in the appendix of the revised paper.
Steering with random vectors. We repeated §3.2 with three random directions, matching the original injection norms, prompts, and evaluation. Across eight pairs per model spanning the steering ranking, our vectors substantially improved steering scores, while random directions did not reproduce this improvement.
The table reports changes from the unsteered baseline at the settings selected in the original paper. Positive values indicate improved steering.
Distribution of steering effects. More than 80% of the 1,034 evaluated pairs per model have positive mean steering effects. The improvement therefore extends across most concepts rather than coming from a few spikes. Below are changes from baseline at representative positions in each model’s full ranking:
We will add all new experiments and a detailed comparison with prior work to the revised paper. We hope this addresses your concerns and would be grateful if you would consider raising your score. We are happy to answer any further questions.