- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents a pipeline for discovering concepts in human preference data, confirming their descriptions, and turning them into model-specific steering vectors. The authors evaluate 48 candidate vectors across two language models and audit 84 existing vectors, recording their meanings, provenance, and evaluation results in a library. None of the newly constructed vectors meets the paper’s behavioral qualification criteria; the authors use this result to discuss limitations in both steering interventions and their evaluation.
Critical strengths
- Transparent reporting of unsuccessful results: The paper retains failed candidates and distinguishes semantic confirmation from behavioral qualification. Its analysis of score saturation, truncation, and judge instability provides useful documentation of potential evaluation pitfalls.
Critical weaknesses
-
Unvalidated contrast signal: Human preference pairs can differ in correctness, length, tone, and other qualities at once. Assigning a concept label to a feature does not establish that the underlying pairs isolate that concept or provide a strong contrast for extracting its direction. The paper does not sufficiently validate this assumption, so the negative results cannot be attributed to the overall construction approach.
-
Limited experimental scope: The study uses a narrow range of data, extraction settings, and evaluation prompts. Without broader experiments or stronger ablations, it remains unclear whether the findings reflect limitations of the construction pipeline or the particular settings tested.
Questions for authors
- Extraction validity: Do the 32 extraction pairs isolate the intended concept, or do they also differ systematically in length, content, and other attributes? Can the authors compare against controlled contrastive pairs and examine how results change with more extraction data?
- Robustness and statistical power: Do the findings persist across a broader layer and strength search, different prompt panels, and repeated extraction samples? More independent test coverage would also help distinguish small behavioral effects from judge variability.
- Presentation: The paper should more clearly distinguish its different evaluation criteria and cohorts, and consistently separate failure to meet the qualification standard from evidence that a steering direction is ineffective. Also the in-text citations appear to have a LaTeX formatting issue. Please check the citation commands and correct the formatting throughout the paper.
- Data construction detail: It would be better if the authors could provide representative preference-pair examples for the extracted concepts, and explain how they determined that these pairs isolate the intended concept rather than other response differences.