- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper examines how to verify that a steering vector represents the intended concept. It must not only cause a general change in model output. The authors state that comparisons with random vectors do not give sufficient evidence. They propose STEERCHECK, a preregistered audit method. This method matches each vector by the amount of change that it causes in the output. It also tests each claim with several control vectors. The main result is “alignment leakage.” Control vectors made with randomized labels often remain closely aligned with the steering vector. Thus, these control vectors give less useful information than expected. The authors apply STEERCHECK to an anti-sycophancy vector. The vector has an effect that is greater than general model disturbance, but the effect is not specific to the intended concept.
Critical strengths
-
The paper does not only examine whether behavior changes. It also examines whether the extracted direction represents the intended concept. This is important because a vector can improve the primary metric for an unrelated reason. The authors’ anti-sycophancy example shows this problem.
-
The alignment-leakage analysis and the dataset-level test help researchers evaluate a sign-randomized control before they use it. Researchers can then determine whether the control gives useful information. This result affects the interpretation of current and future claims about steering vectors.
Critical weaknesses
- The alignment-leakage analysis includes only two related Qwen model series. The effect-alignment profile uses only one model and one layer. The authors use other models, such as DeepSeek and Gemma, only in secondary experiments. They do not test a Llama model. Thus, it is not clear whether the main results apply to other model families.
- The audit uses forced-choice scores and very short generated texts. However, steering affects each generated token. In long texts or reasoning traces, the effect can increase or the model can correct it. Thus, the results might not apply to these increasingly common uses.
- Alignment leakage and its diagnostic use vectors made from differences between contrast pairs. Therefore, they do not directly apply to steering methods based on probes, optimization, or sparse autoencoders. This limits the use of the audit as a general standard.
Questions for authors
- The leakage diagnostic requires only stored activations. Can you report the diagnostic for a non-Qwen model family, such as Llama or Gemma? Also, can you determine whether the effect-alignment profile occurs in models other than Qwen3-14B?
- How do long outputs or chain-of-thought models affect the audit results? In these cases, the steering vector operates on many reasoning tokens. Can you enable thinking mode and report results that use final answers scored after the model completes its reasoning?