- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CLAS extends linear activation steering by retaining a fixed, probe-derived direction and learning an activation-dependent coefficient that varies across tokens and layers. The coefficients are trained using next-token prediction on a small set of target completions. Experiments across 11 tasks and four models demonstrate improvements over fixed-strength steering and competitive performance against LoRA and ReFT. The paper also evaluates cross-task interference, concept monitoring, and selective steering within structured outputs.
Critical strengths
The strongest contribution is the controlled evidence that contextual coefficients improve steering. The paper evaluates 11 tasks across four models and includes learned per-layer constant coefficients, higher-rank LoRA/ReFT, four-seed results, and cross-task interference measurements. These controls strengthen the comparison with fixed-strength steering.
Critical weaknesses
-
Monitoring accuracy does not fully establish interpretability. Section 7 mostly tests whether directions distinguish explicit prompt instructions, such as “C++” versus “Python” or “answer correctly” versus “incorrectly.” This supports recognizing those prompt conditions, but does not establish reliable monitoring of actual generated behavior or explain the learned coefficients.
-
The advantage over LoRA/ReFT has several possible causes. CLAS typically receives 200 labeled probe examples in addition to the 10 training completions available to the other methods. The experiments do not isolate the contribution of concept alignment from extra supervision, freezing one adapter factor, or reduced trainable capacity.
-
- Selective steering is promising but narrowly demonstrated. Appendix B shows opposite ends of one attribute applied to fixed JSON fields, for example, negative sentiment in one field and positive sentiment in another. It does not test independent concept combinations, unfamiliar field arrangements, or comparison with LoRA/ReFT on this setting.
Questions for authors
-
What evidence distinguishes genuine concept monitoring from recognizing the explicit wording of task/non-task prompt templates? Do any reported results evaluate actual generated behavior or generalization to unseen instruction formulations?
-
CLAS receives additional labeled probe data unavailable to LoRA/ReFT. What evidence separates the benefit of concept-aligned directions from extra supervision and the regularization introduced by freezing one adapter factor?
-
Is the claim limited to applying opposite ends of one attribute within a fixed, trained JSON layout, or intended to cover more general contextual control? What evidence supports the broader interpretation?
3 comments
Author Response to Reviewer mKV1 (Continued)
W3. Selective steering is promising but narrowly demonstrated. Appendix B shows opposite ends of one attribute applied to fixed JSON fields, for example, negative sentiment in one field and positive sentiment in another. It does not test independent concept combinations, unfamiliar field arrangements, or comparison with LoRA/ReFT on this setting.
Q3. Is the claim limited to applying opposite ends of one attribute within a fixed, trained JSON layout, or intended to cover more general contextual control? What evidence supports the broader interpretation?
The claim is not limited to the JSON layout. We use the JSON layout because failures to steer selectively, or failures due to violating the JSON schema, can be evaluated automatically. The experiment tests whether a single steering vector can steer different portions of a single generation toward different behaviors, comparing context-dependent and fixed coefficients on the same steering vector. CLAS steers the two fields toward different behaviors with a single steering vector at each block, extracted from a single dataset. To our knowledge, this is the first demonstration of steering different portions of a single generation toward opposite behaviors using a single steering vector at each block. In contrast, LAS cannot achieve this with any fixed coefficient. This reveals a fundamental limitation of steering with a fixed coefficient, the primary approach to activation steering. More general contextual control and concept combinations can be achieved by using more than one steering vector at a given block. Evidence for this is shown in [1], which supplies CLAS with multiple steering vectors at each block.
[1] Gurgurov et al., "Compositional Activation Steering Across Languages and Behaviors with Contextual Gates", arXiv preprint 2026
Author Response to Reviewer mKV1 (Continued)
Thank you again for your review. If our responses have addressed your concerns, please consider raising your score. We are happy to answer any further questions.
Author Response to Reviewer mKV1
We thank Reviewer mKV1 for their feedback and address their individual questions and suggestions below.
The probe datasets for JailbreakBench and ToxicChat do not use task/non-task prompt templates, so their two classes differ only in prompt content rather than instruction wording. For both, concept monitoring results are strong. The directions evaluated in Section 7 are the same directions used for steering in Section 6.1, which evaluates actual generated behavior. Section 6.2 further evaluates the steered models on unseen tasks and instruction formulations. Moreover, the selective steering results in Appendix B use instructions not seen in the probe data. All evaluations are on held-out test sets.
Regarding the learned coefficients, a coefficient only measures how much the activation changes along the steering direction, not where it ends up. For example, two activations can receive very different coefficients yet end up close together along the steering direction, whereas two activations that receive the same coefficient can end up far apart. To show where activations steered with CLAS end up along the steering direction, we add a new appendix (Appendix H) that tracks this across blocks, tokens and JSON fields. Since the steering direction used by CLAS reliably captures the concept, as shown in Section 7 and discussed above, where activations end up after steering can be interpreted in terms of the concept.
We examined the effect of increasing the number of prompt-completion pairs in Table 16, as referenced in Section 4.2. With over 10x as many prompt-completion pairs, CLAS remains competitive with LoRA and ReFT, and at the largest dataset size it is the best method on three of four models. We also note that dataset sizes are not directly comparable between the probe data and the prompt-completion pairs, since the latter carry significantly more information per example and are therefore more expensive to obtain in practice.
The steering direction used by CLAS is not used as a regularizer but is obtained using the activation steering methodology, where the direction is extracted from a probe [1–3]. Since this direction is concept-aligned, steering along it can be interpreted in terms of the concept, as described in our previous response (W1, Q1).
[1] Li et al., "Inference-Time Intervention: Eliciting Truthful Answers from a Language Model", NeurIPS 2023
[2] Rimsky et al., "Steering Llama 2 via Contrastive Activation Addition", ACL 2024
[3] Beaglehole et al., "Toward Universal Steering and Monitoring of AI Models", Science 2026