- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies safety steering when a single prompt can involve more than one harm category. The method, CAM-Steer, estimates a category-wise risk vector from hidden states and uses it both to compose safety directions and to set the strength of a norm-preserving spherical rotation. It is evaluated on BeaverTails’ seven harm categories and three 8–9B open-source models. On mixed-harm prompts, defense success is higher than the authors’ additive and single-direction variants, utility is largely preserved, and ablations suggest that risk modeling and the spherical update each contribute. I read the validation as showing that this combination works on the evaluated benchmark, rather than as establishing a general multi-category steering framework.
Critical strengths
-
The mixed-harm setting is a concrete operational issue, not only a slogan. A prompt can carry more than one harm, and steering a single category can leave another open. The self-harm-plus-violence example and the multi-harm tables make this visible, which is the main reason I take the problem seriously.
-
The method is internally consistent, and the evaluation on this benchmark is strong enough to support acceptance of the empirical claim. One risk vector controls both which directions are mixed and how far the representation is rotated. Relative to the authors’ additive and collapsed-direction controls, mixed-harm defense improves, utility stays in a reasonable range, and the ablations move the result. That is the evidence I would rely on if recommending acceptance.
Critical weaknesses
-
The paper does not isolate a new problem as clearly as the framing suggests. The introduction places safety, activation steering, multi-category leakage, and adaptive coordination of direction and strength side by side, but the links are thin. Related work already includes multi-attribute composition, gating, and rotation, and it is not shown why those methods could not be applied to the same mixed-harm setting. This is consequential because I would accept a working instance more readily than a claim that a new steering paradigm has been identified.
-
The broader robustness language is ahead of the evidence. The main results stay inside one taxonomy, and the risk scores that coordinate the method are only lightly diagnosed when two harms appear in one prompt. That does not undo the BeaverTails result, but it is why I would not treat the paper as having established a generally reliable multi-category framework.
Questions for authors
-
Related work already includes multi-attribute composition and rotation-based steering. Is there already a comparison, in the paper or appendix, that places such a method on the same mixed-harm BeaverTails setting? If so, please point to it; that would change how sharply I separate CAM-Steer from an instance of existing ideas.
-
The risk vector decides both which directions are used and how strong the update is. Do you already report per-category error, or how the scores behave when two harms appear in one prompt? If that analysis is present, I would put more weight on the “coordination” claim. If not, I will keep reading that part as only lightly tested.
-
Are the main directions and risk estimators built only on BeaverTails’ seven categories, or is there already a result on a different taxonomy or domain? If the latter is already in the paper, the robustness wording would be easier to accept.
4 comments
Updated Manuscript and Follow-up
Dear Reviewer,
Thank you again for your constructive feedback. We have uploaded a revised manuscript incorporating the additional experiments and clarifications discussed in our rebuttal. The corresponding revisions are highlighted in red for ease of reference.
The main updates addressing your concerns can be found in:
- Section 5.2 (Table 4) and Appendix D.2 (Tables 14–15): Multi-label risk-score evaluation and harmful leakage analysis.
- Section 4.2 and Appendix F (Table 23): Zero-shot transfer evaluation using frozen BeaverTails configurations across three backbones.
We hope these revisions address your concerns and would greatly appreciate any further feedback or questions you may have.
Best regards, The Authors
Thank you for the clarification and the additional analyses. After considering the response, I am updating my rating from 2 (weak reject) to 3 (weak accept).
First, thank you for pointing out the comparisons with MAT-Steer, K-Steering, and ORBIT-R/B in the submitted paper. These comparisons address my question about evaluating existing multi-attribute and rotation-based approaches in the same mixed-harm setting. I had not sufficiently credited this baseline coverage in my initial assessment. I also appreciate the clarified positioning: a category-adaptive steering mechanism, rather than a new steering paradigm.
Second, the multi-label detection and category-specific leakage analyses strengthen the evidence that the risk estimates provide useful information when harms co-occur. The detection results also make the limitations clearer: the estimator does not consistently recover every active category. I therefore view these results as supporting useful category-aware control, rather than complete category identification.
I also find the multi-category comparison with Top-1 Selection and Uniform Weights informative. The additional improvement from the full method on Llama provides more direct evidence that risk-weighted composition contributes beyond selecting a single direction.
My main remaining reservation concerns generalization. The frozen-configuration experiments are valuable, but the gains on external attacks are modest, and several attack families remain poorly defended. These results should be distinguished from experiments that reconstruct directions for the new attack families, and should not be presented as establishing broad robustness.
Overall, I already found the experimental core credible in my initial review. The clarified baseline coverage, stronger component analysis, and more precise positioning now make the case for acceptance more convincing. I support weak acceptance, while encouraging the authors to keep the generalization claims bounded by the reported results.
Thank you for reviewing our responses and for raising your score. Your suggestions have really helped us better position CAM-Steer and strengthen the experimental evidence for its key components. We sincerely appreciate your time and constructive feedback, and we will ensure that our contributions and limitations are clearly presented in the final version.
Response to Reviewer Cuzp
A1: We appreciate the reviewer’s comments on the relationship to existing multi-attribute and rotation-based steering methods. We would like to clarify that the submitted paper already includes these approaches in the experimental comparisons. In Section 4.1 (Baselines), we introduce MAT-Steer and K-Steering as multi-attribute control methods, along with ORBIT-R and ORBIT-B as rotation-based baselines. Table 2 reports the results on Qwen, while Appendix Table 10 provides the complete comparison across Qwen, Llama, and Gemma. CAM-Steer outperforms all four methods across the evaluated models.
We also agree that the contribution should be framed more precisely. Rather than claiming a new steering paradigm, our contribution lies in developing and empirically validating a category-adaptive mechanism that coordinates safety directions and intervention strength for co-occurring harm categories. We will clarify this positioning more clearly in the Introduction and Related Work.
A2: We appreciate the suggestion to provide a more direct analysis of the risk scores when multiple harm categories appear in the same prompt. Figure 5 and Appendix Figure 9 examine the category selectivity of the risk scores and their coverage of the source category. We now extend this evaluation to multi-label detection on prompts containing two to five active harm categories. Table R1 reports micro precision, recall, F1, exact match, and all-category recall. The micro F1 reaches 77.22% on Qwen, 70.58% on Llama, and 67.82% on Gemma. The results indicate that the risk estimator captures useful category information while still missing some active categories. However, we note that incomplete category detection does not necessarily lead to unsuccessful defense, as related safety directions may share representational components that help suppress harmful behaviors associated with missed categories. We also report response-level harmful leakage in Table R2.
Table R1. Multi-label detection of the seven modeled harm categories with K=2 to K=5 active categories.
Table R2. Response-level harmful category leakage across different backbones and steering methods.
A3: We thank the reviewer for highlighting the need for a clearer discussion of generalization. The category directions, prototypes, and risk calibration in the main experiments are constructed from the seven modeled BeaverTails categories. The submitted Appendix Table 12 already evaluates the original directions on seven additional BeaverTails categories and reports an increase in overall DSR from 94.07% to 95.57%. We also evaluate seven external jailbreak attack datasets in Appendix Table 13, where CAM-Steer achieves 82.00% overall DSR, outperforming No Steering by 5.86 percentage points and the strongest baseline by 4.71 percentage points. In this experiment, the steering directions are newly constructed for the seven attack families.
In addition, we conducted a zero-shot evaluation on the same seven jailbreak attack families. For each backbone, we kept the original BeaverTails directions, prototypes, angle budgets, intervention layer, and risk calibration fixed. Under this frozen configuration, overall DSR on the seven jailbreak datasets increased from 76.14% to 77.57% on Qwen, from 30.57% to 33.00% on Llama, and from 38.71% to 39.43% on Gemma. We will also include these new results in the revised manuscript.