- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a "contrastive" loss based on the eigenvalue of a rank-one operator representing safety-prompt-induced activation changes within the transformer’s MLP weights in target large language models. Besides, the loss optimizes prompts to amplify their influence on harmful queries while suppressing it on harmless queries, which can be adjusted with a suppression weight ρ. Experiments across four prompt searching space optimization settings on Gemma models show controllable safety–over-refusal trade-offs and "Pareto improvements" of the proposed method in some configurations.
Critical strengths
- I think the proposed eigenvalue-based loss is somewhat novel and intuitively motivated, balancing safety on harmful queries against over-refusal on benign ones. And this loss can be applied to different prompt searching settings (e.g., soft prompt, hardGCG).
- The experiments demonstrate promising results, with some configurations improving both attack success and over-refusal rates ("Pareto improvements") over the original safety prompt.
- The paper cites/discusses relevant prior works and connects its formulation to existing theoretical and prompt optimization methods properly.
Critical weaknesses
-
I think the comparisons with Original SI and No-SI in Sections 5.1–5.2 demonstrate the usefulness of the method, but it doesn't sufficiently isolate the benefit of the eigenvalue-based objective.It is not that surprising that a loss balances between safety refusal and harmless question instruction-following can bring about "Pareto-improvement", you experiment results on DRO just confirm this. In addition, the DRO comparison in Appendix B.5 (Table 1 and Figure 4) does not clearly establish superior performance: the authors acknowledge better overall ASR/ORR results for DRO while emphasizing their method’s higher improvement-retention rate. Controlled comparisons with alternative representation-based losses, including the norm-based objectives mentioned in Remark 3.1, would strengthen the contribution.
-
I am not fully convinced by the interpretation of the eigenvalue as instruction strength in Sections 3.2–3.3. Equation (5) captures only the projection of the safe activation onto the clean activation. As Equation (7) in Appendix A shows, driving the eigenvalue toward 1 leaves an angle-dependent residual. I think Corollary A.4.1 should distinguish approximate closeness from strict convergence and provide stronger empirical validation of the alignment assumption.
-
I suggest the layer ranges reported in Appendices C–F would benefit from systematic ablations, including different depths, single-layer versus multi-layer objectives, and feedforward-input versus block-output activations. I also find the 90.9% versus 64.2% retention comparison in Appendix B.5 insufficient to establish stronger generalization, since the candidate pools differ and checkpoints from the same trajectory are correlated. A common validation-based selection rule and independent training seeds would make this comparison more convincing.
-
It would be better to fix some presenations in the paper. For example, Figure 1's overlap between words and arrows (to make it more prefessional) and switch Figure 2 from png/jpeg image into some vector images like pdf image (it is blurred when I want to check the image in detail). Another example is the author claims "Mixed-token optimization: 1B model with the Short SI produces no strictly Pareto improving", but claims "*We instantiate the loss across four parameterization landscapes: continuous embedding optimization (Soft), unconstrained discrete vocabulary search (Hard GCG), readabilityconstrained discrete search (HardR), and hybrid suffix adaptation (Mixed), showing that eigenvalue modulation operates consistently across all four." in the contributions. It seems that in "Mixed-token optimization" settings, the performance is not that consisitent to other three settings.
Questions for authors
See weakness.
In addition, why you characterize the proposed loss as "contrastive"? Eq (6) applies different objectives to harmful and harmless inputs, but does not explicitly contrast their representations through an attraction–repulsion or relative-comparison mechanism. I might be a litte confusing to other authors without good explanation on this.
6 comments
Part II
The interpretation of the eigenvalue as instruction strength is not fully convincing: Eq. (5) captures only a projection, and Eq. (7) leaves an angle-dependent residual when . Corollary A.4.1 should distinguish approximate closeness from strict convergence and validate the alignment assumption empirically.
We agree and have made the statement precise. Equation (7) is exact: as , the distance tends to , so reaches the identity only if also tends to zero. The revision states the corollary this way (Appendix A.3). Sections 3.2 and 3.3 now state the alignment condition where they interpret the eigenvalue.
Alignment in practice. The median over all training checkpoints ranges from 0.97 to 1.00 across the four parameterizations, and it drops to 0.78–0.94 in the 5% least aligned checkpoints (Fig. A1). The activations are thus roughly aligned, but not enough to equate the distance with the eigenvalue term: the angle term of Eq. (7) is the larger part of the distance at every checkpoint (Fig. N4), and on harmless queries the operator stays close to the identity without reaching it (median between 0.08 and 0.27). A penalty on the full distance is therefore needed, as Remark 3.1 suggests, and the best-performing of the loss variants above uses one (Fig. M1). The revision reports these values (Appendix A.3).
Eigenvalue and refusal. To test the interpretation directly, we correlated the harmful-query eigenvalue with ASR over all training checkpoints (Fig. N1). A larger eigenvalue goes with a lower ASR in seven of the eight settings (short and long SI; Pearson from −0.17 to −0.70). The exception is HardR on the short SI, where the readable suffix rotates the activation at almost constant norm, so the eigenvalue decreases as the operator moves away from the identity. Splitting the eigenvalue into its norm and angle factors supports the reviewer's point: for Soft, Mixed and Hard GCG on the short SI, stretching along the clean direction and rotating away from it both lower ASR, so the two effects partly cancel inside the eigenvalue. The distance accounts for both and tracks ASR in all four parameterizations on the short SI. The eigenvalue also tracks refusal for DRO, which never optimizes it (Fig. N3). The revision adds these plots (Appendix N), with the correlations of ASR and ORR with (Fig. O1).
Part III
The layer ranges need systematic ablations (depths, single- versus multi-layer objectives, feedforward-input versus block-output activations). The 90.9% versus 64.2% retention comparison is insufficient: candidate pools differ, checkpoints from the same trajectory are correlated, and a common selection rule and independent seeds would help.
Layers. We tuned the layer range of each parameterization on the eval split before the sweep, and the revision reports this sweep in Appendix P (Figs. P1 and P2). With the short SI, every range that we tried, down to three layers, reduced ASR; the range mostly changed over-refusal, in a direction that depends on the parameterization, which is why each parameterization has its own range. A layer-by-layer profile of the trained Soft instructions (Fig. P3) shows that the operator changes little before the first layer of the loss and that the change persists in the last layers, which the loss does not cover. We also ran a controlled ablation on Soft that varies only the layers of the loss (depth of the range, single versus multiple layers), with five repeated runs to measure the variation between runs (Fig. P4). Every setting, including each single layer, lowers the in-training ASR by an amount within the range of the repeated runs, and the change in ORR is within their range for 8 of the 11 settings. In this ablation, we find no effect of the layer choice that we can distinguish from the variation between runs, making the layer range a relatively robust hyper-parameter. We keep the activation site at the input of the first feedforward matrix, the only place where the operator corresponds to a weight update, in line with the implicit update theory of Dherin et al. (2025) and Goldwaser et al. (2026).
Retention. All methods, DRO included, use the same rule: every eval-split Pareto improvement is re-evaluated on the test split, and DRO's pool is larger only because it finds more improvements (137 vs. 11). Counting each training run once, 80% of Soft's improvements hold on test, against 41% for DRO. Both methods still keep at least one test improvement in most of their runs (4 of 5 for Soft, 6 of 8 for DRO), so we agree that the comparison is suggestive rather than conclusive without independent seeds, and the revision says so (Appendix B.5). To account for the correlation between checkpoints of the same run, the revision also compares the methods by the test effect averaged over all their candidates, with intervals from a bootstrap over training runs (Appendix I.5, Fig. I2): on average, DRO lowers ORR far more than Soft, and ASR less.
Presentation: Figure 1 has overlapping words and arrows, Figure 2 should be a vector image, and the contribution claiming that eigenvalue modulation operates consistently across all four parameterizations conflicts with Mixed producing no strict Pareto improvement.
The revision fixes both figures. By "consistently", we meant that controls the ASR–ORR trade-off in all four parameterizations, which holds for Mixed as for the others (correlation with : +0.67 for ASR and −0.62 for ORR; Fig. O1). Pareto improvements are a separate matter: Mixed reaches near-Pareto candidates with the short SI and one Pareto improvement with the long SI. On Gemma 3 4B, 8 of its 16 candidates are Pareto improvements on the test split by point estimate (Table 15). The revision rephrases the contribution to make this distinction.
Questions For Authors
Why call the loss "contrastive"? Eq. (6) applies different objectives to harmful and harmless inputs but does not contrast their representations through attraction–repulsion or relative comparison.
We call it contrastive because it applies opposite objectives to two contrasting sets of queries, with the identity operator (the instruction having no effect) as their common reference: on harmful queries the loss pushes the operator away from the identity (repulsion), and on harmless queries it pulls the operator toward it (attraction). Section 3.3 of the revision states this explicitly.
Ethics flag: the paper includes malicious examples, such as the user prompt "how to make a bomb".
Our harmful examples are of the kind found in standard benchmarks, and the paper shows no harmful completions. The revision adds a content warning to the abstract.
Part I
We thank the reviewer for finding our eigenvalue-based loss "somewhat novel" and "intuitively motivated", for noting the value in that it "can be applied to different prompt searching settings", that our "experiments demonstrate promising results with Pareto improvements”, and that the paper connects its formulation "to existing theoretical and prompt optimization methods properly".
The review rates its soundness concerns as minor or fixable. We address each of them below with new analyses, and we summarize the additions to the revision in our general response; if our answers resolve these concerns, we hope the reviewer will consider raising the score.
Critical Weaknesses
Goal. Our goal is to introduce the safety operators and to show, with the safety eigenvalue as their simplest spectral quantity, that they give a useful control on instruction expression; we do not claim a state-of-the-art algorithm. We nevertheless ran the two controlled comparisons that the reviewer asks for.
Non-spectral baselines. We added vector steering and safety-instruction boosting, run under our protocol (Fig. L1). Neither lowers both rates significantly: none of the 62 steering configurations lowers ASR significantly without raising ORR, and none of the 50 boosting configurations lowers ASR significantly. Our best candidates lower both significantly (Soft: −3.9 pp ASR and −4.0 pp ORR; Hard GCG: −2.8 pp ASR and −3.6 pp ORR). We also ran RPO (Zhou et al., 2024) with a penalty on harmless queries that we add, an output-based loss that balances refusal and helpfulness on discrete suffixes (Appendix B.6, Fig. B1). Of its 64 candidates, 16 lower both test rates by point estimate, none of them significantly, and neither rate decreases significantly on average.
Alternative operator losses. We also compared six losses built on the operator, on Soft and under the same protocol, including the Frobenius-norm suppression of Remark 3.1 (Fig. M1). All six trace the trade-off and produce Pareto improvements, consistent with our claim that the control comes from the operator rather than from one particular loss. The best loss combines a hinged eigenvalue term on harmful queries with the Frobenius suppression on harmless ones (−3.8 pp ASR and −4.6 pp ORR, both significant), whereas the Frobenius suppression paired with the squared-eigenvalue term of our loss ranks last. The revision reports this comparison in Appendix M.
DRO. We agree that DRO's best candidate is stronger than ours and that our comparison does not establish superiority, and the revision tones down our language accordingly (Section 5.2 and Appendix B.5). The DRO result is consistent with our message: an objective defined on the operator alone, without an external probe, also produces Pareto improvements, and it applies to discrete and readable instructions, where DRO does not.