acceptodds
Under review as a conference paper at ICLR 2027

A Jailbreak Success Coordinate Outside the Refusal Subspace: Decodable Before Generation, Reachable from the Input

Abstract

Refusal directions are known to alter a language model's behavior under intervention, but whether they also identify which jailbreak variant will succeed has not been tested: probes for jailbreak success choose their own read-out axis and are never tied to them. In this paper, we ask whether success is decodable before generation from a coordinate outside an evaluated refusal subspace, whether intervention on that coordinate moves the behavior, and whether it can guide input search. Across ten open-weight models, two datasets, and four augmentation families, we compare successful and failed variants of the same harmful request within a family. Using representations at the last prompt position, we estimate success-failure directions on behavior-blocked folds and project out the span of eleven published refusal-direction estimates. The resulting orthogonal direction reaches a median paired AUC of 0.903, against 0.826 for the in-span projection of the same success-fitted direction, and exceeds the best individual refusal direction in 69 of 80 model-dataset-family combinations; with model and family held fixed, cross-dataset transfer reaches 0.872. Intervention oppositely separates the two components: the same projection removal, applied at every layer and position and held through generation, opens the behavior further on the evaluated refusal subspace than on the orthogonal coordinate in nine of the ten models. The coordinate is nevertheless reachable from the input, yielding working suffix, template, and rephrase attacks on the four models of the search grid, with candidate fitness computed from pre-generation activations alone, without generating responses or invoking a response judge. What representation-level intervention acts on does not tell which attack gets through.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, jailbreak, AI safety, refusal direction, representation engineering, red teaming

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tt35OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies whether jailbreak success can be predicted from prompt representations before generation, and whether the predictive direction coincides with directions that mediate refusal. It estimates a success–failure direction and removes its component within a subspace formed by eleven published refusal directions. Experiments across ten models, two datasets, and four attack families show that the remaining direction retains strong predictive performance but has weaker effects under direct intervention.

Critical strengths
  • The paper asks a meaningful question: directions that control refusal may differ from those that predict which jailbreak succeeds. Studying these properties together is a useful contribution.

  • The evaluation is somewhat comprehensive, which covers multiple model families and attack types.

  • The figures and tables seem to be nicely designed.

Critical weaknesses
  • I found the paper very hard to read. The write-up is unnecessarily complicated, and many sentences require considerable effort to understand. I had to use an AI tool to help interpret many of them. The main ideas are interesting, but the write-up needs substantial improvement before publication (if eventually aiming for human readers).

  • The interpretation of "outside the refusal subspace" depends heavily on how that subspace is constructed. The eleven selected directions do not necessarily capture all refusal-related features, and Section 3.2 explicitly excludes additional SOM directions. Although the authors acknowledge this limitation, it remains unclear whether the main result would hold with a broader refusal subspace. The evidence currently establishes separation from the selected directions rather than from refusal mechanisms more generally.

  • I am not fully convinced by the mechanistic interpretation in Section 4.2. The intervention result reaches significance on only two of the four models tested statistically, and the addition experiments are weaker. Moreover, a direction estimated at one late layer is applied across all layers and positions. A weak effect under this particular operation may reflect how the intervention is implemented, rather than a limited role for the predictive feature. The distinction between predictive performance and intervention effects is supported more clearly than a strong separation between the underlying mechanisms.

Questions for authors
  • Does the predictive advantage remain when the refusal subspace includes the additional SOM directions or other broader sets of refusal features?

  • Would the intervention conclusions change if each direction were applied only at its estimation layer, or at a separately selected intervention layer?

6 comments

Response to Reviewer tt35 (1/3)

Overview

Thank you for your detailed review. The Critical Weaknesses consist of three unnumbered paragraphs, so we refer to them in order as W1, W2 and W3, and to the two questions under Questions For Authors as Q1 and Q2. Q1 concerns the same issue as W2 and Q2 the same issue as W3, so we answer them together as [W2/Q1] and [W3/Q2].

We understood your main points to be three. The paper is hard to read, the interpretation of “outside the refusal subspace” depends on how the subspace is constructed, and the mechanistic interpretation in Section 4.2 is not fully convincing. For the first, we will reflect it by improving the writing in the revised manuscript that we will upload during the discussion period. For the second concern, we measured again after widening RR, which is spanned by the 11 published refusal directions, to 27 dimensions by adding the 16 SOM directions, and the effect of removing RR (+0.117, p = 4.2e-04) was maintained. The discrimination information added by the component outside RR shrinks as RR is widened but stays positive even at 27 dimensions. For the third concern, we showed that success moves in the hypothesized direction when the success direction is added at the single layer where the direction was estimated, and that in input search the search that uses the component outside RR as its objective function has a higher success rate than the search that uses the component inside it in all four models. With these results, we have addressed all three points.

[W1] The paper is hard to read

We accept that the paper was hard to read and will improve the writing in the revised manuscript that we will upload during the discussion period. We thank you for finding the main ideas interesting, and we will polish the text of the revision so that these ideas reach readers clearly.

[W2/Q1] The interpretation of “outside the refusal subspace” depends on how that subspace is constructed

We left the SOM directions out of the submission because adding them to RR lowers the share that is specific to refusal, as explained in Section 3.2. This time we re-measured discrimination and intervention with a 27-dimensional subspace made of RR plus the 16 SOM directions; the results are in Table R1. The success direction u^sf\hat u_{sf} is the direction fitted to the difference between the representations of successful and failed augmentations of the same request, and u^sf⊥\hat u_{sf}^{\perp} is that direction with its component in RR removed. We measured discrimination by the gain, which is the increase in AUC when u^sf⊥\hat u_{sf}^{\perp} is added to a classifier fitted on the coordinates of RR. We measured intervention by how much success rises when RR is removed on set A, the original harmful requests.

Table R1. Discrimination and intervention under two definitions of RR. Rows are claims; columns are definitions of RR. Discrimination is the mean gain in paired AUC from adding u^sf⊥\hat u_{sf}^{\perp} to the label-fitted readout on RR's coordinates. Intervention is the ablation effect on set A (the original harmful request) against its magnitude-matched random control, at one layer at relative depth 0.8 from the last prompt position, paired across combinations with a sign test.

ClaimRR = 11 published refusal directionsRR + SOM (27-D)
Discrimination: gain outside RR+0.0296, p 1.5e-16+0.0069, p 1.1e-09
Intervention: ablating RR, set A+0.145, p 4.7e-15+0.117, p 4.2e-04
Share of u^sf\hat u_{sf} squared norm inside RR (mean)0.0750.225

The control in the intervention experiment is a magnitude-matched random control, which moves the representation along a random direction by the same amount as the intervention. Removing RR on the original harmful requests raises success by +0.145 more than this control, while removing only the 16 SOM directions gives a smaller effect, +0.038 (p = 0.033). The effect remains at +0.117 when we widen RR to 27 dimensions. The result therefore does not stop at the 11 selected directions and holds for a broader set of refusal directions.

Response to Reviewer tt35 (2/3)

[W2/Q1] The interpretation of “outside the refusal subspace” depends on how that subspace is constructed (continued)

[Q1] Does the predictive advantage remain when the refusal subspace includes the additional SOM directions or other broader sets of refusal features?

The answer to Q1 is that the advantage remains. With the 27 dimensions that include the SOM directions, the discrimination gain that u^sf⊥\hat u_{sf}^{\perp} adds is still significant at +0.0069, and when RR is widened with PCA it stays significant up to 50 dimensions. The gain is smaller than with 11 dimensions because, as RR grows, the share of the squared norm of u^sf\hat u_{sf} inside RR rises from 0.075 to 0.225, and the share left outside RR shrinks. In the revision we will present this discrimination result as a supporting control. The main conclusion is that intervention, which edits the representation directly, and input search, which changes the input and searches, use different components to change success.

[W3/Q2] The mechanistic interpretation in Section 4.2 is not fully convincing

Compared with the magnitude-matched random control, removing RR by intervention gives p = 4.7e-15 on set A and p = 1.8e-06 on set C, the failed augmentations, across all ten models. It is correct that two of the four models we tested were significant; a single model has at most 8 combinations of dataset and augmentation family, so the smallest p value a sign test can give is 0.0078, and this limit explains the result. When we extend the test of adding u^sf\hat u_{sf} to ten models and test each model separately, the sign test is significant for 5 models and the Wilcoxon test for 7 on set A, and in all ten models the mean success rate is higher than under the random control. The evidence for the effect therefore comes from the consistency across all ten models rather than from individual model tests.

We also examined how the way the intervention is applied affects the effect size. In the submission, we ablated u^sf\hat u_{sf}, estimated at relative depth 0.8, at every layer and every position, and under this setting success rose in only 43 of 77 combinations. Changing only the operation to patching on the same 77 combinations raises this to 65 and the effect from +0.117 to +0.199. Adding u^sf\hat u_{sf} with coefficient 1, from the last prompt token onward, at the single layer where it was estimated, the fitted layer, gives an effect of +0.126 with success rising in 25 of 31 combinations, and coefficient 3 gives +0.204 with 27 combinations rising. These are in Table R2. The effect was therefore unclear because of the way it was applied, and with a different setting the rise in success becomes clear.

Table R2. Effect of u^sf\hat u_{sf} on set A (original harmful request) by application convention. The predicted sign for set A is a rise in success rate. Rows with the same number of combinations are comparable; the 77-combination rows (base success rate 0.148) and the 31-combination rows (base success rate 0.205) are not compared with each other.

ConventionOperationCombinationsEffectCombinations with predicted sign
Relative depth 0.8, all layers and positions (in submission)ablation77+0.11743 / 77
Relative depth 0.8, all layers and positionspatch77+0.19965 / 77
Fitted layer only, from the last prompt positionaddition (coefficient 1)31+0.12625 / 31
Fitted layer only, from the last prompt positionaddition (coefficient 3)31+0.20427 / 31

Response to Reviewer tt35 (3/3)

[W3/Q2] The mechanistic interpretation in Section 4.2 is not fully convincing (continued)

In the same setting we also added u^sf⊥\hat u_{sf}^{\perp}. Under addition with coefficient 1, both u^sf\hat u_{sf} and u^sf⊥\hat u_{sf}^{\perp} raise success on sets A and C and lower it on set B, the successful augmentations; in every case they exceed the random control, and no combination produced broken output. The results are in Table R3.

Table R3. Addition at coefficient 1, at the fitted layer, from the last prompt position (four models, 31 combinations). Set A: original harmful request; set B: successful augmentation (predicted sign: success drops); set C: failed augmentation. The effect is the change in success rate, with the number of combinations that move in the predicted sign in parentheses. The random direction has the same norm as the added direction, and the sign test is paired across combinations against it.

SetDirectionEffect (combinations with predicted sign)Magnitude-matched random directionSign test
Au^sf\hat u_{sf}+0.126 (25/31)+0.0024.9e-05
Au^sf⊥\hat u_{sf}^{\perp}+0.067 (25/31)−0.0053.2e-04
Bu^sf\hat u_{sf}−0.223 (27/31)−0.0435.2e-03
Bu^sf⊥\hat u_{sf}^{\perp}−0.154 (28/31)−0.0158.4e-06
Cu^sf\hat u_{sf}+0.075 (22/31)−0.0101.0e-04
Cu^sf⊥\hat u_{sf}^{\perp}+0.056 (24/31)−0.0091.5e-05

We agree that prediction and intervention must be distinguished, and our conclusion rests on that distinction. Discrimination alone cannot tell whether intervention and input search use different components; the intervention and input search experiments can. In the intervention experiment, removing RR raises success. In the input search experiment, a search whose objective is the component outside RR, u^sf⊥\hat u_{sf}^{\perp}, has a higher success rate than a search using the component inside RR in all four models, and the difference is significant in three.

[Q2] Would the intervention conclusions change if each direction were applied only at its estimation layer, or at a separately selected intervention layer?

Adding u^sf\hat u_{sf} and u^sf⊥\hat u_{sf}^{\perp} with coefficient 1 at the single layer where each was estimated raises success on the original harmful requests by +0.126 and +0.067, with p = 4.9e-05 and 3.2e-04. Removing RR at one separately chosen layer, relative depth 0.8, applied alike across models, gives +0.145 over the magnitude-matched random control (p = 4.7e-15). In the revision we will use the single-layer setting as the default in the main text. The intervention conclusion therefore holds both at the estimation layer and at a separately chosen layer.

Conclusion

After running the checks you asked for, our original claim stands: the success coordinate lies outside the refusal subspace and can be reached from the input, and intervention and input search use different components to change success. Discrimination and intervention hold when RR is widened to 27 dimensions, and whether we apply the direction at the layer where it was estimated or at a separately chosen layer, adding u^sf\hat u_{sf} and u^sf⊥\hat u_{sf}^{\perp} moves success in the direction our hypothesis predicts. In the revised manuscript that we upload during the discussion period, we will restructure Sections 4.1, 4.2 and 5 of the main text around this conclusion and make them easier to read, so that you can check it.

We hope this reply addresses your concerns, and please let us know if you need any further explanation.

Reviewer_qsBXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether jailbreak success can be predicted from representations outside the refusal subspace. The authors construct a success-failure direction by comparing successful and failed jailbreak variants of the same harmful request, and orthogonalize it against a subspace formed by 11 existing refusal directions. Experiments across 10 models, two datasets, and four attack families show that the resulting direction achieves strong jailbreak success discrimination (median AUC of 0.903), while interventions on refusal directions generally have stronger effects on model behavior. Furthermore, the authors demonstrate that this direction can guide jailbreak optimization without response generation or judging during the search process.

Critical strengths
  1. Interesting Research Question and Perspective.

The paper investigates an important distinction between the representations that mediate refusal and those that predict jailbreak success. This perspective provides useful insights into the limitations of relying solely on refusal directions to understand and evaluate jailbreak vulnerabilities.

  1. Extensive Experimental Evaluation.

The experiments cover 10 open-weight models, two benchmark datasets, and four jailbreak attack families. The authors also conduct behavior-blocked evaluations, cross-dataset transfer experiments, and comparisons against existing refusal directions and surface-based baselines, providing substantial empirical support for their observations.

  1. Practical Application to Jailbreak Optimization.

Beyond representation analysis, the paper demonstrates that the identified success direction can serve as an effective optimization objective across suffix, template, and rephrasing attacks. The ability to guide search using pre-generation activations without generating responses or invoking a response judge offers a potentially efficient approach to automated red teaming.

Critical weaknesses

1. Limited Mechanistic Understanding of the Success Direction.

Although the paper identifies a direction that effectively discriminates successful from failed jailbreaks, it remains unclear what this direction actually represents. The method essentially constructs a supervised linear probe using success/failure labels, and its strong predictive performance does not necessarily indicate the discovery of a distinct jailbreak mechanism. Moreover, optimizing inputs along this direction simultaneously changes other representations, making it difficult to establish its causal role in jailbreak success.

2. Incomplete Justification of the Refusal Subspace.

The refusal subspace is constructed from 11 previously identified refusal directions, but these directions do not necessarily capture the full refusal mechanism. The paper's central conclusion therefore depends heavily on how this subspace is defined. In particular, the appendix shows that incorporating additional directions substantially increases the proportion of attack-induced displacement captured by the subspace. The authors should examine whether their discrimination and intervention findings remain robust under alternative, more comprehensive definitions of the refusal subspace.

3. Insufficient Evidence for a Unified Success Direction Across Attack Families.

The paper constructs a success direction by aggregating success-failure differences across four attack families, but it does not sufficiently establish whether these families share a common underlying representation of jailbreak success. Different attacks may exploit fundamentally different vulnerabilities, and averaging their representations could obscure family-specific mechanisms. Although cross-dataset transfer is evaluated, the attack family remains fixed. Cross-family direction similarity and leave-one-family-out transfer experiments would be important for validating the claimed generality of the proposed coordinate.

4. Potential Bias Introduced by Failure-Sample Filtering.

The authors remove approximately 24% of the original failure samples through their empty-compliance filtering procedure. However, responses that do not fulfill the original harmful intent can still reasonably be considered unsuccessful jailbreak attempts, even when they do not explicitly refuse. Excluding these cases changes the failure distribution used to estimate the success-failure direction and may artificially simplify the discrimination task. The paper lacks sufficient ablation studies examining how this filtering affects the learned direction, predictive performance, and main conclusions.

Questions for authors

see weakness

4 comments

Response to Reviewer qsBX (1/3)

Overview

Thank you for your detailed review. We organize our reply by the numbering of your review, W1 to W4.

We understood your main points to be four. The understanding of what the success direction represents is limited at the mechanistic level, the definition of RR is not sufficiently justified, the evidence for a single success direction across attack families is insufficient, and the filtering of failure samples may introduce bias. For the first concern, we did not rely on discrimination performance alone to claim a mechanism and confirmed it with intervention and input search. In the intervention, removing RR raises the success rate by +0.145 over a random direction of the same magnitude (p = 4.7e-15), and in input search, the search that uses the component outside RR as its fitness has a higher success rate than the search that uses the component inside it in all four models, so the two methods use different components to change success. For the second concern, we measured again with a 27-dimensional RR that adds the 16 SOM directions, and the effect of the intervention was maintained at +0.117 (p = 4.2e-04). For the third concern, we showed that the median AUC of a direction fitted without one family and tested on that family is 0.8426, and for the fourth, that the median AUC is between 0.8662 and 0.9104 however the empty-compliance filter is applied. With these results, we have addressed all four points.

[W1] Limited Mechanistic Understanding of the Success Direction

W1 contains two separate points, so we split it into (a) and (b).

(a) What the direction represents, and the predictive performance of the supervised probe

We have two pieces of evidence on what this direction represents. First, in the intervention, removing RR raises success on the original harmful requests by +0.145 more than a magnitude-matched random control, which moves the representation along a random direction by the same amount as the intervention (p = 4.7e-15). Second, adding u^sf⊥\hat u_{sf}^{\perp} at the single layer where the direction was estimated raises success on the original harmful requests by +0.067, while the random control changes it by −0.005 (p = 3.2e-04). What we claim is therefore not a name for the mechanism but the scope established by intervention and search. Intervention, which edits the representation directly, and input search, which changes the input and searches, use different components to change success.

We agree that discrimination performance alone cannot establish a separate mechanism. As Table R1 shows, once label supervision is given, the coordinates of RR alone reach an AUC of 0.8883, which is not significantly different from the 0.8958 of u^sf\hat u_{sf}, and the fully supervised probe reaches 0.9336. Our claim therefore rests not on discrimination but on the fact that intervention and input search use different components. Even for discrimination, adding u^sf⊥\hat u_{sf}^{\perp} to the coordinates of RR raises the AUC from 0.8883 to 0.9179 (p = 1.5e-16), so information about success also lies outside RR. u^sf⊥\hat u_{sf}^{\perp} discriminates better than the component of u^sf\hat u_{sf} inside RR, with a mean AUC of 0.8959 against 0.8181, and the fully supervised probe, whose cosine with u^sf⊥\hat u_{sf}^{\perp} is +0.079 and which is thus nearly orthogonal to it, still separates success. Discrimination is therefore not tied to one specific direction, and the component outside RR accounts for success.

Table R1. Paired discrimination (behavior-blocked mean AUC over the model-dataset-family combinations, relative depth 0.8) of u^sf\hat u_{sf} and the objects it is compared with, and the cosine of each direction with u^sf⊥\hat u_{sf}^{\perp}. Rows are objects; readouts are fitted with the success label on a subspace's coordinates.

ObjectPaired discrimination (AUC)cos with u^sf⊥\hat u_{sf}^{\perp}
u^sf\hat u_{sf}0.89580.815 to 0.998
u^sf⊥\hat u_{sf}^{\perp} (RR projected out)0.89591.000
ΠRu^sf\Pi_R\hat u_{sf} (in-span part)0.81810.000
RR (11 published directions), label-fitted readout0.8883—
RR + u^sf⊥\hat u_{sf}^{\perp}, label-fitted readout0.9179—
PCA of harmful minus harmless, k = 100, label-fitted readout0.9440—
Full supervised probe (displacement difference)0.9336+0.079
Harmfulness direction (Zhao et al.)0.7766−0.351

Response to Reviewer qsBX (2/3)

[W1] Limited Mechanistic Understanding of the Success Direction (continued)

(b) Input optimization also changes other representations

Because input search can also move other representations, we split the same direction into components and built a separate fitness from each. With a fitness built only from the component inside RR, success on the suffix surface ranges from 0.295 to 0.525; with one built from the component orthogonal to RR, it ranges from 0.560 to 0.860. In three of the four models, a test paired at the behavior level gives p = 8.1e-16, 2.1e-19 and 2.6e-05. So even if other representations move together with it, the component that drives success in input search is the one orthogonal to RR.

[W2] Incomplete Justification of the Refusal Subspace

We left the SOM directions out of the submission because adding them to RR lowers the share that is specific to refusal, as explained in Section 3.2. This time we re-measured discrimination and intervention with the 27-dimensional RR that includes the SOM directions; the results are in Table R2.

Table R2. Discrimination and intervention under two definitions of RR. Rows are claims; columns are definitions of RR. Discrimination is the mean gain in paired AUC from adding u^sf⊥\hat u_{sf}^{\perp} to the label-fitted readout on RR's coordinates. Intervention is the ablation effect on set A (the original harmful request) against its magnitude-matched random control, at one layer at relative depth 0.8 from the last prompt position, paired across combinations with a sign test.

ClaimRR = 11 published refusal directionsRR + SOM (27-D)
Discrimination: gain outside RR+0.0296, p 1.5e-16+0.0069, p 1.1e-09
Intervention: ablating RR, set A+0.145, p 4.7e-15+0.117, p 4.2e-04
Share of u^sf\hat u_{sf} squared norm inside RR (mean)0.0750.225

The gain, the AUC increase when u^sf⊥\hat u_{sf}^{\perp} is added to a classifier fitted on the coordinates of RR, remains significant with 27 dimensions. It is smaller than with 11 dimensions because, as RR grows, the share of the squared norm of u^sf\hat u_{sf} inside RR rises from 0.075 to 0.225 and the share left outside shrinks. In the intervention, too, removing RR keeps its effect at +0.117 with 27 dimensions. Removing only the 16 SOM directions gives +0.038 on the original harmful requests (p = 0.033), smaller than the +0.145 from removing RR. Hence the conclusions on discrimination and intervention also hold under a more comprehensive definition of RR.

[W3] Insufficient Evidence for a Unified Success Direction Across Attack Families

The median cosine between the directions of different families is +0.397, about 33 times that of random direction pairs, and the cosine for the same family across datasets stays at +0.942. In all 20 combinations of model and dataset, the cosine within a family is higher than between families (p = 1.9e-06), so each family has its own component while a large component is shared.

Table R3. Cosine between directions (ten models, two datasets). Rows are comparisons; columns are the number of pairs, the median and the range.

ComparisonPairsMedianRange
Between families, same model and dataset120+0.397−0.133 to +0.818
Within a family, fold pairs800+0.996+0.947 to +1.000
Same family, across datasets40+0.942+0.720 to +0.996
Random direction pairs, absolute value20+0.012+0.009 to +0.020

The leave-one-family-out AUC has a median of 0.8426 over the 80 combinations. By the mean over combinations, the direction fitted without LJS discriminates the augmentations made by LJS with an AUC of 0.808, while a control with shuffled labels gives 0.502; the values are 0.910 for AdvPrompter, 0.904 for translation and 0.676 for PAP. The three families other than PAP reach 0.8 or higher, so the direction contains a success component shared across families and datasets, and it discriminates success even in a family it has not seen.

Response to Reviewer qsBX (3/3)

[W4] Potential Bias Introduced by Failure-Sample Filtering

The 24% you mention matches the value of 0.2405 reported in the appendix of the paper, and the filter is the procedure that removes empty-compliance responses from the failure samples. We checked with human labels whether the responses removed by the filter are really failures. Two of the authors read the responses and judged whether each one carried out the request, and we sampled responses that the judges had labeled as success, empty compliance and refusal. The share that humans judged to be successes is 0.136 among empty-compliance responses, 0.003 among refusals and 0.820 among successes. By human judgment, 86% of the responses the filter removes are not successes, so the filter removes real failures.

We re-measured discrimination under different ways of applying the filter. The AUC below is the accuracy with which the projection onto u^sf\hat u_{sf} picks the successful augmentation out of a pair of a successful and a failed augmentation of the same request. When the direction is fitted without the filter, the median AUC is 0.8926; when the filter is left out at evaluation, 0.8662; and when the filter is also applied to successes, 0.9104, which is not far from the 0.9019 of our main setting. These are in Table R4. We report the values with and without the filter, and in Section 5 of the revised manuscript that we will upload during the discussion period we will also state the rule used for the absolute success rates. The concern that the filter made the direction and the discrimination easier therefore does not apply, and the main conclusions hold when the filter is changed.

Table R4. Paired discrimination under different handling of the empty compliance filter (median AUC over the 80 model-dataset-family combinations; behavior-blocked, relative depth 0.8).

ConditionMedian AUC
Main setting (filter applied to failures)0.9019
Filter not applied at fitting0.8926
Filter not applied at evaluation0.8662
Filter not applied at both fitting and evaluation0.8678
Filter applied to successes as well0.9104

Conclusion

After running the analyses you asked for, our original claim stands: the success coordinate lies outside the refusal subspace and can be reached from the input, and intervention and input search use different components to change success. The results for discrimination and intervention held when we widened the definition of RR, when we moved to families not seen in fitting, and when we changed the filter. In the revised manuscript that we upload during the discussion period, we will restructure Sections 4.1, 4.2 and 5 of the main text around this conclusion, so that you can check it.

We hope this reply addresses your concerns, and please let us know if you need any further explanation.

Dear Reviewer qsBX,

Thank you for your response. We are glad that your concerns have been largely resolved. We have now uploaded the revised paper. Red marks new experiments and statements updated to reflect them, and navy marks wording-only edits. The results from our response are in these places:

W1. (a) Added at the layer where it was estimated, u^sf⊥\hat{u}_{sf}^{\perp} raises success on the original harmful requests by +0.067, against −0.005 for a random direction of the same norm (p = 3.2e-04; Section 4.2, Table 11). (b) On HarmBench standard suffixes, the fitness from the orthogonal component is higher than the one from the in-span component in all four models, significantly so in three (Section 5.1, Table 21).

W2. With the 27-dimensional R that includes the SOM directions, the AUC gain stays significant (+0.0069, p = 1.1e-09) and the intervention effect is maintained (+0.117, p = 4.2e-04). See Tables 11 and 16.

W3. The leave-one-family-out median AUC is 0.8426 (Table 15), and the median cosine between the directions of different families is +0.397, about 33 times that of random pairs (Table 18).

W4. Under four alternative handlings of the filter the median AUC is 0.8662 to 0.9104, against 0.9019 in the main setting (Table 5). The success label used for the rates of Section 5 is stated in the caption of Table 2 and in Appendix F.1.

If you find that the revision strengthens the paper, we would be grateful if you could reflect that in your final assessment.

Reviewer_a9eXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether jailbreak success is decodable, intervenable, and reachable on a coordinate that lies outside known refusal directions. Across 10 open-weight models, two datasets and four augmentation families (about 1M judged pairs), the authors fit a “success–failure” mean-difference direction. They fit it on displacements (variant minus original request) at the last prompt token, at relative depth 0.8, on behavior-blocked folds. They then project out the span R of eleven published refusal-direction estimates (one difference-in-means direction, five cone and five representational-independence directions) and keep the orthogonal remainder û⊥. In within-request pairs, û⊥ reaches a median AUC of 0.903, versus 0.826 for its in-span part, and exceeds the best of the eleven refusal directions in 69 of 80 cells. Removing projections of R at every layer and position opens the behavior more than removing û⊥ in 9 of 10 models. Using the projection as a search fitness, with no generation and no judge in the loop, gives working suffix, template and rephrase attacks.

Critical strengths
  • This is a large, careful measurement. It uses request-fixed pairing, behavior-blocked folds, magnitude-matched random controls, variant-slot and character n-gram baselines, slot-blocked evaluation and cross-dataset transfer (0.872). Failures are reported openly: the intervention null on Mistral-7B, the sign flip on Gemma-3-1B, degenerate cells and a limitations section. Testing discrimination and mediation jointly on one coordinate is a clean question for representation-level safety.

  • The potential impact is practical. The ranking of candidate prompts at one forward pass, without a generation or a judge, would cut the main cost of automated red-teaming. The paper also warns that restoring refusal along known axes does not say which attacks still get through.

Critical weaknesses

a. “Outside the refusal subspace” is weaker than it sounds. R is the span of 11 published estimates, an 11-dimensional subspace of thousands of dimensions. û⊥ is a direction fitted to the success label, and it is nearly identical to the unprojected direction (0.9026 vs 0.9019). Beating label-free refusal directions with a supervised direction is expected. Even the in-span part of the same supervised direction (ΠRû_sf) beats the best individual axis in 37/80 cells, so much of the “beats refusal directions” result is supervision, not subspace. R’s size and construction matter and are not varied: the SOM axes were excluded after they raised the covered share, and there is no larger or PCA-based R, no random 11-D control for the in-span share, and no harmfulness-direction baseline (Zhao et al.). Labels also make û⊥ refusal-adjacent by construction: success is the OR of two judges, and 24% of the failure pool (hollow compliance) is removed on the failure side only. Controls cover slot identity but not length, language, perplexity or judge errors. Directions are fitted per family, and cross-family transfer is left as future work, although the practical claim concerns novel prompts.

b. The intervention and search evidence do not support “dissociation” or “reachable”. Zeroing the absolute coordinate at every layer and position of a direction fitted at one layer (the paper says directions rotate across layers) favors refusal directions. For a displacement-difference direction, steering by addition is the natural test, and it is not run. The effect is a sign count in 9/10 models, significant in 2 of the 4 models tested; it fails on Mistral and is unmeasurable on Gemma-3-4B. In the search, the key control is missing. The in-span success direction (ΠRû_sf) is never used as a fitness, so “outside R” is not tested for search. Baselines are label-free refusal directions, and the suffix baselines stop early (74–100% of runs stop before 150 steps, and the rule does not track success). The template seeds appear to overlap the LJS templates used to fit the direction. The rephrase surface reports no intent-preservation or human-validated success. The “success tracks the push” correlation uses only 7–10 settings per cell, with the û⊥ search as the high-leverage point. It falls under controls (suffix median 0.84 to 0.52, negative in Mistral cells). On novelty and applicability, pre-generation decodability and representation-objective attacks are established (Kirch et al. and Lin et al., both cited). The method is white-box, covers 1–14B models, needs about 1M judged pairs to fit, and has no compute comparison with generation-based red-teaming. Dual-use attack code is to be released.

Questions for authors
  1. Do the AUC gap and the 69/80 result survive a larger or alternative R (adding the SOM axes, PCA of the harmful/harmless difference at k = 11, 27, 50, 100) and a random 11-D span? What fraction of û_sf’s norm lies in R versus a random subspace?

  2. Can you add the in-span success direction (ΠRû_sf), a full supervised probe, and a harmfulness direction (Zhao et al.) as discriminators and as search fitnesses, and report their cosines with û⊥?

  3. What is the AUC for a direction fitted on three families and tested on the fourth, and how many labeled variants are needed? Can you report steering by addition of û⊥ to failed variants and original requests, at the fitted layer and position, and per-model significance on all ten models?

4/ Can you report length, language-tier and perplexity controls, the effect of applying the empty-compliance filter to successes as well, human-validated success and intent preservation for searched prompts, and whether the 77 seed templates overlap the 78 LJS templates?

  1. At which layer were the 11 directions estimated, and for all ten models? What is the compute cost of a search versus GPTFuzz?
5 comments

Response to Reviewer a9eX (1/4)

Overview

Thank you for your detailed review. We organize our reply by the numbering of your review, Wa, Wb and Q1 to Q5.

We understood your main points to be two: the conclusion “outside the refusal subspace” leans on the definition of RR and is weaker than it sounds, and the evidence from intervention and input search is not enough to support “dissociation” and “reachable”. For the first concern, we measured again after widening RR to 27 dimensions by adding the 16 SOM directions, and the success direction u^sf\hat u_{sf} still beats the best single refusal direction by +0.0517 on average (p = 3.2e-12), and the effect of removing RR (+0.117) was maintained. The discrimination information added by the component outside RR shrinks as RR is widened but stays positive even at 27 dimensions. For the second concern, we re-measured the effect under different ways of applying the intervention, and in input search we showed that the fitness from the component outside RR gives a higher success rate than the one from the component inside it in all four models, significantly so in three. Removing RR raises success more in the intervention and the component outside RR does so in input search, so the two methods use different components to change success.

[Wa] “Outside the refusal subspace” is weaker than it sounds

As you point out, u^sf⊥\hat u_{sf}^{\perp} and the direction before projection, u^sf\hat u_{sf}, discriminate almost equally well. This is because RR accounts for only 0.075 of the squared norm of u^sf\hat u_{sf} on average. u^sf⊥\hat u_{sf}^{\perp} has exactly zero component inside RR, so it isolates the effect outside RR, and because discrimination does not drop when RR is removed, most of the components that the success direction relies on lie outside RR.

Your observation that label supervision raises discrimination, and that ΠRu^sf\Pi_R\hat u_{sf} beats the best single refusal direction in 37 of the 80 combinations, is also correct. Our claim is not that RR contains no information about success, but that RR does not explain all of it. Adding u^sf⊥\hat u_{sf}^{\perp} to a classifier fitted on the coordinates of RR raises the AUC from 0.8883 to 0.9179; this increase, which we call the gain, is +0.0296 (p = 1.5e-16). In addition, u^sf⊥\hat u_{sf}^{\perp} discriminates better than the in-span component ΠRu^sf\Pi_R\hat u_{sf}, and the fully supervised probe reaches a discrimination of 0.9336 although only 0.5% of its norm lies inside RR. Success can therefore be discriminated from coordinates outside RR as well.

Table R1. Paired discrimination (behavior-blocked mean AUC over the 80 model-dataset-family combinations, relative depth 0.8). Rows are objects; readouts are fitted with the success label on a subspace's coordinates. The cosine is the mean cosine with u^sf⊥\hat u_{sf}^{\perp}.

ObjectPaired discrimination (AUC)cos with u^sf⊥\hat u_{sf}^{\perp}
u^sf\hat u_{sf}0.89580.961
u^sf⊥\hat u_{sf}^{\perp} (RR projected out)0.89591.000
ΠRu^sf\Pi_R\hat u_{sf} (in-span part)0.81810.000
RR (11 published directions), label-fitted readout0.8883—
RR + u^sf⊥\hat u_{sf}^{\perp}, label-fitted readout0.9179—
RR + SOM (27-D), label-fitted readout0.9257—
PCA of harmful minus harmless, k = 11 / 27 / 50 / 100, label-fitted readout0.9074 / 0.9278 / 0.9376 / 0.9440—
Full supervised probe (displacement difference)0.9336+0.079
Harmfulness direction (Zhao et al.)0.7766−0.351

[Wb] The intervention and search evidence do not support “dissociation” or “reachable”

In the submission we ablated u^sf\hat u_{sf}, estimated at relative depth 0.8, at every layer and position, and under this setting success rose in 43 of 77 combinations. Changing only the operation to patching on the same combinations raises this to 65, and adding u^sf\hat u_{sf} with coefficient 1 from the last prompt token onward at the single layer where it was estimated, the fitted layer, raised success in 25 of 31 combinations. The unclear effect was therefore due to the application setting, and it becomes clear under a different setting.

Table R2. Effect of u^sf\hat u_{sf} on set A (the original harmful request) by application convention. The predicted sign for set A is a rise in success rate. Only rows with the same number of combinations are comparable (base success rate 0.148 for 77 combinations, 0.205 for 31).

ConventionOperationCombinationsEffectCombinations with predicted sign
Relative depth 0.8, all layers and positions (in submission)ablation77+0.11743 / 77
Relative depth 0.8, all layers and positionspatch77+0.19965 / 77
Fitted layer only, from the last prompt positionaddition (coefficient 1)31+0.12625 / 31

Response to Reviewer a9eX (2/4)

[Wb] The intervention and search evidence do not support “dissociation” or “reachable” (continued)

Compared with a magnitude-matched random control, which moves the representation along a random direction by the same amount as the intervention, removing RR by intervention gives +0.145 more success on set A, the original harmful requests (p = 4.7e-15), and p = 1.8e-06 on set C, the failed augmentations, across all ten models. When we re-ran the suffix baselines with the budget matched at 150 steps, they improved by up to +0.160, but the ordering is the same in all 12 comparisons and the smallest margin is +0.050, so early stopping did not create our advantage.

Kirch et al. fitted probes on the final-token representation of the given prompt to predict success, and Lin et al. added to existing attacks a term along a direction defined by the contrast between harmful and harmless prompt representations. Both use the representation of a single prompt as it is, so the content of the request and the change made by the attack are mixed together, and neither asks whether the direction lies inside the refusal subspace or whether intervention and input search use the same component. We work with the displacement, which is how far the representation moves from the original request to the augmentation, and pair successful and failed augmentations within the same request, so the content of the request cancels and the success direction is fitted from the effect of the attack alone. We show that this direction lies outside RR, and that intervention, which edits the representation directly, and input search, which changes the input and searches, use different components.

The values you point to are exactly those in the appendix table. The correlation uses 7 or 10 settings per model-dataset combination as its points, so we read its size as the median over combinations. For suffix, the median is 0.839 without controls and remains positive at 0.522 when both controls are applied, with 5 of 8 combinations at 0.3 or above. This correlation does not say that the push is the only cause of success; it says that settings that push the projection further reach higher success rates. The u^sf⊥\hat u_{sf}^{\perp} search is designed to increase the projection and contributes strongly to the correlation, so the conclusion of Section 5 rests not on this correlation but on the budget-matched comparison.

Answer to Questions

[Q1] Do the AUC gap and the 69/80 result survive a larger or alternative R … and a random 11-D span? What fraction of û_sf’s norm lies in R versus a random subspace?

The 69/80 result holds. The 80 combinations are combinations of model, dataset and augmentation family; u^sf⊥\hat u_{sf}^{\perp} beats the best single refusal direction in 69 of them, and u^sf\hat u_{sf} is ahead of that direction by +0.0768 on average (p = 3.2e-12). With the 27-dimensional RR that includes the 16 SOM directions, it is still ahead by +0.0517 on average (p = 3.2e-12). With 27 dimensions the gain falls to +0.0069 but remains significant (p = 1.1e-09), because the share of the squared norm of u^sf\hat u_{sf} inside RR rises from 0.075 to 0.225, and the effect of removing RR stays at +0.117. The discrimination with the PCA-widened subspaces is in Table R1; it is higher because these are data-fitted multi-dimensional classifiers whose subspaces contain 47% to 63% of the squared norm of u^sf\hat u_{sf}, against 7.5% for RR. The discrimination of u^sf\hat u_{sf} with a random 11-dimensional subspace removed is 0.8994, at the same level as the 0.8959 obtained by removing RR.

Of the squared norm of u^sf\hat u_{sf}, 92.5% lies outside RR. The share of RR is 10.3 times that of a random 11-dimensional subspace under a uniform orthogonal null, whereas under a null drawn within the subspace where the displacement lies, the share of RR is 0.0284, below the 0.3906 of a random subspace. The advantage over the best single refusal direction therefore survives a larger RR and a random subspace.

Response to Reviewer a9eX (3/4)

Answer to Questions (continued)

[Q2] Can you add the in-span success direction (ΠRû_sf), a full supervised probe, and a harmfulness direction (Zhao et al.) as discriminators and as search fitnesses, and report their cosines with û⊥?

As discriminators, all three are in Table R1, and their cosines with u^sf⊥\hat u_{sf}^{\perp} are 0.000 for ΠRu^sf\Pi_R\hat u_{sf}, +0.079 for the fully supervised probe and −0.351 for the harmfulness direction. As search fitnesses, we compared ΠRu^sf\Pi_R\hat u_{sf} and u^sf⊥\hat u_{sf}^{\perp}; the results are in Table R3. In all four models the u^sf⊥\hat u_{sf}^{\perp} fitness gives the higher success rate, significantly so in three, and the rephrase surface shows the same direction, with p = 6.7e-03 for a test pooled over behaviors. The success rate with the fully supervised probe as the fitness is 0.425 to 0.840 on suffix and 0.640 to 0.869 on rephrase across models, and with the harmfulness direction it is 0.125 to 0.425 on suffix and 0.575 to 0.695 on rephrase, so both work as fitnesses. Using the component outside RR as the fitness therefore gives a higher success rate than using the component inside it.

Table R3. Suffix-surface search with orthogonal versus in-span fitness (150 steps, HarmBench-standard, 200 behaviors; exact McNemar test on behavior-level pairs).

Modelu^sf⊥\hat u_{sf}^{\perp} fitnessΠRu^sf\Pi_R\hat u_{sf} fitnessp
Qwen2.5-7B0.6750.2958.1e-16
Mistral-7B0.8600.4802.1e-19
Llama-3.1-8B0.5600.3802.6e-05
Gemma-3-4B0.5600.5254.6e-01

[Q3] What is the AUC for a direction fitted on three families and tested on the fourth, … steering by addition of û⊥ to failed variants and original requests, at the fitted layer and position, … per-model significance on all ten models? … how many labeled variants are needed?

The AUC of a direction fitted on three families and tested on the fourth has a median of 0.8426 over the 80 combinations. By held-out family, the mean AUC is 0.808 for LJS, 0.910 for AdvPrompter, 0.904 for translation and 0.676 for PAP, and a control with shuffled labels on LJS gives 0.502.

We measured addition with coefficient 1, from the last prompt token onward at the fitted layer, on set A (the original harmful requests), set B (the successful augmentations) and set C (the failed augmentations). The predicted direction is for success to rise on sets A and C and to fall on set B; u^sf⊥\hat u_{sf}^{\perp} moves in this direction on all three sets, exceeds the random control, and no combination produced broken output. Extending to ten models and testing the set A addition of u^sf\hat u_{sf} model by model, the sign test is significant for 5 models and the Wilcoxon test for 7, and in all ten models, including Mistral, the mean success rate is higher than under the random control. On set B the sign test is significant for 3 models and the Wilcoxon test for 4, and on set C both tests are significant for 2 models. A single model has at most 8 combinations, so a sign test cannot give a p value below 0.0078; the evidence for the effect is therefore the consistency across models. Even when we reduce the labels to one tenth, the AUC is 0.8953, against 0.9019 with all labels. The direction therefore discriminates success in unseen families, the addition effect holds across models and sets, and few labels are needed.

Table R4. Addition at coefficient 1 at the fitted layer from the last prompt position (four models, 31 combinations). The effect is the change in success rate (combinations with predicted sign in parentheses); the random control adds a random direction of the same norm, and the sign test is paired against it.

SetDirectionEffect (combinations with predicted sign)Random controlSign test
Au^sf\hat u_{sf}+0.126 (25/31)+0.0024.9e-05
Au^sf⊥\hat u_{sf}^{\perp}+0.067 (25/31)−0.0053.2e-04
Bu^sf\hat u_{sf}−0.223 (27/31)−0.0435.2e-03
Bu^sf⊥\hat u_{sf}^{\perp}−0.154 (28/31)−0.0158.4e-06
Cu^sf\hat u_{sf}+0.075 (22/31)−0.0101.0e-04
Cu^sf⊥\hat u_{sf}^{\perp}+0.056 (24/31)−0.0091.5e-05

Comments on the paper

2 comments

Rebuttal Summary and Revision Plan

We sincerely thank all three reviewers for their constructive feedback and valuable suggestions. We have carefully addressed their concerns and posted a detailed, point-by-point response as an official comment under each review, and we will upload the revised manuscript during the discussion period.

Our additional analyses confirm that the paper's core claims and contributions hold: the success coordinate lies outside the evaluated refusal subspace, is decodable before generation, and is reachable from the input. Below is an overview of how we addressed each reviewer's comments.

1. Summary of our responses

  1. Does the conclusion depend on the definition of R? (a9eX, qsBX, tt35): We measured again after widening R, which is spanned by the 11 published refusal directions, to 27 dimensions by adding the 16 SOM directions. The success direction beats the best single refusal direction by +0.0517 in AUC on average, and removing R raises the success rate over a random direction of the same magnitude by +0.117. The discrimination information that the component outside R adds shrinks as R is widened, but stays positive even at 27 dimensions with the SOM directions added. Revision: Sections 4.1 and 4.2.
  2. Do intervention and input search use different components? (a9eX, qsBX, tt35): In the intervention, removing R raises the success rate over a random direction, and changing how it is applied (patch, or addition at the single layer where the direction was estimated) makes the effect of the success direction clear; the model-level tests are significant in 5 of 10 models with the sign test and in 7 with the Wilcoxon test. In input search, the search that uses the component outside R as its objective function has a higher success rate than the search that uses the component inside R in all four models and significantly so in three (for example, 0.675 versus 0.295 for Qwen2.5-7B), and the component that drives success is the one outside R even when other representations move together. Revision: Sections 4.2 and 5.
  3. Generalization to an unseen attack family and filter bias (a9eX, qsBX): The median AUC of a direction fitted without one family and tested on that family is 0.8426, and it is between 0.8662 and 0.9104 when the way the filter is applied is changed or when length, language, and perplexity are controlled. 86 percent of the responses the filter removes were not successes by human judgment. Revision: Section 4.1.
  4. Other directions and the difference from prior work (a9eX): We reported the in-R component, a fully supervised probe, and a harmfulness direction as discriminators and as the objective function of the input search, and in the search we compared the in-R component directly with the component outside R. Kirch et al. and Lin et al. use the representation of a single prompt, whereas we use the displacement within a request. In the human validation on the rephrase surface, our method's success rate is also higher than those of RDO and DIM. Revision: related work, discussion, and Section 5.
  5. Readability (tt35): We will improve the writing in the revised manuscript.

2. Revision plan

The revised manuscript will include the following changes.

  1. The paper will be reorganized around the contrast between representation intervention and input search.
    • Sections 4.1 and 4.2: The conclusions that success is decoded from a component outside the refusal subspace and that removing the refusal subspace raises the success rate are the same as in the submission.
    • The two results of Sections 4.1 and 4.2 will be grouped as representation intervention and contrasted with the input search in Section 5.
    • Section 5: A comparison with a search that uses the component inside the refusal subspace as its objective function will be added.
    • Overall: Representation intervention and input search use different components to change success.
  2. The results and interpretation of the additional experiments above will be reflected in the text, tables, and figures of the main text.
  3. The experimental settings and scope will be stated explicitly in the main text. We will state the default application setting in the main text (addition at the single layer where the direction was estimated), the judging rule for the absolute success rates in Section 5, the white-box assumption and the model size range (1B to 14B), the estimation layer of each model, the conditions of the baseline comparison with the search budget matched at 150 steps, and the search compute cost relative to GPTFuzz.
  4. The scope of the released code and artifacts will be revised.
  5. The writing of the main text will be improved.

Revised paper uploaded

Dear reviewers,

Thank you for your reviews and for the discussion. We have uploaded the revised paper. It follows the revision plan above.

  • The title, the six sections and the main claims are the same as in the submission. Sections 4.1 and 4.2 keep their conclusions, Section 3.4 is new and reports the norm share and the cosines of the success-failure direction (Table 18), Section 5 adds the comparison with the fitness built on the component inside R, and the paper closes on the contrast between representation intervention and input search.

  • The additional experiments are in the main text and the appendix. The settings are stated in the main text: the default application (Section 4.2), the white-box assumption and model sizes (Section 3.3), the estimation layers (Section 3.2, with the per-model layers in Table 7 of the appendix), the success label for the rates of Section 5 (Table 2), and the 150-step budget (Section 5.1).

  • The suffix baselines now run the full 150 steps without early stopping, so the baseline numbers in Table 2, Figure 4 and Tables 22 and 26 are updated, and Table 21 reports the 150-step comparison. The search cost is given in wall-clock time against I-GCG and GCG. Against GPTFuzz the difference is that our fitness needs no generated response and no judge (Section 5, Appendix F).

  • The ethics statement now states the scope and intended use of the released code, and the writing of the main text is simplified.

In the PDF, red marks new experiments and statements updated to reflect them, navy marks wording-only edits, and black text is unchanged from the submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.