- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The author proposes CFA2, which contrasts task-matched harmful/jailbroken pairs in SAE space to find a concept direction and orthogonalizes output weights. Experiments report on HarmBench and AdvBench for 4 LLMs against 11 baselines.
Critical strengths
The paper's insight on pairing jailbroken variants to fix task semantics while varying refusal behaviors is interesting and well-grounded.
Critical weaknesses
The paper assumes content invariance s(x)=s(x⁺) across pairs, which may not always be valid and needs further justification.
Questions for authors
The Figure 3 is overly large to occupy signfiicant space, yet the information it provides is limited. Please consider revising the paper presentation.
2 comments
Dear Reviewer qW8Y,
Thank you again for your time and thoughtful feedback on our paper. As the discussion period is nearing its conclusion, we would greatly appreciate knowing whether our responses have addressed your concerns regarding the task-semantic consistency assumption and the presentation of Figure 3.
If so, we would be grateful if you would consider reflecting this in your updated rating. Of course, we fully respect your assessment and sincerely appreciate your time and consideration regardless of your final decision.
Best regards,
The Authors
Response to Weakness 1:
We sincerely thank Reviewer qW8Y for this important comment. Our contrastive pairs are constructed from the same underlying HarmBench task rather than independently sampled harmful and compliant requests. For each original request x, we construct successful jailbreak variants x+ that preserve the requested task objective while changing the model's response from refusal to compliance, as illustrated in Table 4.
The condition s(x)=s(x+) in Proposition 4.2 describes the ideal case of content invariance, which our task-matched construction aims to approximate. Importantly, the ablation in Table 9 provides empirical support for this design: replacing conventional harmful--harmless contrasts with task-matched refusal--compliance pairs improves the average ASR from 38.17% to 48.50%, while keeping the representation space and weight orthogonalization unchanged. This supports the practical value of controlling task-objective variation during defense-direction identification.
We emphasize that task-objective consistency motivates approximate latent invariance rather than guaranteeing exact equality for every pair. We will clarify this distinction and the scope of the theoretical assumption in the revised manuscript.
Response to Question 1:
We sincerely thank Reviewer qW8Y for this helpful suggestion. Figure 3 illustrates two complementary aspects of the feature analysis. Panel (a) illustrates the activation distributional shift of defense-related features between refusal and compliance states, while Panel (b) shows the activation consistency of task-related features and the distinct variation of defense-related features across the two conditions. We agree that its current presentation can be made more concise. We will reduce the figure's visual footprint, simplify the legends, and refine the caption to communicate the complementary roles of the two panels more clearly.