- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the role of DTHs in rejection behaviors. The author proposes ACP, which improves the edge attribution patches by combining geometric projection scores, and sorts DTHs based on the attention degree of repeated tokens. The author constructed a dataset derived from Wikidata, trained the rejection behaviors of birthplace and deathplace questions using DPO, and applied token replacement to repeated entity mentions. Experiments on Llama, Mistral, and Openchat models show that the attack success rate can reach up to 39.79%, and the recovery rate can be as high as 46.24% through clean activation head patches. At the same time, by using a fixed DTH guiding strategy, significant rejection recovery effects were achieved in previous successful attacks.
Critical strengths
- This paper employs intervention on the internal attention heads, by testing various head budgets and comparing DTH patches with random head patches, providing a valuable starting point for studying how repeated entity processing affects model outputs.
- Combined with controlled rejection tasks, three models, task-related SFT comparisons, and input-independent guidance, it has practical significance.
Critical weaknesses
- The attack success metric cannot reliably prove actual security bypass. In Section 4, only the premise mentions of a certain entity were tampered with, while the question itself continued to inquire about the original entity, resulting in a query that might lack basis, as acknowledged in Appendix A.1 itself. Moreover, the assessment process regarded publicly accessible Wikidata attributes as confidential information through custom data protection officer rules, so these results cannot prove the existence of widespread harmful content security bypass.
- The claimed DTH-specific security mechanism and actual repair functions have not been fully verified. By repeating token attention selection of the head and inserting clean activation, it can be shown that these heads affect the output results, but it is impossible to distinguish the rejection-specific intermediary effect from their regular role in entity binding or fact retrieval. Randomly selecting heads for insertion is not a controlled experiment matching hierarchy, activation norms, or task importance; there is no necessity for sensitivity testing or direct proof of the proposed DTH to the security path. The geometric score in Formula (3) measures one-dimensional activation projection, and it has not been compared with the standard EAP score based on behavioral fidelity. Finally, fixed guidance was only evaluated for previously successful attacks (Section 5; Appendix A), and there was a lack of measurement of false rejection or utility under clean input. These omissions severely weaken the core causal claim.
Questions for authors
- Why is the response with i = 4 marked as correct in Table 8 of Appendix E, even though no information about the X Y Z V of the query is provided? What proportion of the reported success rate actually answered the original target entity without rejecting? Please provide independent audits by the model and operator, based on the automatic speech recognition results of the target.
- Can the author control the referential errors and semantic rootlessness issues by comparing the edits that maintain the same entity in the two mentions, and evaluating comparable tasks without using the precise repetition word template? Does the rejection bypass effect still exist?
- In the unattacked Citizenship and AlpacaFarm examples, for each head budget and turning coefficient, what are the false rejection rates and utility costs of the fixed DTH turning?
3 comments
Response to Reviewer XYgi (2/2): questions
Q1: why is the i = 4 response in Table 8 marked CORRECT? What share of the success rate answered about the original target? In our pipeline a response counts as a successful attack if it discloses the ground-truth place for either person in the premise. Because the two persons share a city by construction, the place of the queried person can then be concluded, so answers that decline for the queried person but name the other person's city are counted as successes (the i = 4 response in Table 8). We agree with the reviewer that the paper should separate these cases. As requested, we additionally re-judged all successful outputs (Born/Died, our three operators and the Saliency and Rand-loc baselines) with a target-aware judge: CORRECT only if the place is stated for the queried entity, INDIRECT if it is given only for the comparison entity, otherwise INCORRECT or REFUSAL. The original ASR counts all outputs labelled CORRECT by the original judge; some of them contain no place at all (REFUSAL after re-judging), so it is an upper bound for the direct-plus-indirect rate. Below we report direct disclosure (CORRECT) separately.
Strict ASR (direct disclosure only), % of pairs with a refused clean response, mean over 1-4 tokens (target-aware judge labels):
| Attr. | Freq | Glue | Random | Saliency | Rand-loc | |
|---|---|---|---|---|---|---|
| Llama | Born | 15.18 | 13.87 | 15.58 | 7.85 | 14.79 |
| Llama | Died | 11.75 | 18.98 | 10.54 | 4.12 | 6.33 |
| Mistral | Born | 3.76 | 0.26 | 1.32 | 0.93 | 2.25 |
| Mistral | Died | 0.24 | 0.00 | 0.49 | n/a* | n/a* |
| OpenChat | Born | 0.85 | 1.70 | 2.21 | 0.92 | 2.04 |
| OpenChat | Died | 0.75 | 2.69 | 1.03 | 1.60 | 1.38 |
*To share the results quickly we post them without these two cells (Mistral Died, Saliency and Rand-loc). They need to be recomputed, which we will probably not finish during the rebuttal period. n per cell is 332-2,060 pairs x tokens for our operators (up to 5,460 for Saliency); Wilson 95% intervals will be added to the revised tables.
Where the outputs counted as successful in the paper (original CORRECT) went after re-judging (% of old CORRECT, pooled over models and attributes):
| Attack | CORRECT | INDIRECT | REFUSAL | INCORRECT |
|---|---|---|---|---|
| Freq | 43.0 | 36.6 | 19.4 | 0.9 |
| Glue | 44.4 | 43.2 | 12.4 | 0.0 |
| Random | 36.5 | 41.1 | 22.4 | 0.0 |
| Saliency | 40.1 | 43.8 | 15.5 | 0.7 |
| Rand-loc | 46.2 | 37.2 | 15.8 | 0.9 |
So only 37-46% of the originally CORRECT outputs disclose the place for the queried entity. About 40% name only the comparison entity's city (INDIRECT), and 12-22% do not disclose any place. Llama stays clearly vulnerable under the strict reading, and our operators exceed Saliency in all cells. For Mistral and OpenChat strict ASR is at most 3.8% in every cell, and the differences between our operators and the baselines are small (overlapping intervals). We will therefore restrict the claim that targeted operators outperform the untargeted baselines to the cells where the intervals separate, and describe the leakage of Mistral and OpenChat as mostly indirect.
Q2: controlling referential errors. We agree that this is the right control: the pipeline requires an entity that appears twice and corrupts one mention, which can leave the question without a clear referent (see Limitations). We will try to run a small trial sample with edits that keep the same entity in both mentions, and report whether the refusal bypass persists, during the rebuttal period. If this is not possible in time, we will state it as a limitation and list the claims that depend on it.
Q3: false-refusal rate and utility cost. We will try to measure the false-refusal rate on unattacked Citizenship questions and utility on AlpacaFarm for a sample of head budgets and steering coefficients, and report the results here. If the full grid cannot be completed, we will report the subset we have, and describe steering as a proof of concept without claims about utility.
Update on W2(b), W2(d), W2(e) and Q3. The new results are in our general comment above, for Mistral-7B-DPO.
(b) The random control now uses 4–5 independent seeded draws per budget; it is still not layer- or norm-matched, and we keep that as a limitation.
(d) With the judge that separates real refusals from broken output, DTH steering shows a modest advantage over random heads on Mistral (clear + indirect +17.5 pp at N = 20).
(e)/Q3 A partial false-refusal check on 17 benign questions found no refusals; the Citizenship and AlpacaFarm evaluation is not done.
Response to Reviewer (1/2): weaknesses
We thank the reviewer for the detailed review. Where experiments are still running we say so, and we will report the results in this thread during the discussion period.
W1: the metric cannot prove a real security bypass. We agree and will revise the wording. Our setup measures the bypass of a trained refusal behaviour: DPO fine-tuning gives the models a controlled refusal on a defined attribute set, whose circuit we can study. The attributes are public Wikidata facts, chosen because the ground truth is known and disclosure is checkable, not because they are confidential. ASR is therefore not a measure of real-world harmful content. In the revision we will replace "safety bypass" and "harmful content" with "bypass of a trained refusal behaviour" in the abstract, introduction and conclusion, and state in the Limitations that the results do not by themselves transfer to harmful-content refusals. We will also state in Section 4 that only the premise mention is corrupted while the question still asks about the original entity (Appendix A.1), and discuss the resulting referential ambiguity (see Q2).
W2: the DTH-specific mechanism and the repair are not verified.