- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether external safety monitors detect harmful content expressed through word substitution, even when the substitutions are explicitly defined. It compares fixed generated answers under substituted and restored wording, separates request- and answer-wording effects, and examines mapping-dependent hidden states and decisions. Additional experiments investigate demonstrations, fine-tuning, sequential alarms, and output-release policies. Across seven monitor configurations, substituted inputs receive fewer alarms on independently selected requests, although the effects vary substantially across monitors and evaluation settings.
Critical strengths
-
Careful controls make the behavioral finding convincing: Holding generated answers fixed avoids confusing wording effects with differences in generated content. The crossed request/answer comparisons and unrelated-answer-word control further show that an alarm under an explicit request does not establish interpretation of the substituted answer. On the 30 new requests, all seven monitors show lower alarm rates under substitution, with five comparisons surviving Holm correction.
-
The paper separates internal sensitivity, detection, and actual protection: Mapping changes produce distinguishable hidden states and score changes without necessarily changing decisions. The authors avoid interpreting this as proof of comprehension and report that activation interventions fail to establish mapping specificity. Likewise, the release-policy experiment holds warnings fixed, isolating the consequences of withholding flagged text. This restraint makes the diagnostic conclusions more credible.
Overall, the paper addresses a critical and timely vulnerability in external safety guardrails. The empirical evaluation across seven safety monitors is thorough and well-controlled, and the internal probing experiments provide insightful evidence of a disconnect between internal concept representation and final safety decision thresholds. The inclusion of practical mitigation analyses further strengthens its value to the AI safety community.
Critical weaknesses
-
The monitor evaluation is narrow despite covering several configurations - The independent harmful set contains only 30 responses, 26 from DeepSeek, and is selected using automated harmfulness judgments. Seven configurations represent four monitor families with different policy coverage and intended inputs. These results establish a vulnerability on the tested examples, but provide limited evidence about its prevalence across generators, harm categories, and realistic benign traffic.
-
The internal analysis establishes distinguishability rather than a mechanism for mapping failure - The probe uses 24 constructed pairs with the target codeword always in the first mapping row, allowing classification from the concept at that position. Attention measurements are descriptive, and controlled activation replacements do not identify a mapping-specific effect. The authors acknowledge these limitations, but the analysis consequently leaves unresolved why contextual information fails to produce the appropriate decision.
-
The mitigation evidence remains preliminary - Fine-tuning is evaluated on 24 harmful examples and 20 controls; benign controls are also much shorter than harmful answers. Under matched inference, only one of three seeds in each training scheme has an AUC-gain interval excluding zero. Paired substituted/restored training shows no clear advantage over repeated restored examples. Thus, the experiments suggest adaptation can help, but do not establish a robust substitution-specific defense.
-
The harmful-delivery estimates depend on a potentially vulnerable evaluator - Released prefixes are judged by Llama Guard 3, which the paper itself shows is sensitive to substituted wording. The authors disclose this dependency, but it leaves uncertainty about absolute delivery counts and whether apparently safe released prefixes contain recoverable harmful content. An independent human audit would strengthen this part of the study.
Questions for authors
-
Does the paired detection gap persist on a larger, generator-balanced set with human-verified harmfulness and monitor-policy applicability?
-
Does probe performance persist when mapping-row order, codeword identity, and harmful-concept identity are held out or counterbalanced?
-
Do adaptation gains survive length-matched benign controls and evaluation at a fixed false-positive rate?
-
How often do human judgments disagree with Llama Guard’s released-prefix labels, particularly under substitution?
5 comments
Response 2 of 4
W2 and Q2. Counterbalanced probe
We trained the probe with the target in one mapping row and evaluated it in the other. We also held out base pairs with unseen codewords and concepts. Separate tests held out entire behavior families or request templates.
The assay contains 216 base pairs from 18 behavior families × 12 templates, expanded to 1,728 factor conditions, plus 432 explicit-concept controls. The primary readout uses layer 22 of Qwen3-14B with a 20-dimensional PCA fitted within the training fold and a linear classifier.
The probe achieves an AUC of 0.9109 on held-out base pairs with unseen concepts and codewords. Cross-row transfer yields an AUC of 0.5949 when training on row 1 and evaluating on row 2, and 0.5811 in the reverse direction. Training on queries for codeword A and testing on B gives 0.9150, and the reverse direction gives 0.9017. Holding out behavior families and request templates gives AUCs of 0.8982 and 0.8988. Figure 4(a) shows these comparisons.
Using all 5,120 hidden dimensions gives cross-row AUCs of 0.4592 and 0.6320. Under both readouts, moving the target to the other row produces the largest drop in transfer performance.
In the input-intervention evaluation (Section 4.2), we held the query, answer, all non-table text, and the right-column concept order fixed. We exchanged only the two codewords in the left column. The harmful assignment scored higher in 864/864 pairs, by 10.6603 on average. The score follows which concept the queried codeword denotes when the concept positions are held fixed.
Reversing the rows preserved the mapping, query, answer, token counts, sequence length, and decision position. In the score-distribution pass, 102/432 harmful-target pairs and 0/432 benign-target pairs changed decisions. In the input-intervention baseline, the corresponding counts were 102/432 and 1/432, or 103/864 pairs in total (Appendix D.3). The 103 pairs span 77 base pairs and all 18 behavior families.
Response 3 of 4
W2 and Q2 (continued). Score calibration and state interventions
With the same monitor instruction and native alarm rule (d > 0), explicit harmful controls triggered 7/24 alarms in the single-template lexicon set and 210/216 in the 12-template scenario set. We selected both sets independently of monitor scores (Appendices D.1–D.3).
Figure 4(b) shows the score-distribution pass on the 216 counterbalanced base pairs. For coded targets, the harmful and benign medians are −2.00 and −12.75 (AUC 0.9618). For explicitly named targets, the medians are +14.25 and −19.00 (AUC 0.9971). At the original threshold of zero, coded harmful and benign targets trigger 290/864 and 0/864 alarms, compared with 210/216 and 2/216 for explicit targets (Table 13).
We recalibrated the threshold on these same scores (Appendix D.3). We selected a threshold on 17 behavior families and tested it on the held-out family, repeating this for each of the 18 families. At a 5% benign-alarm calibration target, 683/864 coded harmful targets and 43/864 coded benign targets trigger an alarm.
Moving the queried concept from row 1 to row 2 raises the midpoint of the harmful and benign scores by 2.3568 on average and widens their separation by 1.0353. Both scores rise on average. In the score-distribution pass, the native threshold gave 103/432 harmful-target alarms in row 1 and 187/432 in row 2. Ranking changes little across the two positions (AUC 0.9651 and 0.9656).
We also held 89 answer strings fixed and changed only the accompanying request. With the coded and original requests, respectively, Llama Guard 3 flags 52 and 74 answers. Under the coded and original questions, alarm counts were 36 and 86 for Qwen3Guard-0.6B, 46 and 80 for 4B, 50 and 75 for 8B, 8 and 37 for ShieldGemma, 57 and 61 for ReasoningShield-1B, and 62 and 67 for 3B (Table 1). In a separate comparison on all 132 paired responses, restoring only the question raises more alarms than restoring only the answer for every monitor. For Qwen3Guard-4B, the count rises from 60 with both fields substituted to 69 when only the answer is restored, 121 when only the question is restored, and 124 when both are restored (Figure 3). With the original question and unrelated words in place of the answer codewords, it still flags 117 responses.
At layer 22 we compared two donors that preserve the target class. One exchanges the codeword assignment and queried key together, and the other reverses the rows. Across 1,728 conditions, the mean assignment-minus-row donor contrast is 0.00362 when both donors are scored in the same direction. Replacing the mapping-span states leaves the recipient's other states at that layer intact.
We wrote identical layer-22 donor states at two destinations on 864 conditions from 108 base pairs and weighted the 18 families equally (Appendix D.6.3). Assignment and row-order donors produce mean score changes of +0.3692 and +0.3695 at the original mapping span, compared with −0.2520 and −0.2653 at an equal-length output-template span. The differences between destinations are +0.6212 and +0.6348, positive in all 18 families. Moving the recipient's own mapping states into the template also changes the score (−0.3947). The effect of the replacement therefore depends on where the states are written for both donor types.
Response 1 of 4
Thank you for your thoughtful review of our work. We appreciate your attention to generator balance and to the judgments of harmfulness and policy applicability. We have revised certain sections of the paper as well as the appendix based on the reviews.
W1 and Q1. Generator balance and qualification
We evaluated 132 native responses from 59 requests and 78 generator–request pairs across six generators. The response counts were DeepSeek (65), Grok (45), Qwen (13), GLM (5), GPT Sol (3), and GPT Terra (1). Two model judges independently assigned the highest harmfulness label to the exact restored prefix scored by the monitors. Human harmfulness and policy-applicability ratings were not collected for this comparison. Table 1 gives the following alarm counts.
| Monitor | Substitution alarms /132 | Restoration alarms /132 | Difference (pp) |
| Llama Guard 3 | 75 | 116 | +31.06 |
| Qwen3Guard-0.6B | 56 | 122 | +50.00 |
| Qwen3Guard-4B | 60 | 124 | +48.48 |
| Qwen3Guard-8B | 76 | 123 | +35.61 |
| ShieldGemma-2B | 33 | 67 | +25.76 |
| ReasoningShield-1B | 77 | 88 | +8.33 |
| ReasoningShield-3B | 86 | 99 | +9.85 |
All seven monitors flag more restored responses. Restricting the comparison to the 124 responses with explicit mappings gives increases of 8.06–50.81 percentage points. DeepSeek and Grok contribute 110 of the 132 responses, so we also selected three requests for each of the five generators with at least three distinct requests. This gives 22 responses from 15 generator–request pairs and 13 distinct requests. After weighting the five generators equally, the gain is 55.33 points for Llama Guard, 58.00/66.00/67.33 for the three Qwen3Guard sizes, 20.00/29.33 for the two ReasoningShield sizes, and 26.67 for ShieldGemma. Llama Guard and all three Qwen3Guard sizes show the same direction in each of the five generators (Appendix C.2).
We screened 4,985 published HarmBench and JailbreakBench responses for overlap and retained 2,225 across 119 request clusters. Both judges labeled 500 of the retained responses directly harmful. These 500 span 113 request clusters and eight generator versions. ShieldGemma flagged 285/500 (57.0%), Qwen3Guard-4B flagged 446/500 (89.2%), and ReasoningShield-1B flagged 415/499 valid outputs (one parsing failure). The 500 include three Claude-2.1 and four GPT-4-0125-preview responses. Each monitor received the released attack prompt and the full published answer (Table 7).
We evaluated 819 control pairs across 813 distinct request identifiers, including 240 synthetic benign, 156 refusal, 195 metaphor, and 228 definition/fictional pairs. Table 1 shows where wording affects benign alarms. Qwen3Guard-4B flags 75/195 substituted metaphors and 13/195 restored metaphors. On definitions and fictional contexts the counts are 126/228 and 77/228. Six monitors passed 8/8 control calibration checks. ReasoningShield-3B passed 7/8.
The seven monitor configurations span four families, including three Qwen3Guard scales and two ReasoningShield scales. Each paired comparison holds the policy and monitor fixed. Appendix C.1 gives the native alarm rules and explains how we used ReasoningShield's reasoning-trace interface to assess answers.