- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TRACE combines a label-supervised cross-layer vector-quantized codebook, a causal GRU detector, and concept-conditioned activation steering. The detector jointly predicts completed-response harmfulness and monotone cumulative streaming risk from response-level labels; steering independently gates token-level, norm-preserving edits using harmful-to-benign concept-mean differences. Post-hoc LLM descriptions make the recorded concept sequences and edit pairs readable. Evaluation covers three 8B generators on WildGuard and S-Eval, with HarmBench-based judgments of regenerated outputs.
Critical strengths
-
A coherent, inspectable safeguard design. Sharing a discrete representation across detection, intervention, and logging provides a concrete way to inspect what the system encountered and edited. Unlike prompt-conditioned CAST, steering is reassessed at each generated-token activation, allowing intervention decisions to change during a response. This integration is practically meaningful even without treating concept names as causal explanations.
-
Substantial empirical support for detection. TRACE improves streaming F1 over the reproduced, layer-matched Kelp baseline in five of six settings, including 0.935 versus 0.896 on Qwen/S-Eval and 0.961 versus 0.919 on Llama/S-Eval (Table 1). Threshold-independent metrics and validation-calibrated detection-position analyses strengthen the evidence beyond a single operating point, although these comparisons do not establish statistical significance. The appendices disclose baseline-reproduction choices and report detector-seed variation, while correctly noting that the latter fixes the VQ representation.
Critical weaknesses
-
The distinctive contribution is not sufficiently isolated. The cross-layer codebook builds on CLVQ-VAE; SCAR already combines concept detection and steering, and Breaking Bad Tokens includes token-conditioned detoxification (Section 5). Relative to these precedents, TRACE's distinctive contribution appears to lie in its shared discrete representation, dual-head temporal detector, and concept-pair steering rule. Table 9 shows slightly higher activation-only streaming F1 on Qwen/WildGuard (0.844 versus 0.837) and only a 0.001 advantage for combined inputs on Qwen/S-Eval; the authors appropriately acknowledge that concepts are not the main source of detection accuracy. The remaining case for their value relies on steering and auditing, but named concepts cover 46.4% of Qwen/WildGuard test-token assignments, while the unnamed share averages 75.75% across true-positive traces after consecutive concept repetitions are merged (Table 5). This limits semantic coverage, without invalidating the recorded edits; controlled evidence that the representation improves diagnosis or that nearest-benign targets outperform simpler alternatives would strengthen the contribution.
-
The safety-utility claim is only partially validated. Three selected settings do not satisfy all quality criteria used to approve stronger steering: mean non-repetition is 0.39 for Llama/WildGuard and 0.48 for InternLM/S-Eval, while 26/200 originally safe validation cases newly fail the quality check for InternLM/WildGuard (Table 11). The stated selection rule permits retaining strength 0.5 in these circumstances, so this is not a contradiction in the procedure; rather, it leaves degradation as a possible contributor to reduced ASR, as the authors acknowledge. Stable aggregate, phrase-based refusal rates on originally safe responses do not establish preserved helpfulness, correctness, or individual refusal decisions, especially when many such responses already refuse. Finally, judging only the first 512 classifier tokens of outputs capped at 2,048 generator tokens leaves later harmful content unassessed (Appendix D.2); this limitation applies to all compared steering methods.
Questions for authors
-
Contribution-isolation request: Beyond the Qwen input ablations, does the activation-only, dual-head GRU retain TRACE's detection gains on Llama and InternLM? With the harmful-concept gate and update rule fixed, do nearest-benign targets outperform a global refusal direction or randomly selected eligible benign targets at comparable output quality and validation-calibrated strength?
-
Safety-utility validation request: Appendix F.2.2 explains that strength 0.25 is excluded from selection because it reduces ASR less, although Table 11 shows that it meets all four quality criteria in the three problematic settings. Do the safety-utility conclusions persist if strength selection requires every reported setting to satisfy those criteria? Full-output harmfulness assessment and helpfulness evaluation on benign, non-refusal tasks would materially strengthen this assessment.
-
Methodological clarification: Equation (12) rotates the activation toward the normalized source-to-target difference vector, not toward the benign mean itself; norm preservation alone does not guarantee proximity to the target concept. This is a valid steering construction, not an algebraic error, but what evidence supports its intended benign effect beyond reduced ASR and the existing direct-addition and harmfulness-weighting ablations?
-
Streaming-safety validation request: In a manually annotated subset, how often do threshold crossings precede the first harmful information, and how much harmful text is released before detection? The paper correctly disclaims onset localization; normalized detection position and response-level F1 nevertheless do not by themselves establish prevention of harmful disclosure.
7 comments
Response to Q2 (Part 1/2): Steering-strength selection
A2 Thank you for suggesting this test. When we require every selected strength to pass all four validation quality checks, three of the six settings switch to a lower strength of 0.25: Llama/WildGuard and both InternLM settings. At this strength, text quality improves, but the reduction in ASR is much smaller than at 0.5. The other three settings retain their reported strengths.
To test this, we included 0.25 in the strength selection and chose the strength with the lowest validation ASR among those passing all four quality checks. The table compares test ASR and text quality for the three affected settings. Safe quality failures count originally safe responses that pass the checks before steering but fail afterward.
| Setting | Strength | ASR before → after | Non-repetition | Safe quality failures |
|---|---|---|---|---|
| Llama/WildGuard | 0.25 | 0.874 → 0.820 | 0.77 | 108/1,519 |
| Llama/WildGuard | 0.50 | 0.879 → 0.592 | 0.45 | 133/1,519 |
| InternLM/WildGuard | 0.25 | 0.862 → 0.862 | 0.90 | 90/1,486 |
| InternLM/WildGuard | 0.50 | 0.870 → 0.435 | 0.59 | 150/1,486 |
| InternLM/S-Eval | 0.25 | 0.782 → 0.655 | 0.92 | 77/615 |
| InternLM/S-Eval | 0.50 | 0.766 → 0.171 | 0.51 | 86/615 |
For Llama and InternLM on WildGuard, 0.25 leaves harmful generation largely unchanged. This explains why we originally used 0.5 as the default: it reduces ASR much more, while 0.25 better preserves text quality. Even at 0.25, InternLM/S-Eval exceeds the 10% safe-response quality failure limit on the test set, at 12.5%.
We report the full-answer harmfulness and benign-task evaluations in Part 2 below.
Response to Q2 (Part 2/2): Full-answer and benign-task evaluation
A2. Full-answer assessment. For all test comparisons of TRACE, steering baselines, and steering ablations, we now use all the available space in HarmBench's 2,048-token context. We keep the judging instructions and original request in full, reserve one token for the verdict and eight as a buffer, and use the remaining space for the response. Answers that do not fit are truncated at the end. The test ASRs reported here use these judgments. We still choose the steering strength using HarmBench’s judgments of the first 512 tokens of each validation response. Appendix D.2 describes the HarmBench evaluation.
We also sent Gemini 3.1 Flash-Lite the complete unsteered and steered answers for all 284 Qwen/WildGuard prompts whose original dataset responses were labeled harmful, using the reported steering setting (lambda=0.75). Gemini did not see method names and judged content and quality separately. Partly coherent answers had quality problems but remained understandable, while substantially degraded answers had meaning that could not be understood.
Gemini judged fewer unsteered answers harmful than HarmBench: 125 versus 257 among the 280 answers judged by both. Our manual checks suggest that Gemini was sometimes misled by adversarial framing in the requests and classified harmful answers as safe. This raises concerns about relying on Gemini alone for harmfulness judgments in this setting. We therefore retain HarmBench for the reported ASR and use Gemini to examine whether the complete steered answers provide understandable refusals or safe alternatives, or become degraded.
HarmBench judged 261 of the 284 unsteered answers harmful. After steering, 169 of these were judged non-harmful and 92 remained harmful. The other 23 answers were judged non-harmful both before and after steering. To examine what accounts for the ASR reduction, the table below shows Gemini’s assessment of response quality for those 169 steered answers.
| Outcome | Number of answers |
|---|---|
| Coherent refusal or safe alternative | 47 |
| Partly coherent refusal or safe alternative | 38 |
| Substantially degraded | 78 |
| Harmful answer, coherent or partly coherent | 1 |
| Partly coherent, but content unclear | 2 |
| Judging request blocked | 3 |
Of these 169 answers, Gemini classified 85 as understandable refusals or safe alternatives and 78 as substantially degraded. These results suggest that the reduction in ASR comes from both safer responses and degraded text.
Helpfulness on benign requests. We also evaluated the Qwen/WildGuard setting on 804 AlpacaEval requests. We gave Gemini each request and two answers, one generated without steering and one with TRACE. We randomized which answer appeared first and did not reveal which used steering. Gemini compared their helpfulness based on correctness, relevance, completeness, and instruction following. It also checked whether each answer provided what the user asked for.
| Measure | Result |
|---|---|
| Similarly helpful after steering | 60.7% |
| More helpful after steering | 9.0% |
| Less helpful after steering | 30.3% |
| Adequately addresses the request, before → after | 87.9% → 75.9% |
Most steered answers were judged equally or more helpful than the originals. However, steering reduced the proportion of answers judged to adequately address the user's request.
Correctness on math questions. We evaluated all six reported steering settings on the 1,319 questions in the GSM8K test set. We compared final numerical answers with the references, counting missing or invalid answers as incorrect. The table shows accuracy before and after steering.
| Generator | WildGuard-trained (%) | S-Eval-trained (%) |
|---|---|---|
| Qwen | 94.2 → 93.6 | 93.7 → 21.5 |
| Llama | 86.4 → 84.9 | 86.1 → 3.3 |
| InternLM | 90.5 → 84.6 | 89.9 → 73.0 |
With WildGuard-trained steering, accuracy remains broadly stable for Qwen and Llama but decreases for InternLM. The S-Eval-trained settings cause larger drops across all three generators. These differences suggest that the data used to learn the concepts and select steering settings may affect how well correctness is preserved on tasks beyond the safety benchmarks.
Appendix F.2.2 reports the complete-answer, helpfulness, and correctness evaluations. Appendix F.2.3 gives the Gemini prompts.
Q1 Contribution-isolation request: Beyond the Qwen input ablations, does the activation-only, dual-head GRU retain TRACE's detection gains on Llama and InternLM? With the harmful-concept gate and update rule fixed, do nearest-benign targets outperform a global refusal direction or randomly selected eligible benign targets at comparable output quality and validation-calibrated strength?
A1. Thank you for suggesting these comparisons. On both WildGuard and S-Eval, Llama and InternLM show similar detection performance with activation-only and activation-plus-concept inputs, consistent with the Qwen ablation. Their streaming F1 scores differ by at most 0.030 across the four settings. The activation-only detector also exceeds Kelp’s streaming F1 in all four. The concept representation therefore maintains competitive detection while providing the concepts used for selective steering and auditing. Appendix F.1.7 reports the detector variants.
For steering, we compared nearest-benign targets with a global refusal direction and randomly chosen eligible benign targets. We repeated the random-target experiment with three seeds, assigning each harmful concept a benign target at random. Both alternatives retained TRACE’s concept assignments, harmfulness scores, and update rule. We used the same validation procedure to select each method’s strength.
The below table reports averages across all six generator–dataset settings, with detailed results for each setting in Appendix F.2.6. Random-target results are averaged across three seeds. For these comparisons, we recomputed test ASR using the longer HarmBench judging window described in A2. Steering strengths were still selected using the original validation procedure.
Compared with the average of the three random mappings, nearest-benign targets have higher non-repetition and fewer safe-response quality failures in all six settings. Random targets achieve lower ASR on average, but produce more repetition and cause more safe responses to fail the quality checks. Choosing a nearby target therefore helps preserve quality, even when a random target reduces ASR more.
The global refusal direction achieves lower average ASR, higher non-repetition, and fewer safe-response quality failures. However, it causes a larger change in the aggregate refusal rate on originally safe responses in every setting. Nearest-benign steering keeps this rate closer to its unsteered value. This global-direction ablation still uses TRACE’s concept assignments and harmfulness scores to decide when and how strongly to edit. It replaces only the direction with a shared refusal direction.