- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper reveals that explicit chain‑of‑thought mostly verbalizes pre‑existing latent moderation judgments and only alters outcomes for uncertain boundary cases via circuit‑level analysis of safety‑moderation models. Building on this mechanistic insight, it presents UTM, an uncertainty‑guided two‑stage moderation pipeline combining intermediate‑layer sparse‑feature readout and conditional full‑CoT fallback. UTM routes merely ~14% of samples to costly full‑CoT reasoning, obtains near‑full‑CoT F1 scores, and substantially reduces inference latency against reasoning‑based moderation baselines.
Critical strengths
-
The problem is important and practically relevant. The paper investigates the real‑world role of explicit CoT in safety‑moderation models, which is both mechanistically puzzling and practically critical for trustworthy AI content safeguards.
-
The method is practical. UTM works as a plug‑and‑play inference‑time pipeline and requires no retraining of the original safety model, bringing substantial inference‑cost reduction for reasoning‑based content moderation.
-
The paper combines theoretical insights with practical application. It delivers circuit‑level mechanistic findings alongside the UTM moderation pipeline prototype.
Critical weaknesses
-
The core mechanistic claim has restricted generalizability. Key circuit‑level causal evidence is derived primarily from RSafe, with only partial observational replication on GuardReasoner‑8B and no full validation across diverse safety‑moderation models.
-
The interpretability analysis relies on linear approximations for nonlinear Transformer computations. Zero‑baseline EAP and feature‑negation interventions adopt first‑order linear assumptions to estimate feature contributions, which may introduce approximation error and bias the quantitative results of circuit attribution and causal flipping experiments.
-
Practical deployment barriers remain substantial. UTM requires per‑model trained transcoders and feature extraction, which adds nontrivial engineering overhead and prevents simple off‑the‑box usage for arbitrary safety models.
-
The methodological novelty is incremental. Transcoder‑based circuit tracing, edge attribution patching and intermediate‑layer readout are existing interpretability tools; this work’s main contribution lies in applying them to safety CoT rather than introducing fundamentally new interpretability techniques.
Questions for authors
-
Beyond RSafe and the correlational observations on GuardReasoner‑8B, what indirect evidence or reasoning supports that the observed CoT mechanism exists across other safety‑moderation model frameworks, rather than being unique to RSafe’s training setup and architecture?
-
What is the magnitude of approximation error introduced by zero‑baseline EAP and feature‑negation within nonlinear transformers? Can the authors provide empirical estimates or ablation studies to characterize these approximation effects?
-
Can the authors report cross‑sample feature‑overlap baselines between circuits of fully unrelated safety inputs, to quantitatively disentangle input‑specific decision pathways from universal task‑background features?
-
Can the authors systematically ablate different uncertainty thresholds tau_safe / tau_unsafe, reporting their joint impact on UTM F1 performance and stage‑2 routing ratio?
3 comments
Responses to Reviewer ZCVX (Part 2 / 3)
#[W3 - Deployment overhead.]: UTM only adds a one-time training cost and is efficient to deploy.
Thank you for raising this practical concern. As the reviewer notes, UTM does not retrain the moderator, and its extra cost is offline. Stage 1 needs the transcoder encoder of a single layer (L22) and a logistic readout, both trained once per model. At inference, Stage 1 stops the forward pass at layer 22 and adds one encoder multiplication, so it is faster than even Direct answering on one A100, and the full pipeline serves 5.6× more requests per second than Full CoT (our reply to Reviewer Eo7F, W2). When training a transcoder is not desired, a dense probe at the same layer can replace the sparse readout with a small loss in F1, at most 0.006 at the same routing budget (our reply to Reviewer rK96, W1 & Q1). The routing pipeline itself therefore does not depend on transcoders. The readout and the thresholds still need to be fitted for each new model, and we will state this cost in the revised version.
#[W4 - Incremental methodological novelty.]: Our contribution is the question, the mechanistic findings and their use in UTM, not a new interpretability tool.
We agree that transcoders, attribution graphs and intermediate-layer readouts are existing tools, and we do not claim a new interpretability technique. Our motivation is a question that matters for trustworthy moderation. When a safety moderator reasons before deciding, does the reasoning determine the decision, or does it mostly verbalize a decision that has already formed? Our contributions are the answers and their consequences.
- The moderation label is selected from the query and then copied into the answer, and CoT mostly carries an existing judgment, changing the label mainly on uncertain inputs.
- Where the label is copied from depends on the moderator, the system prompt in RSafe and its own CoT in GuardReasoner. We establish this with targeted experiments, namely instruction rewording, label-word replacement and unrelated-input baselines.
- These findings directly yield UTM, which invokes CoT only when the Direct-mode state is uncertain and keeps CoT's benefit at a fraction of its cost.
We will make this positioning explicit in the introduction of the revised version.
#[Q3 - Separating input-specific decision pathways from task-background features.]: We separate them with targeted experiments that change one part while holding the other fixed.
Thank you for this question. The input-specific decision pathway corresponds to label selection, which chooses the label from the evidence in the query, and the task-background features correspond to label copy, which writes the chosen label into the answer format for every input. A cross-sample overlap baseline shows how much two circuits share, but not which features play which role. We therefore separate the two directly, by changing one while holding the other fixed.
Experiment 1: reword the instruction, keep the query and the verdict (RSafe). On 48 held-out queries from OpenAI, Aegis, XSTest and Toxic-Chat, half Direct/CoT agreements and half disagreements, we reworded every instruction sentence, block marker and all 18 category descriptions, leaving 17% 4-gram overlap with the original, while keeping the query, category numbering and answer format. We then regenerated the CoT, attributed the graphs again, and compared each graph with its reworded version on the 29 queries whose Direct and CoT verdicts are both unchanged.
| Direct | CoT | |
|---|---|---|
| Query-token features in layers 0–7 that survive rewording | 97.2% | 98.7% |
| Removed features recurring in ≥15/29 queries (share of all removals) | 1,037 (82.5%) | 1,049 (82.7%) |
| Added features recurring in ≥15/29 queries (share of all additions) | 836 (75.2%) | 838 (77.8%) |
Label selection is query-driven and survives the rewording almost entirely, since 97.2% (Direct) and 98.7% (CoT) of the query-token features in layers 0–7 are retained. Label copy behaves like task background. When the instruction changes, the same features are removed and added for nearly every query, since 82.5% (Direct) and 82.7% (CoT) of the removed features and 75.2% and 77.8% of the added features recur in at least 15 of the 29 queries.
Responses to Reviewer ZCVX (Part 3 / 3)
Experiment 2: replace the label words (RSafe and GuardReasoner). In the label-word replacement test of W1 & Q1, the query and the recorded CoT are kept, and only the label words are replaced with A/B. Replacing them only in the system prompt changes RSafe's answer to A/B (P(A or B) = 0.970), so its output label is copied from the template rather than computed from the input, while GuardReasoner copies it from its own CoT.
| Model | A/B placed in | P(A or B) | P(original label word) |
|---|---|---|---|
| GuardReasoner | system prompt only | 0.009 | 0.991 |
| GuardReasoner | CoT only | 0.255 | 0.744 |
| GuardReasoner | both | 0.527 | 0.473 |
| RSafe | system prompt only | 0.970 | 0.000 |
| RSafe | CoT only | 0.000 | 1.000 |
| RSafe | both | 0.971 | 0.000 |
Together, the two experiments identify the input-specific pathway as the query-driven label selection and the shared background as the template-driven label copy. In the revised version, we will add both experiments and describe the shared computation in these terms.
#[Q4 - Sensitivity to τ_safe and τ_unsafe.]: We report UTM F1 and the routing ratio over a joint grid of thresholds.
For RSafe on the four Table 4 test sets, we evaluate UTM S1+2 over τ_safe ∈ {0, 0.01, 0.05, 0.1, 0.3} and τ_unsafe ∈ {0.7, 0.9, 0.95, 0.99, 1}. The thresholds are not tuned on these test sets, and the default (0.05, 0.95) row corresponds to UTM S1+2 in the revised Table 4. The symmetric bands are shown below (F1 / routed share).
| τ_safe, τ_unsafe | OpenAI | Aegis | XSTest | SafeRLHF |
|---|---|---|---|---|
| 0.3, 0.7 | 0.783 / 1.6% | 0.822 / 1.2% | 0.851 / 1.6% | 0.895 / 3.4% |
| 0.1, 0.9 | 0.788 / 4.5% | 0.821 / 4.1% | 0.863 / 4.1% | 0.907 / 9.0% |
| 0.05, 0.95 (default) | 0.792 / 5.6% | 0.820 / 4.1% | 0.864 / 4.2% | 0.915 / 14.9% |
| 0.01, 0.99 | 0.779 / 11.0% | 0.831 / 8.3% | 0.865 / 8.8% | 0.917 / 19.3% |
| 0, 1 (Full CoT) | 0.771 / 100% | 0.831 / 100% | 0.870 / 100% | 0.921 / 100% |
We use the default thresholds because they balance efficiency and accuracy. They send only a small share of inputs to CoT, while keeping F1 close to that of Full CoT. In the revised version, we will add the full grid to the appendix.
Responses to Reviewer ZCVX (Part 1 / 3)
We sincerely appreciate the reviewer's recognition that the role of explicit CoT in safety moderation is an important and practically relevant problem. We have spent considerable time adding the empirical studies you suggested, which make our work more complete.
#[W1 & Q1 - Generalizability beyond RSafe.]: We explain why RSafe and GuardReasoner are representative of reasoning moderators, and summarize the new experiments on GuardReasoner-8B.
Thank you for this important question.
(a) Why these two models
Our study concerns a moderator's own reasoning before it decides, so it requires a moderator that reasons before answering, which makes the Direct and CoT modes comparable. The newer moderators we considered do not meet this requirement. Qwen3Guard hard-codes an empty thinking block and emits its label directly after "Safety:", and YuFeng-XGuard outputs its risk category as the first token, with an explanation afterwards.
RSafe and GuardReasoner represent the two main training paradigms of reasoning moderators, reinforcement learning (RSafe) and supervised fine-tuning on reasoning traces followed by preference optimization (GuardReasoner). They also differ in architecture, Qwen2.5-7B for RSafe and Llama-3.1-8B for GuardReasoner. Other reasoning moderators differ in implementation details but follow one of these two paradigms, so findings shared by both models are unlikely to be specific to RSafe's training setup or architecture.
(b) New experiments on GuardReasoner-8B
Feature interventions (our reply to Reviewer rK96, W3 (a)). Negating the label-associated features flips all 96 GuardReasoner labels in both modes, and setting them to zero removes 80–90% of the decision margin, so these features are causally necessary, as in RSafe.
Label-word replacement (Q3 below, and our replies to Reviewer rK96, W3 (b), and Reviewer Eo7F, W1 & Q1). GuardReasoner copies its label from its own CoT, whereas RSafe copies it from the system prompt. This explains the lower overlap and the persistent CoT attribution of GuardReasoner in App. D.8.
The main findings therefore hold in both models, now with causal evidence. Both models separate label selection from label copy, their label-associated features are causally necessary, and their Direct and CoT circuits diverge more on disagreements. In the revised version, we will add these experiments to App. D.8.
#[W2 & Q2 - Approximation error of linear attribution.]: Attribution is used only to select features, every causal effect is measured with an actual intervention, and we quantify the first-order error.
Thank you for raising this point. We agree that zero-baseline EAP is a first-order approximation. We therefore use attribution only to build the graphs and select candidate features, and we measure every causal effect with an actual intervention. In Table 1 and in all new intervention experiments, the selected features are modified and the model is run forward again, with all MLP and LayerNorm layers recomputed and attention patterns kept fixed, as in the standard circuit-tracer intervention. The reported label flips are therefore measured outcomes, not first-order estimates.
To quantify the approximation error (Q2), we compared the attribution-based prediction with the measured change of the label-logit margin in the full model when a fraction α of the selected feature contributions is removed (8 cases in Direct and CoT mode, three feature groups each).
When the attention patterns, normalization scales and MLP outputs are frozen, as the attribution assumes, prediction and measurement agree within 2×10⁻⁴ logits, so the graph edges are computed correctly. In the full model, the error is negligible for small perturbations but grows with their size, up to about 2 logits when 5% of the contributions are removed. Attribution scores should therefore be read as a ranking of candidate features rather than as effect sizes, which is why all our causal claims rest on interventions. The key mechanism results also hold in an experiment that uses neither transcoders nor attribution. In the revised version, we will report these error estimates and state this distinction explicitly.