- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates why an LLM agent may correctly judge an action as unsafe while still preferring it. Across three open-weight models and synthetic tool-use scenarios, the authors show that safety information remains readable in action representations. However, interventions that strongly control safety judgments transfer weakly to action preference compared with action-native interventions. Counterfactual state swaps and subspace analyses further suggest limited overlap between judgment and action control. The paper concludes that representing safety information does not necessarily grant it causal authority over action selection.
Critical strengths
The paper clearly separates information availability from causal control. This matters because correct safety judgments or self-critique are often treated as evidence of safer behavior.
The combination of probing, causal interventions, state swaps, subspace analysis, held-out scenario families, and three model architectures provides broader evidence than a single steering experiment.
Critical weaknesses
The high-level judgment–action gap is not itself novel, as prior work has already shown that agents may recognize safety, privacy, or abstention constraints while failing to act accordingly.
The controller comparison is not fully diagnostic. The action-native controller is directly optimized for the evaluated action margin and injected at the action position, whereas the judgment controller targets another endpoint and is sometimes injected earlier. Its weaker action effect may therefore reflect endpoint optimization, intervention position, or perturbation decay.
The shared-trajectory experiment fixes a plausible mediator. Teacher-forcing the same BLOCK token removes the pathway through which judgment may normally affect action. Indeed, explicit BLOCK text strongly changes action preference. The evidence therefore supports weak latent-state transfer conditional on a fixed judgment token, but the broader conclusion that safety judgments fail to govern action is somewhat overstated.
Questions for authors
Can the authors include a matched upstream action-relevant intervention to distinguish semantic decoupling from generic attenuation across positions?
What is the total causal effect when the ALLOW/BLOCK judgment is generated rather than teacher-forced? Can it be decomposed into latent direct and explicit-token-mediated effects?
Do the conclusions hold for free-form tool generation or actual execution in a stateful environment, rather than scored candidate actions?
4 comments
3. Explicit judgment context and natural-state causal influence
A separate factorial intervention crosses the latent judgment intervention with externally supplied ALLOW/BLOCK text:
| Model | Text BLOCK − ALLOW SafeEff | Latent under ALLOW | Latent under BLOCK |
|---|---|---|---|
| Qwen3-32B | 0.132 | 0.00696 | 0.00170 |
| Gemma3-27B | 0.0989 | -0.0109 | 0.00093 |
| Mistral-Small-3.2-24B | 0.152 | 0.00672 | 0.00256 |
BLOCK text strongly shifts action preference; latent effects are smaller and architecture-dependent. The latent effect is attenuated under BLOCK for Qwen and Mistral, while Gemma changes sign across textual contexts.
Separately, within-trajectory interventions hold the explicit BLOCK judgment fixed. At Qwen L32, Gemma L32, and Mistral L24, strongly shifts the judgment readout but has little effect on later action preference:
| Model | Judgment effect of (95% CI) | Later action effect | Paired vs. random (95% CI) | Downstream |
|---|---|---|---|---|
| Qwen | +4.17 [+4.07, +4.28] | +0.000924 | +0.00133 [-0.000134, 0.00276] | +0.0435 |
| Gemma | +18.7 [+18.1, +19.3] | -0.000065 | -0.00132 [-0.00346, 0.000772] | +0.212 |
| Mistral | +6.03 [+5.98, +6.08] | +0.00135 | -0.000127 [-0.00120, 0.000929] | +0.0944 |
The paired effects relative to matched random perturbations include zero in all models, whereas downstream action-native effects are larger.
Natural-state counterfactual swaps replace the full residual state immediately before explicit judgment with a matched donor state (60 held-out pairs; both directions, 120 interventions per model), without gradients, learned directions, scaling, or norm matching. We measure action effects with BLOCK fixed and when the explicit label follows the model's ALLOW/BLOCK readout:
| Model | Judgment alignment (95% CI) | Action, judgment fixed (95% CI) | Action, label unchanged (95% CI) |
|---|---|---|---|
| Mistral L24 | +0.0604 [+0.0505, +0.0698] | +0.00004 [-0.00063, +0.00071] | -0.00070 [-0.00147, 0.00007] |
| Qwen L32 | +0.0156 [-0.00417, 0.0359] | -0.00027 [-0.00099, 0.00047] | -0.00039 [-0.00122, 0.00041] |
| Gemma L32 | +0.410 [+0.318, +0.507] | +0.00051 [-0.00090, 0.00196] | +0.00013 [-0.00149, 0.00173] |
Intervals are pair-cluster bootstrap 95% CIs (10,000 resamples). With the label unchanged, downstream effects are near zero. In a descriptive Mistral subset, 10 label-crossing interventions produce a donor-aligned action effect of +0.135 [0.117, 0.152]; crossings occur in 1/120 Qwen interventions and 0/120 Gemma interventions. This is descriptive evidence of coupling, not a formal mediation estimate. Explicit judgment context and latent judgment control are distinct causal inputs.
4. Free-generation and sandbox evaluations: behavioral scope
Free-generation baselines, without teacher-forcing BLOCK, are:
| Model | Generated BLOCK | BLOCK outcomes: Unsafe / Safe alt. / Other tool / Non-tool | ALLOW: Unsafe / Total |
|---|---|---|---|
| Qwen3-32B | 80/90 | 0 / 11 / 0 / 69 | 10/10 |
| Gemma3-27B | 78/90 | 41 / 17 / 20 / 0 | 12/12 |
| Mistral-Small-3.2-24B | 76/90 | 0 / 20 / 48 / 8 | 14/14 |
Gemma shows a pronounced BLOCK-then-unsafe pattern (41/78); Qwen often produces non-tool/final responses, while Mistral often proposes other tools. These latter outcomes are not counted as successful safe execution.
Matched-upstream controllers were applied once at the final prompt token before the judgment stage; models then freely generated judgment and action under greedy decoding (temperature 0):
| Model | Controller | Unsafe-proposal change (pp, 95% CI) | Unsafe → non-unsafe / reverse | Tool-call rate change (pp) |
|---|---|---|---|---|
| Qwen | 0.0 [0.0, 0.0] | 0 / 0 | 0.0 | |
| Qwen | 0.0 [0.0, 0.0] | 0 / 0 | +4.4 | |
| Gemma | -2.2 [-6.7, 0.0] | 2 / 0 | 0.0 | |
| Gemma | -1.1 [-3.3, 0.0] | 1 / 0 | 0.0 | |
| Mistral | -2.2 [-6.7, 0.0] | 2 / 0 | -12.2 | |
| Mistral | -2.2 [-6.7, 0.0] | 2 / 0 | 0.0 |
Qwen's converts four non-tool/final responses into safe alternatives without changing unsafe proposals. In Mistral, both controllers convert two unsafe targets into other-tool proposals; also reduces tool calls. Gemma changes are small. These interventions do not consistently reduce unsafe proposals across models.
Across Direct, Safety Reminder, Explicit Judgment, and Neutral Deliberation, the table combines generated proposal categories (first four outcome columns) with three-step in-memory sandbox outcomes:
| Model | Condition | Unsafe target | Safe alt. | Other tool | Non-tool/final | Tool calls | Unsafe transitions | Conditional rate |
|---|---|---|---|---|---|---|---|---|
| Qwen | Direct | 12 | 10 | 3 | 65 | 25 | 12 | 48.0% |
| Qwen | Safety Reminder | 51 | 3 | 36 | 0 | 90 | 57 | 63.3% |
| Qwen | Explicit Judgment | 10 | 13 | 8 | 59 | 31 | 10 | 32.3% |
| Qwen | Neutral Deliberation | 56 | 10 | 7 | 17 | 70 | 50 | 71.4% |
| Gemma | Direct | 70 | 2 | 5 | 13 | 75 | 68 | 90.7% |
| Gemma | Safety Reminder | 83 | 4 | 3 | 0 | 90 | 86 | 95.6% |
| Gemma | Explicit Judgment | 64 | 18 | 8 | 0 | 90 | 69 | 76.7% |
| Gemma | Neutral Deliberation | 70 | 3 | 10 | 7 | 81 | 73 | 90.1% |
| Mistral | Direct | 12 | 2 | 6 | 70 | 20 | 12 | 60.0% |
| Mistral | Safety Reminder | 22 | 23 | 31 | 14 | 79 | 27 | 34.2% |
| Mistral | Explicit Judgment | 18 | 27 | 40 | 5 | 85 | 22 | 25.9% |
| Mistral | Neutral Deliberation | 31 | 22 | 36 | 1 | 87 | 35 | 40.2% |
All counts are out of 90; conditional rate = unsafe transitions / tool calls. The sandbox extends evaluation to multi-step state transitions in an in-memory simulation, not external execution. Explicit Judgment yields parseable judgments in all cases, with BLOCK rates of 88.9% (Qwen), 72.2% (Gemma), and 88.9% (Mistral). Relative to Direct, Safety Reminder increases unsafe proposals by 43.3, 14.4, and 11.1 percentage points, respectively, and increases tool calls in all models. Safety reminders change both entry into the tool-use path and the resulting action distribution; they are not equivalent to latent judgment control or to the externally supplied BLOCK/ALLOW intervention in the factorial experiment. Mistral's conditional rate decreases despite more total unsafe transitions.
5. Discrete action selection: top-1 ranking changes
At baseline, unsafe top-1 rates are 45/90 (Qwen), 30/90 (Gemma), and 29/90 (Mistral). In Qwen, changes 8/45 unsafe top-1 cases to non-unsafe, versus 3/45 for ; all eight action-native flips occur in the review family. Although case-paired McNemar gives , the family-cluster CI is [-26.7, 0.0] pp and the family-level exact test over 512 sign configurations gives , so the case-level result does not establish cross-family improvement. Neither controller produces beneficial flips in Gemma or Mistral.
Conclusion
Our results identify an asymmetric causal relationship between safety judgment and action selection. Safety information remains available and explicit judgment context can influence action preference, but the tested judgment-native direction exerts substantially weaker causal control over action than the action-native direction, even under matched upstream interventions. Representing a safety constraint is not equivalent to granting it control over action selection.
Follow-up on Judgment–Action Causal Control
Dear Reviewer,
Thank you again for your detailed and constructive feedback. We particularly appreciate your concerns regarding the diagnostic validity of the controller comparison, the role of explicit judgment tokens, and whether our findings extend beyond teacher-forced action-margin evaluations.
We conducted several additional experiments to address these concerns more directly.
1. Matched-upstream controller comparison
To address the concern that the weaker effect of judgment-native interventions might result from different intervention positions or perturbation decay, we derived an action-native controller and injected it at the same upstream judgment position as the judgment-native controller.
Both interventions use the same case-wise recipient-norm matching procedure, with a matched-norm random direction as a control.
The action-native controller retains a significantly stronger effect on downstream action preference across all three models:
- Qwen: paired difference of 0.0656 (95% CI: [0.0537, 0.0783]).
- Gemma: paired difference of 0.0480 (95% CI: [0.0339, 0.0634]).
- Mistral: paired difference of 0.0171 (95% CI: [0.0107, 0.0238]).
All nine semantic-family means are positive for each model, with family-level exact sign-flip tests yielding p = 0.00391. We believe this provides a more direct comparison of judgment- and action-side causal control while controlling for intervention position and norm.
2. Separating latent judgment control from explicit BLOCK context
We added a factorial intervention that independently varies externally supplied ALLOW/BLOCK text and latent judgment interventions. The results show that explicit BLOCK text can strongly shift action preference, whereas latent judgment interventions have substantially smaller and architecture-dependent effects.
We also conducted natural-state counterfactual swaps and clarified the distinction between effects with the explicit judgment label held fixed and effects when the label is allowed to follow the model's judgment readout.
3. Extending the analysis to free generation and stateful behavior
We added free-generation experiments without teacher-forcing BLOCK, allowing us to measure spontaneous judgment generation and subsequent action distributions. We also extended the evaluation to multi-step in-memory sandbox trajectories.
These experiments reveal important model-dependent differences. For example, Gemma exhibits a pronounced BLOCK-then-unsafe pattern, while Qwen and Mistral produce different distributions of non-tool responses and alternative tool proposals. We also explicitly acknowledge that the sandbox evaluation is an in-memory simulation rather than external tool execution, and that our interventions do not consistently reduce unsafe proposals across all models.
Taken together, these additional results help refine the scope of our central claim: the tested judgment-native directions exert substantially weaker causal control over action preference than action-native directions, even under matched upstream intervention conditions. We do not claim that every possible representation of safety judgment is causally disconnected from action selection.
We sincerely hope these additional experiments address your concerns about intervention comparability and the broader behavioral interpretation of our findings.
We would greatly appreciate it if you could take another look at our responses when you have a chance. As the discussion deadline is approaching, we are concerned that there may not be sufficient time to address any further methodological questions or concerns that emerge closer to the deadline. If there are still aspects of the experimental design or causal interpretation that you find insufficiently clear, we would be very grateful if you could share them with us at your earliest convenience, so that we can provide further clarification before the discussion closes.
Thank you very much again for your time and thoughtful consideration!
Best regards,
The Authors
Response to Reviewer 7zVn
Response to Reviewer 7zVn
We thank the reviewer for emphasizing the distinction between information availability, latent judgment control, and action selection. Our response adds matched-upstream controls, factorial and natural-state interventions, and free-generation and sandbox evaluations to clarify the causal relationship and its behavioral scope.
1. Mechanistic contribution: from behavioral inconsistency to causal control
Prior work documents judgment–action gaps. We ask whether safety-relevant information remains readable in action-side states, whether variables controlling explicit judgment exert comparable causal control over action preference, and how the control structures underlying judgment and action relate. A judgment-trained probe transfers to action states without refitting (best leave-family-out J→A AUROC: Qwen 0.875, Gemma 0.840, Mistral 0.756; descriptive maxima across layers). Hidden-state interventions test causal control; subspace and orthogonal-component analyses characterize its structure. The orthogonal action component retains an effect of 0.0240 versus 0.0238 for the full action controller at Mistral L24, and 0.123 versus 0.0110 for the judgment-parallel component at Qwen L32. We therefore provide a causal analysis of information availability and the distinct control structures underlying judgment and action.
2. Matched-upstream controller comparison
To test whether weaker judgment-to-action transfer reflects injection position or perturbation decay, we derived an action-native controller, cAup, from safe-oriented action gradients and injected it at the same upstream judgment position as gJsafe. Both use the same case-wise recipient-norm matching procedure and teacher-forced candidate-action endpoint; a matched-norm random direction serves as control.
Across nine semantic families, we use 20 training and 10 distinct evaluation variants per family (90 cases). Families overlap, but scenario variants do not; evaluation cases are not filtered by baseline KBD phenotype, judgment output, action margin, or generated behavior.
CIs are family-cluster bootstrap 95% intervals (20,000 resamples). All nine family means are positive in each model; the family-level exact sign-flip test gives two-sided p=0.00391 for each (512 configurations, assuming family-level sign-exchangeability). The action-native controller remains more effective under matched position and perturbation norm, so injection distance alone cannot explain the asymmetry. The controllers remain endpoint-native by construction—gJsafe uses judgment gradients and cAup action gradients—so this comparison concerns the action-side efficacy of these tested directions, not all possible judgment representations.