acceptodds
Under review as a conference paper at ICLR 2027

The Monitorability Geometry of Language Models

Abstract

Linear activation monitors, whether probes, subspace monitors, or a model's own self-report, read only a few directions of a high-dimensional hidden state, so something always falls in their blind spot. We give a label-free, closed-form way to measure how much control over a model's behavior survives outside any rank- monitor, from two Jacobians taken at one hidden state, one for what the model does and one for what it says about itself. From their geometry we bound what a hidden intervention can achieve, find the best monitoring subspace, and connect visibility to detection. One geometry governs both sides. A monitor fixed before deployment captures of a model's behavioral leverage at rank and at rank (Llama, Qwen3.5, Gemma, and OLMo; 1B–70B), but rank is not the lever. Against an attacker who accepts visibility, widening from rank to lowers guaranteed evasion only from to . What visibility concedes, no rank buys back. The blind spot is concrete. A steer along the refusal direction a rank- activation-PCA monitor cannot see flips refusal on AdvBench across four families while staying invisible (), whereas the same steer in the monitor's view barely moves it. The cause is that the variance directions these reads occupy align poorly with what drives behavior. On Gemma-4-12B a rank- PCA read captures of the refusal direction, below the random null, against for the matched Jacobian subspace. Spending that visibility is also what catches an attacker. Once the steer is strong enough to flip refusal, a Fisher-based monitor detects it (AUROC –) where PCA stays blind (). Introspection follows the same geometry in sharper form. – of the behavior-driving direction lies outside the self-report subspace, so a self-report can be forged, moved to say one thing while behavior stays fixed. A single overlap between the report and behavior directions predicts how faithful that forgery can be, which we confirm causally by fine-tuning models to move it. The same account predicts when a model's report that it is being evaluated comes apart from how it acts. Together these results give a concrete calculus for choosing a monitor's width, threshold, placement, and subspace, and for knowing what it must miss.

Primary area:
foundation or frontier models, including LLMs
Keywords:
interpretability, AI safety, activation monitoring, linear probes, mechanistic interpretability, Fisher information, steering vectors, model introspection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2bGZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the limitations of internal monitoring in LLMs from a geometric perspective. The authors introduce a framework based on behavior and self-report Jacobians to characterize the blind spots of rank-limited monitors. The analysis provides bounds on monitor capacity, adversarial evasion, and self-report reliability. Experiments across multiple LLM families support the theoretical findings and demonstrate that Fisher-based monitors can detect behavioral interventions that conventional activation-based monitors miss.

Critical strengths
  1. The paper addresses an important problem in LLM safety, providing a principled perspective on the limitations of activation monitoring and model self-report.

  2. The theoretical framework is interesting and connects monitor capacity, adversarial evasion, and introspection through a unified geometric formulation.

  3. The experiments cover multiple model families and include meaningful intervention studies. The comparison between Fisher-based and PCA-based monitors provides useful evidence supporting the proposed analysis.

Critical weaknesses
  1. The theoretical guarantees primarily rely on local, first-order approximations at a single hidden-state position. It remains unclear how well these results extend to nonlinear behavioral changes or interventions distributed across multiple tokens and layers.

  2. The analysis focuses mainly on rank-limited linear monitors. Although several nonlinear probes are evaluated, the broader applicability of the theoretical conclusions to more sophisticated monitoring architectures is not fully established.

  3. The self-report analysis relies on a fixed introspective query and a restricted set of concept labels. This may not adequately capture the complexity of natural language self-reporting.

  4. The practical computational cost of constructing the required Jacobians and Fisher subspaces is insufficiently discussed. More detailed scalability and efficiency evaluations would help establish the feasibility of the proposed monitoring framework.

Questions for authors

Please refer to Weaknesses.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer 2bGZ (1/3)

Thank you for the positive assessment and for valuing the unified framework, which reads monitoring capacity, evasion and introspection from two Jacobians taken at one hidden state, and the evidence that behaviour-sensitive subspaces outperform activation PCA across model families. You asked how far the local, first-order guarantees extend to nonlinear behavioural changes and to interventions across tokens and layers, whether the conclusions hold for more sophisticated, nonlinear monitors, whether a fixed introspective query with restricted labels captures natural-language self-report, and what the computation costs; the revision answers each with new measurements. We show where the local theory stops (steering at every position, and refusal, which flips like a switch) and how it still gives a monitor across layers: a read of two adjacent layers catches the steering that every single-layer monitor misses, while a LoRA, which also changes the earlier layer, gets past it. For nonlinear monitors, the blind spot holds exactly for any monitor that reads a low-rank projection of the state, MLP probes do not transfer across attack families, and monitors that read SAE features, which we added, miss almost all of what drives behaviour. All changes are in the revised PDF, marked in blue.

In all the new experiments we grade an attack on the text the model generates, where the HarmBench classifier judges harm and a second LLM judge grades refusal and coherence, and an attack succeeds only when the completion is harmful, coherent and not a refusal. We fit every monitor on clean traffic only and report detection at a 1% false-alarm rate on clean prompts.

W1. Local, single-position guarantees: nonlinear changes and interventions across tokens and layers

We measured how far the single-site result reaches and where it stops. Across positions and under nonlinear change, the best fixed steering direction we found on Llama-3.1-8B, i.e., the difference-of-means refusal direction, gives 61% harmful, coherent completions at 0.6‖h‖ when we add it at every position, compared to 7% when we add it at the last position only. Its first-order effect is small, since our theory predicts a 2.2 log-odds change at 1.0‖h‖, while we measure 25. This discrepancy shows the limit of local, first-order approximations for nonlinear behavioural transitions: refusal behaves like a switch, which an edit with a small local effect can still flip once the edit is large enough.

Across layers, the public emergent-misalignment fine-tunes of Llama-3.1-8B change every layer, and the whitened Fisher read catches them at AUROC 0.99 to 1.00 (activation PCA: 0.44 to 0.71). For steering that evades every single-layer monitor, the paper's result that monitors combine across layers (Prop. 2) gives a monitor that catches it: we predict the steered layer from the layer before it and apply a rank-16 whitened read to the part the prediction misses. When we do not know where the edit is, reading all 31 adjacent layer pairs at once catches the difference-of-means steer injected at any of six layers whenever it gives 44% or more successes (Fig. 13).

Llama-3.1-8Bsuccessundetected by every single-layer monitorundetected by the cross-layer read
difference-of-means steer, every position, 0.6‖h‖0.610.600.00
same steer, last position only0.07
LoRA on layers 0 to 19, trained against the single-layer monitors0.850.850.85

The method stops at attacks that also change the earlier layer the cross-layer read uses as its reference, such as the LoRA in the last row, which gets 85% of completions past every monitor we have, and the limitations now say this.

Changed in the paper: one sentence after Def. 1 (Sec. 3.2); Sec. 5.3; the cross-layer note in App. G and its construction in App. K; Fig. 13 (App. O.1); limitations in Sec. 6.

AuthorsOpenReview ↗

Reply to Reviewer 2bGZ (2/3)

W2. Mainly rank-limited linear monitors; more sophisticated monitors

Part of our result holds exactly for any monitor, because a monitor that reads the state through a rank-r projection, f(Ph), cannot see any edit that P maps to zero whatever f is, and any differentiable monitor with k outputs acts, around a given state, like a rank-k linear read, so our first-order results apply to its Jacobian.

In experiments, more expressive monitors do not escape this picture. MLP probes trained on one attack family and tested on others are unreliable (mean AUROC 0.53 to 0.56 on seven models from 1B to 70B), while the unsupervised whitened Fisher read reaches 0.65 to 1.00 on the same held-out families (Table 12). With a public SAE for Llama-3.1-8B (65,536 features), a Mahalanobis read of the most variable features catches no steer below 1.0‖h‖, and our theory explains why: locally such a read spans a median of 14.5 directions and misses 99.5% of the Fisher leverage, about as much as a random read of that rank (99.8%; the whitened read misses 57%). The table gives the detection rate at a 1% false-alarm rate (Llama-3.1-8B, Table 3 of the revision):

monitorrandom steer, 0.6‖h‖difference-of-means steer, 1.0‖h‖local rankFisher leverage unseen
whitened read (rank 16)1.000.00160.57
full-rank Mahalanobis1.000.0040960
SAE, 16 most variable features0.000.0031.00
SAE, 256 most variable features0.000.0014.51.00
SAE reconstruction error1.000.28

A more expressive monitor is also not automatically safer: a steering vector trained against full-rank Mahalanobis, the norm and the whitened read together evades all three (55 to 68% coherent harmful completions over three seeds), and only the read across layers catches it.

Changed in the paper: a bullet on nonlinear monitors and Table 3 (SAE monitors) in App. M; Table 12 (held-out families).

W3. Fixed introspective query and restricted labels

We re-measured the report geometry under nine query wordings (the paper's, the variants our experiments used, and five paraphrases), on two disjoint halves of the 288-concept bank, and with a free-text versus a closed read, on six models. The report subspace changes with the wording but stays far from chance, since the overlap between any two queries is 0.23 to 0.90 (chance 0.07 to 0.14), and free and closed reads give the same subspace (overlap 1.00). The restricted labels do matter, and the paper now says so: with the label sets our experiments used (8 to 16 labels), 94 to 99% of the behaviour-maximising direction lies outside the report subspace on Llama, Qwen3.5 and OLMo, and 80 to 89% on Gemma-4-12B, while over the full concept bank 42 to 75% lies outside on Llama, Qwen3.5 and OLMo. The abstract now says "for the concept labels we query" (Table 1 of the revision):

Llama-3.1-8BOLMo-3-7BQwen3.5-2BQwen3.5-4BQwen3.5-9BGemma-4-12B
report-subspace overlap across nine wordings0.46 to 0.900.23 to 0.670.39 to 0.870.45 to 0.890.30 to 0.850.75 to 0.85
chance0.070.070.140.110.070.08
behaviour direction outside, small label sets0.97 to 0.990.97 to 0.990.97 to 0.990.94 to 0.980.98 to 0.990.80 to 0.89
same, full concept bank0.67 to 0.700.71 to 0.750.50 to 0.570.42 to 0.500.67 to 0.720.00 to 0.01
AuthorsOpenReview ↗

Reply to Reviewer 2bGZ (3/3)

W3 (continued)

For natural-language self-report, we reran the introspection experiment with real concept vectors injected into the model, a Gaussian control of the same norm, and free-text answers graded by an LLM judge. The table gives the change in the yes/no detection log-odds at 0.8‖h‖:

injected vectorQwen3.5-9BOLMo-3-7B
concept+0.68+3.55
its part outside the report subspace+0.64+3.28
its part inside the report subspace+0.27+1.97
Gaussian vector, same norm+0.07+0.78

The part of the concept outside the report subspace moves the answer about as much as the whole concept, while the Gaussian control moves it much less. When we inject at the last token only, no arm moves detection by more than 0.06 log-odds on these two models, and the model rarely names the injected concept (at most 16%), while Llama-3.1-8B and Gemma-4-12B give mixed detection results. The first-order report subspace therefore does not predict what a model says it detects, and the submitted Fig. 3 showed a flat blind arm only because its construction did not inject any concept.

For forgery, we measured fidelity at finite edit size. The first-order prediction holds to within 0.01 on Qwen3.5-2B, 4B and 9B and OLMo-3-7B for edits up to 0.05‖h‖, where the overlap between report and behaviour is 0.02 to 0.09. When the overlap is large, as with the LoRA on Llama-3.1-8B that we trained to make the report depend on behaviour (overlap 0.72), the measured fidelity is higher than predicted by 0.05 to 0.46. We ran Gemma-4-12B in bfloat16, and its report changes at these edit sizes are smaller than the precision of the measurement, so it does not test the prediction. The table gives measured minus predicted fidelity, median over prompts (Fig. 16 of the revision):

model, reportoverlap0.01‖h‖0.05‖h‖0.1‖h‖
Qwen3.5-9B, concept labels0.03-0.000-0.000+0.001
OLMo-3-7B, concept labels0.06+0.000+0.002+0.006
Qwen3.5-2B with its LoRA, concept labels0.06+0.001+0.007+0.016
Llama-3.1-8B with its LoRA, comply or refuse0.72+0.052+0.281+0.460

The abstract now says that the overlap predicts fidelity when the overlap is low.

Changed in the paper: abstract, intro and Sec. 5.4 ("for the concept labels we query"); Table 1 (App. I.4); the introspection paragraph of Sec. 5.4 with Fig. 3 and Fig. 21 (the submitted Figs. 3 and 17); forgery in Sec. 5.2, Tables 14 and 16, and Fig. 16 (App. O.2); examples in App. O.7.

W4. Computational cost of the Jacobians and Fisher subspaces

We compute the Jacobians and the monitor once, offline, and at deployment a rank-16 monitor costs one 16 x d projection and one 16 x 16 quadratic form per token. Measured on one RTX A6000 (Table 2 of the revision):

Llama-3.2-1BLlama-3.1-8BQwen3.5-9BGemma-4-12BQwen3.5-27B (4-bit)
one generated token13 ms47 ms49 ms48 ms61 ms
Def. 1 Fisher, one prompt (top-64 rows plus 16 sampled)0.4 s1.6 s2.8 s0.7 s6.3 s
Def. 1 Fisher, 256 prompts, two layers (offline, once)1.3 min1.8 min9.4 min6.4 min27 min
fit all monitors (CPU, once)5 s34 s34 s22 s52 s
score one state, rank-16 whitened read (GPU)44 us41 us38 us38 us38 us
same, batched (per state)0.2 us0.2 us0.2 us0.2 us0.3 us

Scoring a state costs 0.06 to 0.3% of the time to generate one token when done one state at a time, and is negligible when batched. The per-prompt and per-token rows use float32 for 1B to 9B, bfloat16 for Gemma-4-12B and 4-bit for 27B; the 256-prompt row is the time our pipeline took (float32 for 1B, bfloat16 for 8B and 9B, 4-bit for Gemma-4-12B and 27B; Llama-3.3-70B in 4-bit took 34 min).

Changed in the paper: Table 2 (App. K) with a pointer from Sec. 4.

We hope these measurements answer your questions about scope, and we would be glad to answer any further question.

AuthorsOpenReview ↗

Gentle reminder before the discussion closes

Dear Reviewer,

Thank you again for your encouraging assessment and valuable suggestions. With the discussion period nearing its end, we wanted to kindly follow up on our detailed responses. We have added new experiments addressing all four concerns, including nonlinear and cross-layer interventions, more expressive monitors, expanded introspection evaluations, and computational efficiency measurements. We have also included qualitative examples with code snippets in Appendix O.7, and highlighted all revisions in blue.

We would be very grateful if you could revisit the updated manuscript and let us know whether the additions have addressed your remaining concerns. If you find the revisions satisfactory, we would sincerely appreciate your reconsideration of the assessment.

Thank you again for your time and consideration.

Reviewer_rWrzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the structural blind spots of activation monitors through the geometry of two Jacobians evaluated at a shared hidden state: a Fisher-weighted Jacobian of the next-token distribution and a Jacobian of a fixed self-report query. The authors characterize the sensitivity captured by rank-limited monitoring subspaces, derive an evasion frontier under visibility constraints, and analyze interventions that separate behavioral changes from self-report changes. Experiments across several language-model families compare Fisher-based monitoring with activation PCA and supervised probes, examine refusal interventions and monitor-aware attacks, and manipulate report–behavior overlap through fine-tuning. The results show that subspace selection matters for detecting internal interventions, while also revealing limitations in translating local geometric alignment into reliable behavioral monitoring.

Critical strengths

The paper makes a useful distinction between identifying behavior-related information in activations and observing interventions that causally change model outputs. The joint behavior–report framework provides a coherent way to analyze these different forms of observability. Its treatment of monitor capacity, visibility, and residual sensitivity offers a concrete diagnostic perspective beyond ordinary probe accuracy. The empirical investigation includes meaningful controls and interventions. Comparisons between PCA and behavior-sensitive subspaces demonstrate that high activation variance need not identify directions relevant to output changes. The fine-tuning experiment that increases report–behavior overlap is also valuable: it makes report rewriting harder without producing a consistently reliable self-report monitor.

Critical weaknesses

The central measure of “behavioral leverage” captures local sensitivity of the next-token distribution, whose relationship to safety-relevant behavior under finite interventions remains insufficiently established. Figure 21 illustrates this gap directly: a difference-in-means direction with very small local leverage reduces refusal from 0.95 to 0.28, whereas the top Fisher direction, with approximately 260 times greater leverage, does not flip refusal and instead produces repetition. Although the paper acknowledges this distinction and includes task-level checks, these results substantially limit the interpretation of Fisher-energy coverage as behavioral monitorability. The geometric framework is informative, but its incremental contribution to monitoring consequential behavior is less clearly demonstrated. The constructive defense claims are broader than the guarantees and experiments justify. The theory concerns local, single-site perturbations and requires a norm budget; Appendix G explicitly notes that an unrestricted attacker can amplify the blind component. It remains unclear how the proposed injection-magnitude constraint would be enforced when internal changes arise through adversarial inputs rather than an observable activation edit. The adaptive experiments demonstrate a tradeoff for particular attacks, but do not establish that an effective attacker must generally be detected or become harmless. The reported residual refusal-flip rate of 0.47 on Qwen also warrants a more qualified interpretation.

Questions for authors
  • Given Figure 21, what precise safety-relevant property does high Fisher-energy coverage guarantee beyond local next-token sensitivity? Which existing task-level results demonstrate that it predicts successful monitoring of coherent behavioral changes?

  • How would the required perturbation-norm budget be measured and enforced in deployment when the intervention is induced by an adversarial prompt and the corresponding unperturbed hidden state is unavailable?

  • For monitor-aware attacks, could you report the joint rate of successful, coherent harmful completions that evade detection at a fixed clean false-positive rate? In particular, how does the remaining 0.47 refusal-flip rate on Qwen translate into successful and undetected attacks?

4 comments
AuthorsOpenReview ↗

Reply to Reviewer rWrz (1/3)

Thank you for the careful reading and the precise questions, and for valuing the joint framework for behaviour and self-report, the diagnostics that go beyond probe accuracy, and the LoRA experiment on the overlap between report and behaviour. Your central question, what the local Fisher geometry guarantees about attacks that succeed on generated text, was our main priority in this answer and it greatly improved the paper. The answers are now measured on generated text across five models: a closed-form detection rate predicts which edits a monitor detects, the subspace the theory selects catches the edits whose effect is local, and the paper's result on combining monitors across layers gives a monitor that catches the strongest steering vector we found, which every single-layer monitor misses. We also separate what we prove from what we measure and state the limits plainly. All changes are in the revised PDF and are marked in blue color.

In all the new experiments, we grade an attack on the text the model generates, where the HarmBench classifier judges harm and a second LLM judge grades refusal and coherence, and an attack succeeds only when the completion is harmful, coherent and not a refusal. We fit every monitor on clean traffic only and report detection at a 1% false-alarm rate on clean prompts.

Changed in the paper: Sec. 4 and the Sec. 5 intro (judging protocol); App. K (protocol paragraph).

W1. Leverage is local next-token sensitivity, and Fig. 21 limits reading Fisher coverage as monitorability

When we graded the submitted Fig. 21 (now Fig. 25) again on text, your point became even clearer. The submitted panel graded refusal without checking coherence, so when the top Fisher direction made the model repeat itself we counted that as refusal, and leverage now also uses the Def. 1 Fisher. Judged on text on Llama-3.1-8B, the two directions compare as follows:

steering directionlocal leverage (Def. 1, relative to the top Fisher direction)lowest judged refusalhighest success (harmful, coherent, not refused)
difference of means0.0080.03 (from 0.91)0.61
top Fisher direction10.730.15 (repetition)

The difference-of-means direction has only 0.008 of the top Fisher direction's local leverage, yet it takes refusal from 0.91 to 0.03 and gives up to 61% harmful, coherent completions, while the top Fisher direction never brings refusal below 0.73, gives at most 15%, and makes the model repeat itself (Fig. 25). The reason is that refusal behaves like a switch: for the difference-of-means direction, our first-order theory predicts a 2.2 log-odds change at 1.0‖h‖, while we measure a change of 25. Leverage therefore ranks edits by their local effect rather than by whether they flip a switch-like behaviour, so we no longer read Fisher-energy coverage as monitorability of the behaviour, and Q1 explains what leverage does predict on text.

Changed in the paper: one sentence after Def. 1 (Sec. 3.2); Fig. 25 (the submitted Fig. 21) and its paragraph in App. O.4; examples in App. O.7.

W2. The constructive defence claims are broader than the guarantees

You are right that our theory is local, applies to one site and needs a norm budget, and that "caught or harmless" claims more than that, so the revision now states three things separately. What we prove is that, at one site, an edit the monitor cannot see has only a small local effect (Theorem 1). What we measure, for activation steering, is that a whitened read across two adjacent layers catches the steering that every single-layer monitor misses, and that, paired with a probe for harmful requests, it leaves at most 2% of completions successful and undetected across 222 steering settings that were not trained against both (Llama-3.1-8B). The limits are that an attacker trained against both the cross-layer read and the probe evades both (47 to 74%), and that a LoRA trained against the single-layer monitors gets 85% (Llama-3.1-8B) and 91 to 94% (Qwen3.5-9B) of completions past every monitor we have, as Bailey et al. (2026) found for latent-space monitors. Q2 explains how we measure the budget when the change comes from the input.

Changed in the paper: contribution 4, retitled "A monitor that detects what single-layer monitors miss" (p. 2); Sec. 5.3 retitled; limitations in Sec. 6.

AuthorsOpenReview ↗

Reply to Reviewer rWrz (2/3)

W3. The adaptive experiments show a trade-off for particular attacks; the 0.47 on Qwen

Thank you for focusing on this, because the 0.47 measured the wrong thing, it counted the sign of a log-odds margin on the first token rather than the generated text. Judged on text, steering works much less well on Qwen3.5-9B, where the difference-of-means steer reaches 26% and the cross-layer read leaves at most 1% of completions successful and undetected over all 119 steering settings, the same 1% as with no attack. We agree with your reading that each monitor, or pair of monitors, is evaded by an attacker trained against it while catching attackers trained against something else, and Q3 gives the full joint rates.

Changed in the paper: the 0.025 and 0.47 sentences in Sec. 5.3 replaced by rates judged on text; Fig. 11 and its bullets in App. O.1; Table 7.

Q1. What Fisher coverage guarantees beyond local sensitivity, and which task-level results show it predicts monitoring

The guarantee itself is local: at one site, an edit that a rank-r monitor cannot see changes the next-token distribution by at most 1/2 sigma_{r+1}^2 ‖delta‖^2 to second order, and the main text now claims only that.

At the task level, the same geometry predicts which edits a monitor catches. Using only clean statistics and the edit, a noncentral chi-square gives a predicted detection rate at a 1% false-alarm rate, which we compared with the measured rate for every fixed steer and trained attack vector on five models. The mean absolute error is 0.03 for the whitened read, 0.07 for the Def. 1 Fisher read and 0.06 for full-rank Mahalanobis, and the per-model errors are below (Table 10 of the revision):

modeleditswhitened readDef. 1 Fisher readfull-rank Mahalanobis
Llama-3.2-1B1890.0290.0660.052
Llama-3.1-8B2290.0490.0730.074
OLMo-3-7B1930.0370.0630.075
Qwen3.5-9B2010.0360.0830.063
Gemma-4-12B (4-bit)2030.0080.0660.042

Most edits are either clearly caught or clearly missed, and near the threshold the measured detection switches on more sharply than the Gaussian prediction (Fig. 12).

The geometry also explains which edits each monitor catches. Edits whose effect is local, such as the refusal-margin gradient and the top Fisher direction, are invisible to a rank-16 PCA monitor (detection at most 0.05) and are caught by the whitened Fisher read at every norm; they change the output a lot but give at most 21% coherent harmful completions, which is what Fig. 25 shows.

The switch-like edit is caught across layers instead. The difference-of-means steer moves the state into the region of the steered layer where benign requests sit, so every single-layer monitor misses it (detection at most 0.07 up to 1.0‖h‖), but the layer just before the edit still carries the content of the request. We therefore predict the steered layer from that earlier layer and apply a rank-16 whitened read to the part the prediction misses, which catches the steer at every norm from 0.3‖h‖ up (detection 1.00). It uses only results already in the paper: monitors combine across layers (Prop. 2), and the whitened read applies with the conditional covariance. When we do not know where the edit is, reading all 31 adjacent layer pairs at once catches the steer injected at any of six layers whenever it gives 44% or more successes (Fig. 13). What the coverage does not predict is whether a switch-like behaviour flips, and Sec. 3.2 and the limitations now say so.

Changed in the paper: Sec. 5.3; the cross-layer note in App. G; App. O.1 (Figs. 12 and 13, Table 10).

Q2. Measuring the norm budget when the change comes from an adversarial prompt

We do not need an unperturbed state, because every monitor scores a state against clean statistics, the mean and covariance of normal traffic, rather than against a paired clean state. The budget is therefore the state's Mahalanobis distance from clean traffic, which the full-rank monitor measures on any state, and the cross-layer read takes its reference from an earlier layer of the same forward pass.

On prompt attacks this measure is well defined, but it flags the unusual prompt rather than the harm. On Llama-3.1-8B, 20 HarmBench human jailbreak templates give 0 to 81% successful completions, and when we wrap the same templates around harmless requests, the whitened read and the cross-layer read flag a median of 100% of them. A soft prompt trained against the single-layer monitors also gets past the cross-layer read (up to 69% successful and undetected), because it changes the read's earlier reference layer too, and the probe for harmful requests catches only part of it (at most 51% gets past both). We therefore state that our defence results are for activation steering.

Changed in the paper: a bullet on the norm budget in App. G; the limitations in Sec. 6; Table 8 (App. O.1).

AuthorsOpenReview ↗

Reply to Reviewer rWrz (3/3)

Q3. Joint rate of successful, coherent, harmful and undetected completions at a fixed clean false-alarm rate

The refusal-flip rates in Fig. 11, including the 0.47, counted the sign of a first-token log-odds margin, and we replaced them with the joint rates you asked for, judged on text. The table gives them on Llama-3.1-8B, with 150 held-out requests per setting and a 1% false-alarm rate on clean prompts; for trained attackers it gives the range over the penalty weight, seed 0:

attacksuccessundetected by every single-layer monitorundetected by the cross-layer readundetected by the cross-layer read and the probe
no attack0.050.050.050.00
difference-of-means steer, 0.6‖h‖0.610.600.000.00
trained against whitened read, Mahalanobis, norm0.57 to 0.680.57 to 0.660.00 to 0.020.00
trained against cross-layer read, Mahalanobis, norm0.63 to 0.730.63 to 0.730.63 to 0.730.01 to 0.02
trained against the probe alone0.17 to 0.270.000.000.00
trained against cross-layer read, Mahalanobis, norm, probe0.47 to 0.740.36 to 0.690.47 to 0.740.47 to 0.74
LoRA trained against whitened read, Mahalanobis, norm0.850.850.850.85
soft prompt trained against single-layer monitors0.00 to 0.690.00 to 0.690.00 to 0.690.00 to 0.51

Across all 119 fixed steering settings, the cross-layer read leaves at most 11% of completions successful and undetected (5% with no attack). An attacker trained against the whitened read, Mahalanobis and norm monitors evades every single-layer monitor but is caught by the cross-layer read. An attacker trained against the cross-layer read evades it by editing along the directions where the mismatch between layers already varies most, which are exactly the edits Theorem 1 says a fixed monitor must miss, but it leaves the content of the request readable, so the probe flags 98% of its requests. With the cross-layer read and the probe together, at most 2% of completions are successful and undetected across the 222 steering settings not trained against both, while attacks on the weights or the input change the read's earlier reference layer and get through.

On Qwen3.5-9B, the model of the 0.47, 1% of completions succeed with no attack. The difference-of-means steer reaches 26%, all of it past every single-layer monitor, while the cross-layer read leaves at most 1% undetected across all 119 steering settings, and with the probe at most 1% gets through over the 198 steering settings not trained against both. An attacker trained against both evades both (30 to 75%), and a LoRA trained against the single-layer monitors gets 91 to 94% of completions past every monitor. The other models agree (Table 6 of the revision):

modelno steerdifference-of-means steer: successits share missed by every single-layer monitormost successful and undetected by the cross-layer read, over 119 steerssame, cross-layer read and probe
Llama-3.2-1B0.070.611.000.500.15
Llama-3.1-8B0.050.610.990.110.02
OLMo-3-7B0.000.070.730.010.01
Qwen3.5-9B0.010.261.000.010.01
Gemma-4-12B (4-bit)0.010.690.990.030.01

Llama-3.2-1B is the exception, because weak steering vectors already work on such a small model, so up to 50% pass the cross-layer read and the probe brings that down to 15%.

False alarms also need care, because a monitor fit on narrow traffic flags benign traffic of other kinds. When we fit the cross-layer read only on the paper's clean set, it flags 92% of non-English chats, and when we add a third of each benign source to the fit, false alarms on held-out prompts from those sources fall to 0 to 8% at a nominal 1%, while the read still catches the steering attacks at 1.00 (Table 9 of the revision):

modelcross-layer read on non-English chats: narrow fitbroad fitprobe on OR-Bench hard (benign, looks harmful): narrow fitbroad fit
Llama-3.2-1B0.870.010.820.06
Llama-3.1-8B0.920.010.880.05
OLMo-3-7B0.640.010.930.05
Qwen3.5-9B0.590.030.970.06
Gemma-4-12B (4-bit)0.550.030.970.05

A deployed monitor therefore has to be fit on traffic like the traffic it will see, and the limitations now say so.

Changed in the paper: Sec. 5.3 (joint rates); App. O.1 (Tables 6 to 9, Fig. 18); limitations in Sec. 6.

We hope these answers address your concerns, and we would be glad to answer any further question.

AuthorsOpenReview ↗

Gentle reminder before the discussion closes

Dear Reviewer,

Thank you again for your detailed and constructive feedback. With the discussion period nearing its end, we wanted to kindly follow up on our responses and the revised manuscript. We have made substantial additions to address your concerns, particularly through generated-text evaluations, cross-layer monitoring experiments, and clearer distinctions between our theoretical guarantees and empirical findings. We have also added qualitative examples and code snippets in Appendix O.7, with all changes highlighted in blue.

We would be very grateful if you could revisit our responses and let us know whether your concerns have been addressed. If so, we would sincerely appreciate your reconsideration of the assessment before the discussion period closes.

Thank you once again for your time and consideration.

Reviewer_Z99MOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies whether activation monitors observe the hidden-state directions that change a language model’s behavior. It constructs two Jacobians at the same hidden state: one for the model’s next-token distribution and another for its self-report. The paper uses their geometry to characterize the behavioral sensitivity that remains outside a low-rank monitor, derive an optimal monitoring subspace, and analyze the trade-off between behavioral changes and monitor visibility.

Experiments across several model families compare Fisher-based monitors with activation-PCA and supervised probes, evaluate held-out and adaptive attacks, and examine how changing report–behavior overlap affects self-report manipulation.

Critical strengths
  • The distinction between classifying examples of a behavior and detecting interventions that change that behavior is insteresting. The paper provides a concrete framework for studying why success on the first task does not necessarily imply success on the second.

  • The evaluation looks nice and goes beyond geometric measurements. The monitor-aware attack in Figure 11 and the report–behavior overlap intervention in Table 6 provide useful tests of the proposed framework, although I have concerns about what these results establish. See W1.

Critical weaknesses

W1. Low leverage does not mean an attack fails. I find the “caught or harmless” claim a little bit too strong. The theory measures local changes to next-token probabilities, not whether an attack succeeds, which I don't think they are necessarily equivalent. I think they should narrow the claim or show how the bound predicts actual attack success.

W2. The baseline comparison is not fair. In Table 3, the Fisher monitor’s directions are estimated within the test attack family, while the supervised baselines must generalize to a different attack family. I do not see how this isolates subspace choice. Figure 11 uses a matched setup, but only on Llama-3.1-8B. I encourage the authors to use that setup across the models in Table 3.

W3. Does the theory apply to the implemented monitor? Definition 1 constructs the Fisher matrix using all possible next tokens, weighted by their probabilities. However, Appendix K, if I understand it correctly, builds the attack monitor from gradients of only the most likely token per prompt. These are not generally equivalent, so I am not entirely sure why the optimality result should carry over. The authors should justify this approximation and compare it with the construction used in the theory.

W4. The 99.7% coverage claim is misleading. Figure 12 claims that a rank-16 monitor covers 99.7% of the solid angle. However, this number is derived from a sensitivity ratio, not an angular measurement. The ratio tells us how strong an unseen effect can be relative to the strongest possible effect, not what fraction of directions the monitor sees. I think this distinction matters: a weak unseen effect is not the same as a small blind region.

Questions for authors

See W1-W4

4 comments
AuthorsOpenReview ↗

Reply to Reviewer Z99M (1/3)

Thank you for your time and for the detailed technical review. We also thank you for your critical comments and encouragement. It is encouraging to hear that you found the distinction between classifying examples of a behaviour and detecting the interventions that change it to be interesting, this distinction is the core of our paper. Our contribution is a label-free, closed-form measurement framework that uses the behaviour Jacobian at a single hidden state to quantify how much of the behavioural change that edits can cause is observable by a rank-r monitor, which subspace maximises this observability, and how much an edit invisible to the monitor can still change the model's behaviour, all before a detector is trained or an attack is observed. The revision shows that the subspace selected by the theory can detect attacks built independently of any monitor, where even activation PCA and supervised probes fail. Further, it shows that a closed-form detection rate predicts which edits can be detected by a monitor, and that the theory itself identifies a monitor that can detect the strongest steering vector we found. All changes are marked in blue color in the revised draft.

In all the new experiments, we grade an attack based on the text generated by the model, where the HarmBench classifier judges harm and a second LLM judge grades refusal and coherence. An attack succeeds only when the completion is harmful, coherent, and not a refusal. We fit every monitor on clean traffic only and report detection at a 1% false-alarm rate on clean prompts.

Changed in the paper: Sec. 4 and the Sec. 5 intro (judging protocol); App. K (protocol paragraph).

W1. "Caught or harmless" is stronger than what the theory proves

You are right that Theorem 1 bounds how much an edit changes the next-token distribution at one site and does not bound whether an attack succeeds. What we prove is that an edit that remains undetectable by a rank-r monitor has only a small local effect at that layer, and contribution 4 now restricts its claim to this.

The same discrepancy is evident in the model's generated text. On Llama-3.1-8B, steering along the difference-of-means refusal direction at 0.6‖h‖ makes 61% of completions harmful and coherent (compared to 5% without steering). Yet, our first-order theory predicts only a 2.2 log-odds change at 1.0‖h‖, whereas the measured change is 25. This suggests that refusal behaves like a switch (a threshold), which small edits barely move and a large enough edit flips. Consistent with the small local effect that our theory predicts for this edit, none of the single-layer monitors we evaluated detects it (with detection rates remaining at or below 0.07 even at 1.0‖h‖).

Still, the theory still tells us how to detect this attack. Monitors can combine information across layers (Prop. 2), and the whitened read works with any clean covariance. Thus, we predict the steered layer from the preceding layer and apply a rank-16 whitened read to the residual, the part that this prediction does not capture. This cross-layer read detects the steering vector at magnitudes from 0.3‖h‖ up (detection 1.00). Across all 119 fixed steering settings we tried, at most 11% of completions are both successful and undetected, compared with 5% in the absence of any attack. The other models show the same pattern (Table 6 of the revision):

modelno steerdifference-of-means steer: successits share missed by every single-layer monitormost successful and undetected by the cross-layer read, over 119 steerssame, cross-layer read and harmful-request probe
Llama-3.2-1B0.070.611.000.500.15
Llama-3.1-8B0.050.610.990.110.02
OLMo-3-7B0.000.070.730.010.01
Qwen3.5-9B0.010.261.000.010.01
Gemma-4-12B (4-bit)0.010.690.990.030.01

Llama-3.2-1B is the exception because even weak steering vectors are enough to produce harmful completions in such a small model. Beyond these rates, a closed-form detection rate, computed only from clean statistics and the edit, also predicts which edits will be detected by the monitor, with a mean absolute error of 0.03 for the whitened read over every fixed steer and trained attack vector evaluated on five models (Fig. 12).

AuthorsOpenReview ↗

Reply to Reviewer Z99M (2/3)

W1 (continued)

No monitor is safe against an attacker trained to evade it, as the revised paper now states explicitly. The table below gives the rates on Llama-3.1-8B, with 150 held-out requests per setting and the range over the penalty weight for trained attackers:

attacksuccessundetected by every single-layer monitorundetected by the cross-layer readundetected by the cross-layer read and the probe
no attack0.050.050.050.00
difference-of-means steer, 0.6‖h‖0.610.600.000.00
trained against whitened read, Mahalanobis, norm0.57 to 0.680.57 to 0.660.00 to 0.020.00
trained against cross-layer read, Mahalanobis, norm0.63 to 0.730.63 to 0.730.63 to 0.730.01 to 0.02
trained against cross-layer read, Mahalanobis, norm, probe0.47 to 0.740.36 to 0.690.47 to 0.740.47 to 0.74
LoRA trained against whitened read, Mahalanobis, norm0.850.850.850.85

A steering vector jointly trained against the whitened read, Mahalanobis and norm monitors evades every single-layer monitor with 55% to 68% success across three different seeds. However, the cross-layer read detects this steering (0 to 2% successful and undetected). Paired with a probe for harmful requests, the cross-layer read leaves at most 2% of completions successful and undetected across 222 steering settings not trained against both. An attacker jointly trained against both evades both (47 to 74%). Similarly, a LoRA adapter trained to evade the single-layer monitors enables 85% of completions to bypass every monitor we have, as observed by Bailey et al. (2026) for latent-space monitors.

Changed in the paper: contribution 4, retitled "A monitor that detects what single-layer monitors miss" (p. 2); one sentence after Def. 1 (Sec. 3.2); Sec. 5.3, retitled; limitations in Sec. 6; App. O.1 (Figs. 11, 12, 13, 18; Tables 6, 7, 8, 10); examples in App. O.7.

W2. Table 3 is not a matched comparison

Thank you for spotting this, because the caption claimed that Table 3 isolates the choice of subspace when it did not. Hence, to address the comment, we have replaced it with a matched comparison. On all seven models, we built 21 attack families without using any monitor (CAA and RepE vectors for seven behaviours, built as in the authors' code, and logistic-probe directions), and no monitor or probe sees the family it is tested on. The table provides detection AUROC from the lowest to the highest family, at rank 16 and edit size 0.4‖h‖ (Table 12 of the revision):

monitorLlama-3.2-1BLlama-3.1-8BLlama-3.3-70B (4-bit)OLMo-3-7BQwen3.5-9BQwen3.5-27B (4-bit)Gemma-4-12B (4-bit)
activation PCA0.50 to 0.580.50 to 0.540.50 to 0.520.50 to 0.520.50 to 0.560.51 to 0.650.50 to 0.71
Def. 1 Fisher, unwhitened0.53 to 0.710.57 to 0.690.52 to 0.660.52 to 0.650.57 to 0.870.63 to 0.950.61 to 0.87
random 16-dim read0.58 to 0.890.64 to 0.830.58 to 0.760.64 to 0.820.65 to 0.980.77 to 1.000.87 to 1.00
whitened Fisher read0.85 to 0.990.72 to 0.980.65 to 0.870.73 to 0.970.88 to 1.000.98 to 0.990.99 to 1.00
full-rank Mahalanobis0.96 to 0.980.94 to 0.970.81 to 0.830.94 to 0.960.991.001.00
supervised probes from another family (mean)0.530.550.530.530.550.530.55

On all seven models, the whitened read catches its weakest family better than PCA catches its strongest, while probes trained on one family do not transfer to the others. The table also shows why the paper's choice of subspace matters. Without whitening, the Fisher subspace is dominated by directions in which normal activations already vary a lot. Hence, it can do worse than a random read. On the other hand, whitening addresses this problem by picking directions with high Fisher relative to natural variance, as the restated Prop. 1 requires. Thus, the whitened read beats a random read of the same rank on 17 to 21 of the 21 families on six models. On Qwen3.5-27B, the random read is also close to 1.00 on most families (the whitened read wins on 8 of 21). Hence, we make no such claim in this case, and at 70B every detector is weaker at this edit size. Following your suggestion, the monitor-aware attackers of Fig. 11 are now evaluated on five models (Llama-3.2-1B, Llama-3.1-8B, OLMo-3-7B, Qwen3.5-9B, Gemma-4-12B) and are graded on text.

Changed in the paper: Table 3 replaced by the table above (Table 12; the in-family table is kept as Table 13); Fig. 17 (held-out families); Fig. 11; the Table 3 sentence in Sec. 5.3; abstract and contribution 3.

AuthorsOpenReview ↗

Reply to Reviewer Z99M (3/3)

W3. The attack monitor used the greedy token's gradient, not the Def. 1 Fisher

Thank you for catching this, because the two constructions are were slightly different, and the paper now uses the one that Def. 1 defines:

Llama-3.1-8Bsix models
overlap of the two rank-16 subspaces (mean squared cosine of principal angles)0.320.23 to 0.46
same, random subspace0.002 to 0.008
AUROC on the emergent-misalignment fine-tunes, Def. 1 read0.91 to 1.00
same, greedy-token read0.75 to 0.97

Every number in the revision now uses the Def. 1 Fisher, built for each prompt from its 64 most likely next tokens, which agrees with an unbiased sampled estimate at subspace overlap 0.85 to 0.93 over six models. We also restated Proposition 1 for the monitor we actually run: among rank-r monitors that score a state by its Mahalanobis energy inside their subspace, the top-r generalized eigenspace of (F, Sigma) gives the largest expected detection signal for edits with second moment F, and the proof takes four lines.

Changed in the paper: Prop. 1 and its proof (App. B) and the sentence in Sec. 3.7; the attack-monitor description in App. K, with the comparison of the two constructions.

W4. The coverage of "99.7% of the solid angle" is a ratio of strengths

You are right that we never measured an angle and that a weak unseen effect is not the same as a small blind region. The blind region is in fact large, since it contains every direction the monitor does not read, and what is small is the effect that an edit confined to it can have. We therefore removed the solid-angle panel and restated Theorem 3(iv) as a fraction of effect: an edit the read cannot see keeps at most sigma*(r) = sigma_{r+1}/sigma_1 of the effect of the strongest single edit, with a median of 0.046 at r = 16 over 270 model, layer and behaviour cells.

Changed in the paper: Fig. 14 (the submitted Fig. 12: bar panel only, floor recomputed); Theorem 3(iv) and its proof (App. B); the sentence in Sec. 5.1.

We hope that the matched comparison, the Def. 1 monitor and the attack results judged on text address your soundness concerns, and we would be glad to answer any further question.

Thank you for the detailed responses and substantial revisions. I appreciate that the authors addressed all my concerns with additional clarification and experiments.

Overall, I find the revised paper substantially stronger, and my original soundness concerns have largely been addressed. I am raising my rating to 3. Great work!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.