acceptodds
Under review as a conference paper at ICLR 2027

Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal States

Abstract

Language model agents can now perform sophisticated sequences of actions via tools and harnesses, which has increased the scope of the damage they can cause. Guard models, however, are mainly built for content moderation and thus are not well-suited to detecting this agentic risk. To address this, we proceed by first conducting a representational analysis, then use the resulting insights to build a solution. In our analysis, we focus on two types of trajectory-level agentic harms: harmful content, which is expressed directly, and unsafe tool use, which depends on whether an action is consistent with the interaction that produced it. We investigate how open-source guard models represent these two types of harm and find that they are linearly readable inside the model, even though guard models predict no better than chance on pairs that differ only in the called tool's schema. The two harm types also follow nearly orthogonal internal directions, and neither reliably serves as a proxy for the other. These results motivate reading trajectory safety directly from internal states. We introduce TACIT, a readout of a frozen backbone's internal states that decodes no tokens. Trained on six trajectory-safety benchmarks, a linear probe raises mean macro-F1 from 62.3 for the strongest open guard to 80.7, and refined readouts reach 86.2. With each benchmark held out of training entirely, the refined readouts still lead the strongest guard (65.7 vs. 61.1). With the same backbone, training data and test split, the frozen readout is on par with full safety fine-tuning, and it improves the fine-tuned model further when applied on top. The probe trains about one millionth as many parameters as full fine-tuning in about a sixth of the time, and TACIT has the lowest latency of the guards we evaluate. Our results establish internal representations as a practical basis for detecting harm in agent trajectories.

Primary area:
interpretability and explainable AI
Keywords:
LLM Safety, Agentic LLM, Internal Representations, Harmfulness Detection, Content Moderation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aK8tOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces TACIT, a guard for agent trajectories that classifies hidden representations from a frozen LLM rather than generating a safe/unsafe label. It distinguishes harmful content from unsafe tool use, showing that both are linearly decodable internally, although standard guard outputs perform near chance on controlled tool-use pairs. A regularized linear probe is trained with binary cross-entropy, while its layer and pooling method are selected through validation. TACIT outperforms released guards across six benchmarks and is competitive with full fine-tuning at substantially lower training and decoding cost.

Critical strengths
  • Important distinction between message-level harmfulness and context-dependent tool-use risk.
  • Controlled paired analysis and unused-tool placebo provide useful evidence.
  • Simple, efficient method requiring few trainable parameters and no decoding.
  • Broad evaluation, including two backbones, six benchmarks, leave-one-benchmark-out transfer, and matched-data fine-tuning.
  • Intermediate-layer selection is empirically validated rather than assumed.
Critical weaknesses
  • The controlled tool-use analysis covers only two schema-degradation categories and may detect schema anomalies rather than general agentic risk.
  • Modified trajectories are not re-executed, so their actual harmful consequences are unclear.
  • TACIT receives benchmark-specific supervision, whereas released guard baselines do not, confounding the main comparison.
  • Layer 22 is dataset/model-specific, and robustness across splits, seeds, and distribution shifts is not established.
  • Evaluation is mainly post-hoc; practical pre-action intervention, false-positive cost, calibration, and adversarial robustness remain untested.
Questions for authors
  1. How stable are the selected layer and performance across random seeds and data splits? Could the selection of layer 22 be dataset-specific?
  2. Can TACIT detect risk before tool execution without blocking trajectories that would end in safe refusals?
  3. What are its recall at low false-positive rates and robustness to adversarial schema or trajectory modifications?
3 comments
AuthorsOpenReview ↗

Authors' Rebuttal

We thank the reviewer for the careful review.

W1 & W2: Beyond schema degradation, on executed trajectories

In Section 3.1, we chose the TraceSafe schema pairs because they provide matched pairs naturally, with every message and the final action byte-identical. However, they cover only how an action relates to its tool. With the same design, we built pairs for the two other relations that define unsafe tool use, from AgentDojo rollouts in which the agent carried out its actions:

  1. Instruction source. The same injected instruction sits either in a tool output, which is unsafe, or in the user's request.
  2. Authorization. The user's request ends by either authorizing the agent to act or asking only to be told what to do, so in the second case the agent acts without authorization.

Every agent action stays the same within each pair. We measure rank accuracy, averaged over the models in each row:

Rank accuracyInstruction sourceAuthorization
Guard output0.680.72
Guard internal0.970.98
LLM backbone internal0.980.99

As controls, the same edits with content unrelated to safety, moving ordinary tool output into the request or ending the request with a sentence on reply length, give only 0.59 and 0.63, so the separation does not come from the edits as such. The internal signal is therefore not limited to schema anomalies.

The schema pairs keep TraceSafe's own labels, which mark the hazard each edit introduces. In the instruction-source pairs, the unsafe member is the exact rollout AgentDojo executed, and AgentDojo's own check confirms the agent carried out the injected task. There, the internal signal holds where the harm happened.

W3: Supervision confound

We fine-tuned the three guards with the same data and recipe as Table 6, and fitted TACIT on each fine-tuned guard's internal states:

Macro-F1 (%)AgentDoG-4BQwen3Guard-4BLlamaGuard3-8B
Fine-tuned guard output82.482.781.4
+ Probe82.683.384.3
+ Ensemble84.184.784.7
+ Multi-layer85.285.885.7

With the same supervision, reading internal states beats decoding a label in all three guards. TACIT's refined readouts on the frozen backbones in Table 1, 85.1 to 86.2, also stay above every fine-tuned guard.

W4 & Q1: Stability across splits and seeds

Figure 5 tests whether the probe depends on a single layer: layers 15 to 24 of Qwen3-4B score within about one point of each other, and layer 22 is the highest.

On the robustness concerns:

  1. Random seeds. The probe is an L2-regularized logistic regression on frozen features, a convex problem with a single solution, so random seeds do not affect it.
  2. Data splits. We redrew the split with three new random seeds. Across the four splits, the probe's macro-F1 has a standard deviation of 1.7 on Qwen3-4B and 0.7 on Llama-3.1-8B, and AgentDoG has 1.6 on the same test rows, so redrawing the split moves TACIT about as much as it moves the guard model.
  3. Distribution shift. With each benchmark held out (Table 2), TACIT's refined readouts still lead the guards in the mean.

W5 & Q2: Detection before the final action

Appendix D discusses this setting. The backbone is causal, so TACIT can score every prefix of a trajectory and run while the agent acts. We trained and tested the probe on trajectories cut just before the agent's final action, where a monitor would decide whether it may run. Macro-F1 reaches 76.8 on Qwen3-4B and 78.8 on Llama-3.1-8B, and the false-positive rate rises by only about four points over complete trajectories. This points to TACIT's potential for streaming detection.

Full streaming detection also needs step-level labels, which the six benchmarks do not provide. As Appendix D notes, we leave it to future work.

W5 & Q3: False positives, calibration and robustness

We measured recall at low false-positive rates and calibration error on the pooled test splits:

Recall at 1% FPRRecall at 5% FPRECE
Qwen3Guard-4B3.716.60.41
LlamaGuard3-8B3.916.10.25
AgentDoG-4B1.78.60.38
TACIT Multi-layer, Qwen3-4B39.356.70.05
TACIT Multi-layer, Llama-3.1-8B41.664.70.04

At a 1% false-positive rate, TACIT recalls about ten times as many unsafe trajectories as any guard, and its calibration error is a fifth or less of every guard's. We leave adversarial robustness to future work, as Section 5.3 discusses.

We thank the reviewer again for the supportive and constructive review, which has made the paper stronger. The revised manuscript includes all of the results and changes above, and we would be glad to answer any further questions.

AuthorsOpenReview ↗

Authors Comment

Dear Reviewer aK8t,

Thank you for your review. We would appreciate hearing whether our response addresses your concerns, and are happy to clarify anything further.

Best regards,

The Authors

Thank the authors for their responses and I keep my positive score.

Reviewer_D9NTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper primarily addresses unsafe tool-calling behaviors in LLM agents, and sets out to provide a solution leveraging LLM backbones used for a number of existing harmful-input detection guardrails models (mainly Qwen3Guard and LlamaGuard3). The central argument of the paper is that harmful input detection guardrail models fall short on this particular task. The authors back this up by providing two forms of evidence: harmful content that is expressed in a user input or action vs. unsafe tool use, defined as an action by the agent that is inconsistent with the provenance or the tool interface that produced it. It first conducts an analysis using matched trajectory pairs that isolate one form of harmfulness at a time, and finds that, even though guardrail models' own output scores rank harmful-content pairs well above chance, they rank unsafe tool-use pairs at or below chance. Additionally, a contrastive mean-difference direction read from intermediate layers ranks unsafe-tool-use pairs to much higher values for both guardrail and backbone LLMs. They further argue that the two tasks occupy near-orthogonal internal directions, and a direction fit for one leads to poor performance for the other.

They then proceed to introduce TACIT (Trajectory Assessment by Classifying Internal Transformer states) that center on a readout from internal layer(s) of the LLM backbone, by either pooling last-token and/or mean hidden states and mapping them to a safety score via an L2-regularized logistic classification. TACIT requires no backbone parameter updates and no token decoding. Evaluations on six trajectory-safety benchmarks show significant improvements in Macro-F1 scores compared to three guard models. Furthermore, leave-one-benchmark-out also achieves enhanced performance in macro F1-score (for the multi-layer solution). They also ablate their results by performing a full-SFT comparison using the same training data on Qwen3-4B, showing that even though the frozen probe underperforms SFT by ~3-4 points on the two LLM backbones, they have a significant edge in terms of latency and training time, and applying TACIT to the SFT'd backbone adds further gains.

Critical strengths
  1. The authors provided decent representational analyses between the two tasks by isolating each risk, and investigating the efficacy of each direction on the other one. They established that unsafe tool-use is underperformed at guardrail outputs compared to internal representations from the backbone LLMs used to train the guard models, and that this is along a near-orthogonal direction to harmful content. This is a genuine mechanistic explanation for why guardrail models fall short on this task, which directly motivates and justifies the solutions presented by the authors.

  2. The full SFT ablations on the same backbone LLMs is a fair control to highlight the efficacy of TACIT and the fact that keeping every other factor constant, the linear Probe is roughly on par with SFT. Also, stacking Multi-Layer TACIT on top of SFT introduces further improvements at a fraction of the training time. This poses as a decent contribution.

Critical weaknesses
  1. The main weakness to be highlighted is that the authors lead with a headline that slightly oversells the improvements of their approach over guardrail models perhaps not fully finetuned on particular sets of agent trajectories. Both the abstract and Figure 1C lead with 86.2% vs. 62.3% over the best-performing guardrail model. However, that comparison actually doesn't isolated the training data, i.e. the trained set comes from the benchmarks evaluated on where these metrics come from, rather than showing significant generalization, while the guardrail models are run zero-shot on the 20% subset of the data from these benchmarks. The authors, however, do share the fairer comparison in Table 2 for leave-one-out evaluations (65.7 multi-layer vs. 61.1 AgentDoG-4B), as well as the matched-data SFT in Table 6 (80.7 for frozen probe vs. 84.2 SFT). These show the true yet smaller margins over a comparably-trained system and that perhaps the larger advantage was mainly driven by the in-domain overfitting effect. The headlines may confuse readers about the magnitude and the source of contributions by TACIT.

  2. The other slight weakness is that the methodological novelty might be incremental/limited. The frozen internal state recipes, including last-token/mean-pooling and the linear probe are mainly borrowed from Alain & Bengio (2016), Tigges et al. (2024), Jiao et al. (2024), and the multi-layer aggregation from salient dimensions across multiple layers particularly from Jiao et al. (2026) which is the best-performing TACIT approach. While the paper's contributions are focused on the analysis of the two forms of the risks and the extension from single messages to trajectories.

Questions for authors
  1. Table 1 shows results from training then testing TACIT on 80/20 splits of the same six benchmark datasets, rather than on official train-test partitions from the benchmarks or on a different set of benchmarks to incorporate a more unbiased training set. Given that, and that Table 6 shows the frozen readout is roughly on par with full SFT, and Table 2 where train and test benchmarks are disjoint, shows a much smaller margin, do you agree the Table 1 gap over opensourced guardrails is predominantly driven by an in-domain fit rather than evidence that internal readouts generalize better than guardrail outputs across the board? Would you be willing to reframe the abstract/Figure 1C around the matched-SFT and leave-one-out benchmark numbers, and state explicitly in the main text (not only in Appendix B.3) that Table 1's splits are self-constructed within-benchmark splits rather than cross-benchmark or benchmark-native splits?

  2. AgentDojo trajectories are generated by the same Qwen3-4B and Llama-3.1-8B backbones that TACIT reads (with labels assigned programmatically). Yet TACIT's AgentDojo performance when trajectories are generated by a different agent (as in OpenAgentSafety) are missing. This would further bolster that the readout does not benefit from reading its own backbone's generations.

  3. In Table 2, TACIT trails AgentDoG on ATBench (58.7 vs 61.3) and almost ties with LlamaGuard on TraceSafe (48.3 vs 48.2). Could you characterize where internal readout does/does not transfer, beyond the ATBench synthesis-pipeline overlap mentioned?

  4. It would be very interesting to see how sensitive the orthogonality hypothesis holds on the borderline hazard-family assignments in Table 3.

  5. A point of confusion when reading the main body was that Table 1 and Table 2 report different values for the identical guardrail baseline on the same benchmarks. Upon more inspection, Appendix B.5 seems to imply that this is because Table 2 scores baselines on 100% of the held-out benchmark's trajectories while Table 1 uses only the 20% test split while using the other 80% for training. Could you confirm this is the only reason, and if so and even better, add an explicit note next to both tables so this doesn't lead to confusion and become a source of inconsistency?

6 comments
AuthorsOpenReview ↗

Authors' Rebuttal

We thank the reviewer for the careful review.

W1 & Q1: The headline comparison and the splits

TACIT still leads the guards when in-domain supervision is removed or matched. We agree that the gap in Table 1 includes the effect of in-domain supervision. First, with each benchmark held out (Table 2), TACIT's refined readouts lead the strongest open guard, 65.7 against 61.1. Second, with the same backbone, training data and test split (Table 6), the frozen refined readouts are on par with full SFT and improve on it when applied on top.

Third, we fine-tuned the three guards with the same training pool, recipe, folds and test split, and fitted TACIT on their internal states. With the same supervision, reading internal states still beats decoding a label in all three fine-tuned guards:

Macro-F1 (%)AgentDoG-4BQwen3Guard-4BLlamaGuard3-8B
Fine-tuned guard output82.482.781.4
+ Probe82.683.384.3
+ Ensemble84.184.784.7
+ Multi-layer85.285.885.7

The Multi-layer readout is higher in each of the 12 fine-tuned models, and the frozen refined readouts of Table 1, 85.1 to 86.2, stay above every fine-tuned guard.

The revised abstract and Figure 1C lead with these results, and Section 4.2 now states that the 80/20 splits are our own within each benchmark.

W2: Methodological novelty

The novelty is a finding about why guard models fail on agents, and a guard built on that finding. Our analysis finds that harmful content and unsafe tool use both remain linearly readable inside the model, while unsafe tool use is weakened at the guard's own output, and that the two forms follow different internal directions. These results motivate reading trajectory safety directly from internal states, so we build TACIT as an internal readout, and it outperforms every released guard. TACIT in turn shows that safety evidence can be read out more accurately from intermediate layers than from the final layer, and that its score uses largely distinct coordinates for the two forms (Section 5). We agree that TACIT builds on established probing tools, which Section 4.1 cites.

Q2: AgentDojo trajectories from other agents

The readout does not depend on reading its own backbone's generations. We generated about 1,000 new AgentDojo trajectories each with two agents that are not TACIT backbones, Qwen3-32B and Qwen3-30B-A3B-Instruct, using the suites, attack and labels of Appendix B.1. TACIT uses the Table 1 probe configuration fitted on our training pool, with no selection on the new data. On both agents, TACIT leads every guard by 11 points or more:

Macro-F1 (%)Qwen3-32B agentQwen3-30B-A3B agent
Qwen3Guard-4B42.742.6
LlamaGuard3-8B55.154.2
AgentDoG-4B52.754.2
TACIT Probe, Qwen3-4B67.272.7
TACIT Probe, Llama-3.1-8B66.174.7

Q3: Where the internal readout transfers

Beyond the ATBench overlap, we broke down the held-out TraceSafe column of Table 2 by attack category. On Qwen3-4B, where the Ensemble and Multi-layer readouts both score 48.3, the Ensemble readout leads every guard on each of the seven categories in which the called tool's interface decides safety, such as HallucinatedTool and DescriptionMismatch. It trails LlamaGuard3 on the three leakage categories, where the leaked record or key appears in the action itself. Internal readouts therefore transfer best when the called tool's interface decides safety, while these categories favor a content guard, since the leaked data is spelled out in the text.

Q4: Sensitivity to borderline hazard families

The result does not hinge on the borderline assignments. The borderline assignments in Table 3 are the two misinformation families of ATBench. To test their effect, we fitted a readout for each risk within ATBench under three assignments and measured the AUROC with which it ranks its own risk above the other. If the two risks shared one direction, this would be 0.5. Averaged over the two readouts, the two risks stay separable in every case, and most strongly with the clearest families:

AUROC, own risk over the otherAgentDoG-4BQwen3Guard-4BLlamaGuard3-8B
Assignment of Table 30.710.710.70
Without the two misinformation families0.720.710.72
Injections vs. jailbreaks and harmful artifacts0.730.730.74

Q5: Guard numbers in Table 1 and Table 2

Yes, this is the only reason. Table 1 scores every system on the 20% test split, whereas Table 2 scores it on all trajectories of the held-out benchmark. We have added this note to both captions.

We thank the reviewer again for the constructive review. The revised manuscript includes all of the results and changes above, and we would be glad to answer any further questions.

AuthorsOpenReview ↗

Authors Comment

Dear Reviewer D9NT,

Thank you for your review. We would appreciate hearing whether our response addresses your concerns, and are happy to clarify anything further.

Best regards,

The Authors

Thank you for the response and the revision of the manuscript. The reframed abstract, disclosed splits, and fine-tuned-guard and cross-agent results partly address my concerns, but some gaps still remain. Once models are fine-tuned on the same data, their outputs reach 81.4–85.8 (Table 3), close to the frozen refined readouts of Table 1 (85.1–86.2), and readouts on the fine-tuned models change their scores by only −0.4 to +4.3. Without paired confidence intervals reported for Table 3, it is unclear which of these differences are beyond noise level, given the split standard deviation of 1.7 (Probe, Qwen3-4B) reported in Appendix E of the updated version. Since the plain probe barely improves on (or falls below) the fine-tuned outputs in four of the five models, the "as simple as a linear probe" framing should be reframed.

Also, Table 2 still compares against zero-shot guards, while a leave-one-benchmark-out SFT baseline is additionally needed to show the ability of internal readouts to generalize better than fine-tuning. The SFT baselines in Table 3 seem to be using a single unswept recipe while the readouts are selected by validation search, and the fine-tuned guards score below the fine-tuned base models (e.g., LlamaGuard3 81.4 vs. Llama-3.1-8B 85.8), with readout gains the largest where the fine-tuned model is the weakest. This suggests the fine-tuned baselines may be under-tuned. I suggest conducting a small learning-rate/epoch sweep, and fine-tuning each guard with its native prompt format, which would make the comparison more convincing.

The per-category TraceSafe breakdown added at the end of the "TACIT transfers to benchmarks absent from training" paragraph (Section 4.3.1) is given only in prose and covers 10 of the 12 attack categories in Table 4, omitting PromptInjectionIn and PromptInjectionOut. Please report per-category numbers for all twelve, since the readout leads on seven but only ties LlamaGuard3 overall in Table 2 (48.3 vs. 48.2, well within noise).

Moreover, Table 14 in Appendix E again compares an in-domain-trained readout against released, uncalibrated guards. Please compare against the fine-tuned or calibrated guards, with confidence intervals.

Finally, the introduction and the opening of Section 4.3.1 still lead with the in-domain vs. zero-shot gaps. Please address those as well.

Reviewer_ZKs8OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at why open guard models struggle on agent trajectories. It separates two kinds of risk: harmful content, which you can see in the text, and unsafe tool use, which only shows up in how an action relates to its tool, its authorization, or the source of its instruction. Using matched pairs, the authors show that guard outputs miss the tool-use distinction even though it’s linearly readable from intermediate layers, and that the two kinds of risk live in roughly orthogonal directions. They then turn this into TACIT, a probe on a frozen backbone’s hidden states, which beats existing open guards, holds up reasonably when benchmarks are held out, and is much cheaper than fine-tuning.

Critical strengths

I like the framing. Splitting “harmful content” from “unsafe tool use” is simple, and it explains well why content-moderation guards don’t carry over to agents. The controlled analysis is also more careful than most probing papers. The placebo edit on an unused tool is a nice touch, as is checking the direction similarity against a split-half baseline. The paper is clearly written, and the case study is honest about what the method can’t yet do.

Critical weaknesses
  1. The method itself isn’t new. Probing frozen hidden states for safety, including the multi-layer aggregation, comes pretty directly from Jiao et al. So the real contribution is the analysis and the move to trajectories. That’s fine, but it means the analysis has to be convincing on its own, and I have some doubts about it (below).
  2. The main comparison doesn’t isolate the paper’s actual claim. TACIT is trained on the same benchmarks it’s tested on, while the guards are used off the shelf. So the big gap mostly tells me that in-domain supervision helps, not that internal states are a better interface than outputs. The question the paper sets up is “given the same supervision, is reading internal states better than decoding a label?” Answering it would mean fine-tuning the guard checkpoints on the same data. The held-out-benchmark experiment is closer to the right test, and there the advantage gets much smaller.
  3. The tool-use pairs don’t really match the risk the paper motivates. The intro is about authorization and provenance: the agent pays when it was only asked to review, or it follows an injected instruction. But the controlled pairs are schema degradations like missing type hints or renamed parameters. Those are closer to interface quality issues than safety violations, so it isn’t surprising that guards ignore them. Also, a supervised mean-difference direction on byte-matched pairs will pick up almost any systematic edit. The placebo shows the signal is tied to the called tool, but that only means the model notices “this tool’s schema got worse,” not that it internally judges the action unsafe. Pairs where only the authorization or the instruction source changes would test the paper’s actual claim much more directly.
  4. I’m not fully convinced by the orthogonality result. The two directions come from different datasets, and the content pairs mostly differ in whether the agent refuses or complies. So the “harmful content” direction may really be a refusal direction. Two directions estimated from unrelated contrasts in a high-dimensional space will tend to be near-orthogonal anyway. The ATBench control is the right idea, but projecting out the shared unsafe-vs-benign direction can push each residual direction away from the other group by construction. That would make the other risk look worse than chance for mechanical reasons. I’d like to see this control done without the projection.
  5. The analysis and the method feel only loosely connected. If tool-use risk is really present inside the model and just lost at the output, a probe trained on other benchmarks should pick it up. But tool-use detection is exactly where the held-out results are weakest. Likewise, the gain on top of SFT only appears with the ensemble and multi-layer readouts, which makes me wonder whether that gain comes from ensembling rather than from internal states. Right now the analysis motivates the method, but the experiments don’t show the method works for the reasons the analysis gives.
  6. “From within” overstates what the method does. TACIT re-encodes trajectory text with its own backbone, and for most benchmarks the trajectory came from a different agent. So it’s a guard that uses hidden-state features, not a monitor of the acting agent’s internals. The more interesting setting is reading the agent’s own activations as it acts, which would be nearly free and closer to the title. It also opens the question of robustness: a single-layer linear probe seems like an easy target for an attacker who can shape injected text, and the paper doesn’t discuss this.
Questions for authors

Could you build matched pairs that isolate authorization or instruction source, and check whether the internal signal holds there? How would a guard fine-tuned on your training pool compare, so that supervision is held constant? Have you tried probing the acting agent itself during generation?

3 comments
AuthorsOpenReview ↗

Authors' Rebuttal (Part 1)

We thank the reviewer for the careful review.

W1: Novelty of the method

The novelty is a finding about why guard models fail on agents, and a guard built on that finding. Our analysis finds that harmful content and unsafe tool use both remain linearly readable inside the model, while unsafe tool use is weakened at the guard's own output, and that the two forms follow different internal directions. These results motivate reading trajectory safety directly from internal states, so we build TACIT as an internal readout, and it outperforms every released guard. TACIT in turn shows that safety evidence can be read out more accurately from intermediate layers than from the final layer, and that its score uses largely distinct coordinates for the two forms (Section 5). We agree that TACIT builds on established probing tools, which Section 4.1 cites.

W2 & Q2: Holding supervision constant

With the same supervision, reading internal states beats decoding a label in all three fine-tuned guards. Comparing internal readout with a decoded label under the same supervision is one of the main lenses of our experiments. Section 4.3 and Table 6 make this comparison on the two backbones, against full SFT on the same training pool, folds and test split. There, TACIT's refined readouts on the frozen backbone are on par with full SFT, and applied to the fine-tuned model, they improve on SFT alone.

Following the reviewer's suggestion, we have now run the same protocol on the three guard checkpoints. We fine-tuned each with the recipe of Table 6 on the same four folds, scored it through its own output on the same test split, and fitted the TACIT readouts on its internal states, so supervision is identical within each column:

Macro-F1 (%)AgentDoG-4BQwen3Guard-4BLlamaGuard3-8B
Fine-tuned guard output82.482.781.4
+ Probe82.683.384.3
+ Ensemble84.184.784.7
+ Multi-layer85.285.885.7

Fine-tuning lifts the guards to 81.4 to 82.7, yet the Multi-layer readout gains 2.8 to 4.3 points over the guard's own output and is higher in each of the 12 fine-tuned models. Moreover, without any fine-tuning, TACIT's refined readouts on the frozen backbones in Table 1, 85.1 to 86.2, stay above every fine-tuned guard. The revision moves the matched-data comparison into Section 4.3 as the new Table 3, with these results added.

W3 & Q1: Pairs that isolate authorization and instruction source

When only the authorization or the instruction source changes, the internal states of the guards and their backbones still separate the pairs far better than the guards' outputs do.

Following the reviewer's suggestion, we built pairs with the design of Section 3.1 for the two other relations that define unsafe tool use, from AgentDojo trajectories in which the agent carried out its actions. Every agent action stays the same within a pair:

  1. Instruction source. The same injected instruction sits either in a tool output, which is unsafe, or in the user's request.
  2. Authorization. The user's request ends by either authorizing the agent to act or asking only to be told what to do, so in the second case the agent acts without authorization.
Rank accuracy, mean over the models in each rowInstruction sourceAuthorization
Guard output0.680.72
Guard internal0.970.98
LLM backbone internal0.980.99

As controls, we applied the same edits with content that does not concern safety, moving ordinary tool output into the user's request or ending the request with a sentence about the length of the reply. These reach only 0.59 and 0.63 along the same directions, so the separation does not come from the edits as such.

These pairs are the violations the introduction motivates, an injected instruction and an action the user did not authorize. Unsafe tool use is therefore weakened at the guards' outputs but kept in their internal states for all three relations, as Section 3.1 finds for the schema pairs. We thank the reviewer for this suggestion and have added these pairs to Section 3.1.

AuthorsOpenReview ↗

Authors' Rebuttal (Part 2)

W4: Do the two risks follow different directions?

Within one benchmark, the two risks remain separable without the projection and without refusal. We thank the reviewer for suggesting this control. Appendix A.4 compares the two risks within a single benchmark, ATBench, and we reran it without the projection. To rule out refusal as well, we added a version of the harmful-content readout trained only against benign trajectories in which the agent does not refuse. Each readout scores the test trajectories of both risks, and we report the AUROC with which it ranks its own risk above the other. If the two risks shared one direction, the two readouts would rank them in opposite orders, so their AUROCs would sum to one and at most one could exceed 0.5.

AUROC, own risk over the otherAgentDoG-4BQwen3Guard-4BLlamaGuard3-8B
Unsafe-tool-use readout0.680.660.70
Harmful-content readout0.740.770.70
Harmful-content readout, no refusal in training0.760.760.72

Without the projection, every readout still ranks its own risk above the other, so each risk has a direction of its own. Trained without refusals, the harmful-content readout does just as well, which shows that it reads harmful content rather than refusal. Since the test compares readouts, not cosines, it does not rely on near-orthogonality in high dimensions. We believe this control strengthens the evidence of Section 3.2 that the two risks follow different internal directions, and we have added it to Appendix A.4.

W5: Does the method work for the reasons the analysis gives?

Yes. First, TACIT reads the two risks from separate parts of the internal state, as the analysis finds. TACIT is trained only on binary safety labels, and no label says which of the two risks a trajectory carries. Yet on matched pairs from Section 3 that it never saw, half of the score gap on the tool-use pairs comes from 23 of the 2,560 dimensions, half of the gap on the harmful-content pairs comes from 16, and each set adds little to the other gap (Section 5.2). Detection is also best at an intermediate layer, while a guard decodes its label from the last layer (Section 5.1).

Second, trained on other benchmarks, TACIT picks up unsafe actions. Held out, the refined readouts lead clearly on OpenAgentSafety, where five other agents act in real tool environments and the label records whether an agent took an unsafe action. Trained only on the other five benchmarks, they lead the best guard there by 6.5 to 11.2 points on both backbones (Table 2), more than on any other held-out benchmark.

Third, ensembling outside the model is not enough, and the gain on top of SFT depends on reading the internal states. To test whether the gain comes from ensembling, we also ensembled at the output. Each model in W2 has four fine-tuned checkpoints, one per fold, and we averaged their outputs. Across the two backbones and three guards, this output ensemble gains 0.6 points over a single checkpoint on average, while still scoring 1.4 points lower than the Ensemble readout on a single checkpoint.

W6 & Q3: "From within" and the acting agent

"From within" describes how TACIT decides: it reads its own internal states instead of decoding a label. TACIT is a guard model, used the same way as every guard we compare with: the agent runs unchanged, and a separate model checks its trajectory and flags the unsafe ones. A guard therefore reads trajectories from another agent, and so does every baseline in the paper.

Reading the acting agent's own activations is a different setting. The agent we protect is often a frontier model whose internal states we cannot access. Even when we can, a deployed agent may have hundreds of billions of parameters, so extracting its states to train a readout is expensive and has to be redone for every agent. A guard avoids both problems: one small model protects any agent, open or closed.

We leave adversarial robustness to future work, as Section 5.3 notes: "a trajectory offers an attacker more surface than text messages, through injected tool results, long context, and the formatting of tool calls, making red-teaming agent trajectories, and the defences that answer it, a direction of its own."

We thank the reviewer again for the detailed and constructive review, and in particular for the suggestion to hold supervision constant. Moving the matched-data comparison into the main text with the fine-tuned guards and the output-level ensemble, and discussing it explicitly, makes the paper's central claim clearer. The revised manuscript includes all of the results and changes above, and we would be glad to address any further questions or suggestions.

AuthorsOpenReview ↗

Authors Comment

Dear Reviewer ZKs8,

Thank you for your review. We would appreciate hearing whether our response addresses your concerns, and are happy to clarify anything further.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttals and Paper Revisions

We thank the three reviewers for their careful reviews. We have uploaded a revised PDF with the revisions marked in blue.

New results in the responses

  1. TACIT still leads the guards when in-domain supervision is removed or matched (all reviewers). With each benchmark held out of training, its refined readouts lead the strongest open guard. With the three guards fine-tuned on our training pool, reading their internal states still beats decoding their labels, and averaging the outputs of four fine-tuned checkpoints scores lower than reading one. The gain on top of SFT therefore depends on reading the internal states.
  2. Unsafe tool use is weakened at the output for all three relations that define it (ZKs8, aK8t). When only the instruction source or the authorization changes, the internal states separate the pairs far better than the guards' outputs, while edits unrelated to safety separate them far less. These are the violations the introduction motivates, so the finding is not limited to schema anomalies.
  3. Harmful content and unsafe tool use follow different internal directions without the projection and without refusal (ZKs8, D9NT). Each readout still ranks its own risk above the other, and the two stay separable under three assignments of the borderline hazard families.
  4. TACIT does not depend on reading its own backbone's generations (D9NT). On AgentDojo trajectories from two agents that are not TACIT backbones, it leads every guard by 11 points or more, so it can guard other agents.
  5. TACIT holds at strict operating points and keeps most of its accuracy before the final action (aK8t). At a 1% false-positive rate, it recalls about ten times as many unsafe trajectories as any guard.

Changes to the paper

  • The abstract and Figure 1C lead with the matched comparison and the held-out results.
  • Section 3.1 and Appendix A.6 add the instruction-source and authorization pairs, and Appendix A.4 the controls for the two forms of risk.
  • Section 4.2 states that the test split is our own, and the captions of Tables 1 and 2 explain why the guard numbers differ.
  • Section 4.3 adds the fine-tuned guards in the new Table 3, the output-level ensemble, the results at strict operating points, the held-out TraceSafe breakdown and the new agents.
  • Section 5 adds the layer and split analysis and the detection before the final action, and the new Appendix E collects the robustness and deployment analyses.

Our responses cite tables by their numbers in the original submission. In the revision, a new Table 3 holds the matched-data comparison, so former Table 3 is now Table 4 and former Table 6 is now Table 9.

We thank the reviewers again and would be glad to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.