acceptodds
Under review as a conference paper at ICLR 2027

Representation Transitions Reveal Emerging Safety Risks in Multi-Turn LLM Agents

Abstract

Multi-turn attacks on agentic systems can compose individually permissible actions into harmful outcomes, challenging defenses that assess actions or states in isolation. We show that such attacks leave a detectable signature in the agent's internal representations: harmful behavior emerges as an accumulated representation transition across context updates, whose triggering context can be identified from the same signal. We further find that naive aggregation is confounded by benign representation drift, as a contrastive safety direction need not assign zero to benign transitions. We address this by denoising the direction, anchoring benign traffic at zero and removing its leading variation directions, with no runtime cost. These findings motivate DART, a runtime framework that detects and attributes representation shifts and intervenes with targeted reminders. Across six models and two multi-turn benchmarks, DART reduces attack success from to on MT-AgentRisk, catching every attack at a mean false-alarm rate of , and from to on ASEval, at costs in benign non-refusal of and respectively. On MT-AgentRisk, it outperforms ToolShield, the state-of-the-art multi-turn defense, on all six models: under the same protocol, ToolShield reaches only . Denoising is critical: on ASEval, the undenoised monitor catches only – of attacks, while the denoised monitor catches –. The same monitor covers single-turn indirect injection without modification and adds only 0.14–0.56 s overhead per monitored step without requiring an auxiliary model, making it a lightweight complement to computation-heavy speculative defenses.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Safety, Multi-turn Attacks, Representation Engineering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qTamOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates representation transitions in multi-turn LLM agents and finds that harmful behavior is associated with accumulated shifts in internal representations. To address benign representation drift, the authors propose a denoised safety direction and introduce DART, a runtime defense framework that detects emerging risks, attributes them to contributing context, and intervenes through targeted reminders. Experiments across six models and two multi-turn benchmarks show reduced attack success rates with limited utility loss and low runtime overhead.

Critical strengths

1.DART uses the acting agent's internal representations without requiring an auxiliary model or gradient-based training. Its low runtime overhead makes the method practical for integration into existing agent systems.

2.The paper identifies benign drift as a source of error in cumulative detection. The experiments show that removing the benign mean and dominant variation directions substantially improves detection on ASEval. The evaluation also covers six models and different multi-turn attack settings.

Critical weaknesses

1.The detection results do not fully support the claimed emerging safety risks. The cumulative score simplifies to At=(ht+1−h0)⊤uA_t=(h_{t+1}-h_0)^\top u, measuring the displacement between the current and initial representations. Although the current representation contains historical context, it doesn't explicitly model the risks arise from interactions across turns. And importantly, Table 13 in Appendix F.2 shows that 98% of benign imperative injections are flagged. This may suggests that the detection signal merely capture injection. So the reported trajectory-level separation can't establish that DART detects emerging compositional risks.

2.Missing key comparisons and ablations. The paper compares DART with ToolShield and AgentSpec, but lacks direct comparisons with existing trajectory-level safety monitors. Though the denoising component is well studied, the necessity of cumulative scoring is not adequately examined. Comparisons with per-step detection and simple prefix-level representation classifiers are needed to determine whether the proposed accumulation offers an advantage. The effectiveness of attribution for multi-turn decomposition attacks also lacks direct validation.

Questions for authors

1.Given that the cumulative score is equivalent to the displacement between the current and initial representations, what evidence shows that DART captures compositional risks instead of merely the general representation changes?

2.Table 13 shows that DART flags 98% of benign imperative injections. If the threshold is adjusted to reduce these false alarms, how much malicious injection detection performance is retained?

3.Can the authors provide comparisons with trajectory-level monitors and simpler detection baselines?

Reviewer_3NGDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores the detection of signatures within models' internal representations across multi-turn attacks, which can be used to identify harmful behaviour. The paper first empirically evaluates the representation shift of models across multiple context updates, while de-noising them by comparing them to isolate the safety-critical direction. Based on this evaluation, the paper proposes DART, which accumulates the representational transitions, detects when they surpass a threshold, and injects a safety reminder in cases where the threshold is surpassed.

Critical strengths
  1. I like the general writing of the paper, as it is quite intuitive and easy to read. All terminology is clearly defined, and methods are well motivated.
  2. I believe the method is well motivated using the empirical evaluation and impact of de-noising, which quite intuitively motivate DART. The empirical results are quite strong and confirm the utility of using this signature to detect malicious behavior.
Critical weaknesses
  1. The experiments are somewhat restricted, making it difficult to clearly verify the claims of the paper. For one, 4/6 models are from the Qwen3 model family, making it unclear whether DART generalises to a large set of models. Further, the chosen benchmarks are quite easy, which means it is unclear how large the drop in benign utility would be for harder benchmarks.
  2. The false positive rate of 8-10% is quite high, which might negate any safety benefit of the method. While this seems to not have a large impact on the model's utility, the impact of false positives on more complex benchmarks is unknown. A mechanism which prevents false positives, or mitigates the impact of them would be interesting.
Questions for authors
  1. Could you compare the impact of DART on benign utility on a more difficult benchmark?
  2. Did you consider any methods for reducing or mitigating false positives?
Reviewer_xUCzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DART, a runtime defense for multi-turn LLM agents against task decomposition attacks and indirect prompt injections that bypass single-step guardrails. It shows that emerging harmful behavior leaves a cumulative footprint in the agent's internal representation transitions across context updates. As naive accumulation of these transitions is confounded by benign representation drift, DART applies an offline low-rank subspace denoising procedure that anchors the benign mean to zero and projects out dominant benign covariance directions. At runtime, DART not only monitors the cumulative shift but also intervenes by injecting a reminder before the next action rollout.

Critical strengths
  • The identification of benign representation drift as the core confounder to cumulative tracking is insightful.

  • For monitoring, DART reuses the existing forward-pass activations, adding only a few time costs. And for recovery, the reminder does not interrupt the execution, preserving agent autonomy as benign tasks.

  • The scale of models and benchmarks adopted in the evaluation looks quite large.

Critical weaknesses
  • DART requires white-box access to hidden states at intermediate layers, thus not applicable to those agents built upon black-box commercial APIs (which is however the mainstream among all real-world practical agents).

  • In DART, fitting the denoised direction, selecting the optimal layer, and calibrating the threshold all require representative pre-collected benign calibration and red-teaming trajectories, which limits the generalization and swift adaptation to different applications.

  • In experiments, for monitoring (say, detection only), the effectiveness of DART itself looks fine, but the baseline (AgentSpec) is quite weak. There is a lack of comparison with important multi-turn baselines such as SSH and TRACES (both of which are in the References), and SOTA single-step guardrail baselines such as Llama Guard 3 when provided with the combined whole multi-turn context.

  • And for recovery, only relying on a single reminder in DART has been proven to cause inversion results on some models like Qwen3-8B and Llama-3.1-8B, and the ASR after recovery can remain >50%, which is a large gap compared with the detection. Considering more advanced recovery methods like rollback might be more useful.

Questions for authors

See weaknesses pls.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.