acceptodds
Withdrawn from ICLR 2027

ADTGuard: Adaptive Depth-Time Safeguard for Streaming Inference in Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large Language Models (LLMs) can generate fluent responses to diverse user requests, yet streaming may expose users to harmful content before generation is complete. Existing guardrails face two limitations in this setting. First, post-hoc moderation at response or chunk boundaries delays intervention, risking harmful exposure when content is released before verification. Second, even with token-level detection, coarse response-level supervision does not specify when harmful content begins, while fixed-layer readouts may miss relevant safety signals at other depths. To address these limitations, we introduce ADTGuard, which checks each token before release, progressively refines token-level training targets, and adapts layer selection to the current prefix. ADTGuard models risk across depth and time through Depth-Time Safety Dynamics (DTSD), which integrates safety evidence across layers at each token and uses it to guide risk accumulation over successive tokens. To adapt the choice of observation layers to the current prefix, we propose the Causal Depth Router (CDR). With only response-level labels available, we further develop Risk-Onset Target Refinement (RTR) to progressively construct token-level training targets for both components. Through extensive experiments across multiple safety benchmarks, we demonstrate that ADTGuard consistently achieves strong streaming detection performance, matching or surpassing state-of-the-art safeguards while requiring only 16M trainable parameters and maintaining competitive per-token latency.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Safeguard, Adaptive, Streaming Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vKpeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to address two core issues: first, feature representation-based methods read in a fixed number of layers, but the safety signals are not fixed within those layers; second, how to refine the response-level supervision signals to the token level as training progresses. The authors use CDR to implement dynamic layer selection, DSD and TSD to implement dynamic context selection, and RTR to refine labels as training progresses.

Critical strengths

I believe RTR has been very insightful to me. In the early stages of model training, it relies on response-level supervision signals to shape potential safety judgments, and then gradually reduces its dependence on these signals, which aligns well with the intuition behind incremental learning.

Critical weaknesses
  • Why does CDR use the current layer selection method? I understand the rationale behind dynamic layer selection, but since CDR currently scores each layer based on prefix representations, I’m curious why it doesn’t simply use the current hidden states of each layer (or perform another MLP mapping) for attention, followed by weighted fusion or Top-K selection (or some other similar, simpler approach)?

  • It makes sense to consider both model depth and generation history. However, the implementation involves two state-space branches, with the depth summary also adjusting the update step size of the time branch. I’d like to understand which of these design elements is most critical: the sequential processing of cross-layer information, cross-token memory, or the coupling between the two? For example, how much worse would the performance be if we simply appended a temporal module to a multi-layer fusion network, or fed the deep summary into the temporal branch via concatenation or gating? Such comparisons might better illustrate why the current architecture is worth adopting.

  • Based on the ablation results, the flow metrics improved significantly after incorporating RTR, so I’m curious: if we apply the same target refinement method to a simpler detector, how much improvement can we achieve? Additionally, since RTR uses the risk changes from the previous prediction cycle to guide the next training cycle, if the previous cycle incorrectly identified the location of the risk, is it possible that subsequent training could reinforce this bias? Would using a small number of manually annotated prefixes or risk origins help verify whether the refined targets are closer to reality?

  • Streaming F1 counts a detection as successful as long as it is triggered at any point in the response. However, there is a significant difference in effectiveness between intercepting harmful content as soon as it appears and intercepting it only after a substantial amount has already been output. Triggering the system prematurely at the start of a normal response may also negatively impact the user experience. Therefore, I would like to know if tests have been conducted to measure detection latency, the amount of harmful content released before interception, and false positives on normal content prefixes. These results can also help determine whether RTR has improved the identification of where risks occur.

  • Judging by the results, the performance of this paper is close to that of Qwen3Guard-Stream. If Qwen3Guard-Stream is already effective enough, why is there a need for a method like the one described in this paper, which is so meticulously designed and, to me, extremely complex?

Questions for authors

See the weaknesses above.

Reviewer_qyBPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ADTGuard performs token-by-token safety detection over a frozen LLM. The Causal Depth Router (CDR) selects a Top-K subset of layers conditioned on the prompt and response prefix. Depth-Time Safety Dynamics (DTSD) uses separate state-space branches to aggregate depth-wise evidence and track temporal risk, with the depth output modulating the temporal update. Risk-Onset Target Refinement (RTR) constructs token-level soft targets from positive increments in the preceding epoch's cached risk trajectory, progressively replacing broadcast response labels. The evaluation covers three backbones and five safety datasets.

Critical strengths

1 Practical motivation Reusing a frozen generator's hidden states avoids invoking an additional large moderation model for every prefix. Combining evidence across model depth with a temporal risk state is also reasonably motivated.

2 Broad classification gains and an empirical benefit from RTR In Table 1, ADTGuard achieves the highest Streaming F1 in 14 of 15 backbone-dataset settings. In Table 3, adding RTR increases Streaming F1 from 87.51 to 93.58. These results support the practical value of addressing the mismatch between response-level labels and prefix-level supervision

Critical weaknesses
  1. Comparison robustness The comparisons differ in training budget, context length, and check frequency: ADTGuard is trained for five epochs versus one for Kelp and the dedicated guards, SIREN uses a 512-token limit versus 4,096 for ADTGuard, and dedicated guards check every ten tokens on SGB versus every token for ADTGuard. Their effects remain unresolved. Moreover, three gains over Qwen3Guard-Stream are below one percentage point, while Public WildGuard performance is 4.42 points lower.

  2. Unclear independent benefit of adaptive layer selection Table 3 shows that adding CDR changes Streaming F1 only from 86.05 to 86.09, with larger gains appearing after DTSD and RTR are introduced. This does not isolate the benefit of adaptive routing. Since component interactions may matter, the key missing control is a fixed Top-K readout with DTSD, RTR, and the training setup retained.

  3. Limited conceptual grounding of risk-onset refinement RTR treats positive increments in the model’s historical risk scores as evidence of harmful onset, but the conceptual link between these quantities is insufficiently justified. Such increments could reflect model uncertainty rather than the emergence of harmful content. The KL characterization establishes the interpolation rule for a target profile; it does not establish that the profile captures meaningful information rather than reinforcing the model’s existing biases.

Questions for authors
  1. Do the reported advantages persist with trained baselines, comparable context coverage, and matched check frequencies? Please also provide uncertainty estimates for the closest comparisons.

  2. With DTSD retained, how does CDR compare with a fixed Top-K layer subset selected using only training or validation data under the same setup?

  3. What assumption links positive risk-score increments to actual harmful onset? Can prefix annotations verify that RTR improves temporal localization beyond changing the detector’s calibration?

Reviewer_kbM5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes ADTGuard, a small add-on module that sits on a frozen LLM and checks each response prefix for harmful content before the token is shown to the user. It picks a few layers to read at each token, combines them with two Mamba-style scans (one across layers, one across tokens), and trains on token-level targets that it builds from its own risk curves from the previous epoch, since the datasets only label whole responses. On five safety benchmarks and three base models, it beats the probe-based baselines SIREN and Kelp and is roughly on par with fine-tuned guard models like Qwen3Guard-Stream, at about 2.3 ms of extra latency per token.

Critical strengths

The refined targets cut false alarms on safe responses from 137 to around 38, which shows a real flaw in copying the response label onto every token and offers a cheap fix. The gains over other probe methods are consistent across the base model and datasets, and the experiments are documented carefully, with latency confidence intervals and released code.

Critical weaknesses

The method stacks several known techniques, and the paper rarely says why each one is there or tests whether it’s needed, since the ablation only adds components one at a time. Some choices even work against the paper’s own argument. For example, the prompt summary uses a uniform average over all layers, exactly the kind of fixed readout the router is supposed to replace. The one formal result for the refined targets just restates that cross-entropy is linear in its target. It doesn’t argue that the targets move toward where the harm actually starts rather than reinforcing the model’s early guesses.

Against the fine-tuned guards, ADTGuard wins Response F1 on only one of five datasets, and most of its Streaming F1 wins are under 1pp, even though those baselines got one training epoch to its five. The adaptive routing in the title adds 0.04 Streaming F1 in the ablation. The paper is motivated by catching harm early, but Streaming F1 gives full credit for an alarm on the last token. Each detector is also trained and tested on a single dataset, everything is from one seed. So it’s hard to say how well this holds up in practice.

Questions for authors

Why is the prompt summary s0s_0 a uniform average over all L layers, when the paper’s main argument is that uniform readouts across depth are inadequate? How does it compare with the final prompt token’s representation, a single mid-depth layer, or running the router over the prompt as well?

Why does the router use only embedding-layer features of the response? Did you try routing from a deeper layer, and how much does the cost actually change?

What does the depth SSM gain over a learned weighted sum or a small attention layer over the K selected layers? Separately, what happens if you remove the γ(dt)\gamma(d_t) coupling and simply concatenate the depth summary with the temporal input?

Can you show the effect of each component on its own rather than only cumulatively? In particular: DTSD with a fixed set of layers, DTSD with all layers and learned static weights, and the MLP baseline trained with RTR targets. Those would show whether the routing or the architecture is doing the work.

Does RTR actually move toward the true onset, or does it lock in the epoch-1 notion of what’s distinguishable from safe responses? A cross-fitted teacher (each example’s curve comes from a model that never trained on it), a fixed linear-ramp target, and recomputing the curve at the end of each epoch instead of caching it mid-training with dropout on would all help. Also, how is the first increment defined in the normalisation? That decides whether harm from the very first token can be represented.

Can you report when alarms fire, not just whether they fire? For example, the distribution of first-alarm positions with and without RTR, and alarm position relative to the onset on data that has onset or sentence-level annotations.

How do results change over several seeds, and with Kelp and the LoRA guards trained for the same number of epochs with the same schedule as ADTGuard?

How does a detector trained on one dataset do on the others? How does the method do on instruct backbones that are monitoring their own generations?

Can you report AUROC or the false-positive rate at a fixed recall on safe responses, and compare ADTGuard’s latency with Kelp and a simple linear probe? That would back up the “competitive latency” claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.