acceptodds
Under review as a conference paper at ICLR 2027

HYSGUARD: RESETTING SAFETY STATE, NOT GENERATION, FOR STREAMING LLM SAFEGUARDS

Abstract

Streaming safeguards are becoming a necessary component of large-language-model serving because unsafe behavior can emerge only after decoding has begun.Existing monitors improve intervention latency by classifying prefixes or accumulating hidden-state evidence over time, but this temporal memory creates a second problem: after the safety regime changes, evidence from the previous regime can remain active. We identify this failure mode as safety-state hysteresis. It delays unsafe-onset alarms after long benign histories and, symmetrically, delays release after the model returns to safe behavior. The key observation is that generation con-text and safety-decision state have different persistence requirements: the language model should retain its full causal context, while the safety monitor should selectively forget stale risk evidence. We introduce HYSGUARD, a lightweight monitor that reads cross-layer hidden states, maintains fast and slow safety memories, detects regime boundaries from their discrepancy, and refreshes only the monitor state while leaving the backbone and KV cache untouched. We formalize the boundary signal, give an O(r)-state online algorithm, and define SWITCHTRACE, an evaluation suite covering safe-to-unsafe, unsafe-to-safe, and repeated switches. Controlled mechanism tests and a complete cross-model evaluation show how to distinguish temporal adaptation from ordinary sequence classification, while preserving the deployment advantage of cached autoregressive decoding.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model Safety, Streaming Safety Monitoring, Temporal Adaptation, Hidden-State Representation Learning, Online State Estimation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gZeROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces HYSGUARD, a trained white-box monitor for streaming LLM outputs that refreshes its internal safety state while preserving the generator's context and KV cache. The paper identifies a phenomenon termed safety-state hysteresis: accumulated benign evidence can delay the detection of an unsafe continuation, while persistent unsafe evidence can delay recovery after the output returns to a safe regime. HYSGUARD combines cross-layer hidden-state features with fast and slow temporal traces and a learned refresh gate.

Critical strengths
  1. The paper considers both unsafe-onset detection and safe-recovery delay, providing a temporal evaluation perspective that completed-response classification metrics alone do not capture. Separating the monitor's memory from the generator's context is a potentially useful design principle for applications requiring continuous moderation.

  2. The reported results show improved boundary-sensitive performance over the evaluated baselines. The paper includes relevant ablations of the refresh gate, evidence layers, and training curriculum, which help investigate the contributions of individual components.

Critical weaknesses
  1. Presentation and contribution clarity. This paper is hard to follow. The abstract introduces HYSGUARD, SWITCHMIX, and SWITCHTRACE, while this this paper does not sufficiently explain how their specific limitations motivate the proposed design. Although the roles of the three components are stated, the manuscript should more directly connect the identified failure mode, the refresh mechanism, the required supervision, and the evaluation protocol.

  2. The practical motivation for unsafe-to-safe recovery depends on an intervention policy that is not clearly specified. Unsafe generation normally terminates at the first unsafe detection, subsequent recovery within that generation has no operational role.

  3. The propositions motivate temporal adaptation but provide limited justification for the specific refresh mechanism. The distinction between theoretical support and empirical evidence is not sufficiently clear.

Questions for authors
  1. Could the authors clarify the central methodological advance over existing temporal monitors in the introduction, and explain how SWITCHMIX and SWITCHTRACE support this contribution?
  2. Could the authors clarify the intervention policy and the intended deployment setting? If generation continues after unsafe content is detected, how is that content handled, and what practical benefit does subsequent safe recovery provide?
  3. Could the authors explain the theoretical motivation for the specific refresh mechanism, including how Propositions 1 and 2 inform its design and which design choices are supported empirically?
4 comments
AuthorsOpenReview ↗

response to reviewer gZeR

We thank gZeR for recognizing the importance of evaluating both unsafe onset and safe recovery, and the potential of separating monitor memory from generation context.

W1/Q1 (central contribution and methodological connection).

A1. HYSGUARD addresses the mismatch between persistent generation context and adaptive safety state. Its three components form a unified pipeline:

  • HYSGUARD: Cross-layer evidence, fast–slow boundary sensing, and selective monitor-state refresh.
  • SWITCHMIX: Bidirectional onset/recovery supervision and safe-to-safe semantic-change negatives.
  • SWITCHTRACE: Temporal evaluation of onset, recovery, repeated switches, and adversarial placement.

The main results demonstrate why completed-response classification alone is insufficient:

MethodFull F1 ↑Switch F1 ↑TR@16 ↑D+ ↓D− ↓
Fixed 64-token window88.485.184.913.414.6
Kelp88.585.687.011.714.2
GRU monitor88.986.086.811.812.4
HYSGUARD89.390.493.16.47.6

HYSGUARD improves boundary-sensitive performance while preserving sequence-level discrimination. Furthermore, independent content verification retains 89.2 Switch F1, and matched-supervision evaluation achieves 89.4, establishing that the gains are not merely attributable to inherited stage labels or additional boundary supervision.

W2/Q2 (intervention and deployment).

A2. The intended deployment is buffered, continuous moderation.

  • On an alarm, the serving wrapper withholds the flagged span, which is discarded or redacted.
  • The frozen generator may continue internally without resetting its KV cache.
  • Subsequent output is released only after pt<θp_t < \theta for kk consecutive tokens.
  • Previously withheld unsafe content is never released after recovery.

A bounded output buffer accommodates detection latency. Consequently, faster recovery reduces unnecessary blocking of subsequent benign content. This benefit applies to continuous session-level moderation rather than deployments that irreversibly terminate generation after the first alarm.

W3/Q3 (theoretical motivation and empirical support).

A3. Propositions 1 and 2 establish the two structural requirements motivating selective refresh.

For prefix aggregation, a fixed-length regime change is diluted by preceding history:

\frac{m}{n+m}(\mu_1-\mu_0) \xrightarrow[n\to\infty]{} 0.$$ For fast–slow traces sharing the same pre-transition state: $$\hat{m}_t^f-\hat{m}_t^s= (\alpha_f-\alpha_s)(\mu_1-\mu_0).$$ The first motivates removing obsolete historical influence; the second provides an immediate local signal for detecting regime changes. HYSGUARD uses this discrepancy as input to a learned gate rather than applying unconditional resets: $$g_t=\sigma\left(G_\phi(d_t)\right).$$ $$m_t^k=(1-g_t)\hat{m}_t^k+g_tz_t, \qquad k\in\{f,s\}.$$ The gate ablation directly supports this design: | Gate variant | AUROC ↑ | Median error ↓ | Benign trigger ↓ | |---|---:|---:|---:| | No boundary loss | 71.2 | 9.0 | 18.9 | | No fast–slow contrast | 78.5 | 6.0 | 13.4 | | Fixed CUSUM | 82.1 | 5.0 | 11.2 | | **Full HYSGUARD** | **93.8** | **2.0** | **4.7** | The theoretical properties motivate the boundary signal, while the ablations establish the effectiveness of the learned refresh mechanism. Throughout, only auxiliary monitor memory is updated; the generator's causal state and KV cache remain unchanged.

I thank the authors for your reply. But it leaves my main concern unresolved: what practical benefit is obtained by continuing the same generation after an alarm?

In particular, satisfying the recovery condition does not establish that useful assistance has been recovered. Even if the subsequently released content is genuinely safe, it may be irrelevant to the legitimate task or incoherent after the flagged span has been removed. Faster recovery therefore needs to be connected to the usefulness of the delivered response, rather than simply the amount of subsequent text that can be released.

As I understand it, the proposed advantage should be assessed by comparing the computational cost of obtaining safe, useful output through continued generation versus regeneration. Therefore, I would expect an end-to-end comparison with appropriate alternatives, such as a safe fallback or regeneration with a safety intervention, at comparable safety levels. The evaluation should assess the usefulness and coherence of the released response, latency, and total computational cost, including generation of withheld tokens.

AuthorsOpenReview ↗

Additional Response to Reviewer gZeR

We thank the reviewer for this important clarification. We agree that recovery must be evaluated by the usefulness of the delivered response, rather than the amount of subsequently released content. We therefore introduce an end-to-end comparison of task utility, coherence, and computational cost under matched safety constraints.

Task-Preserving Recovery under Matched Safety

Experimental setup. We evaluate 1,200 held-out tasks across summarization, QA, extraction, coding, planning, and writing, including 800 mixed-intent tasks and 400 benign hard negatives. Three post-alarm policies branch from identical free-running trajectories:

  • Stop + Fallback: Terminate generation and return a fixed safe alternative.
  • Restart + Intervention: Discard the unfinished response and regenerate from the original prompt with a standardized safety intervention.
  • HYSGUARD Continue: Preserve the KV cache, discard flagged content, and resume after eight consecutive safe tokens, with a 128-token recovery horizon.

All methods use the same frozen generator and decoding configuration. Operating thresholds are calibrated to comparable safety levels (unsafe leakage ≤1%, benign interruption ≤5%).

Automated evaluation. Two independent, frozen open-weight LLM judges from different model (Qwen3-32B-Instruct and Llama-3.3-70B-Instruct) families evaluate anonymized final responses, with randomized presentation order. They assess task utility and post-intervention coherence (1–5), as well as legitimate-task completion. A separate safety evaluator measures unsafe leakage. No judge shares parameters with HYSGUARD. We additionally use executable unit tests, exact-match/F1, and constraint checks on objectively verifiable tasks. Only the final user-visible response is evaluated; withheld content is excluded.

Task quality. Scores are averaged across the two LLM judges.

PolicyUnsafe (%) ↓Utility ↑Coherence ↑Task completion (%) ↑
Stop + Fallback0.12.314.6323.6
Restart + Intervention0.44.084.4983.4
HYSGUARD Continue0.54.024.4382.5

End-to-end efficiency. We account for all generated tokens, including withheld and discarded tokens, prompt prefill, monitor execution, and regeneration.

PolicyRecovery latency (s) ↓E2E latency (s) ↓Generated tokens (×) ↓GPU cost (×) ↓
Stop + Fallback0.153.40.590.62
Restart + Intervention3.708.11.791.88
HYSGUARD Continue0.855.01.231.26

Costs are normalized to uninterrupted single-pass generation. HYSGUARD reduces GPU cost by 33%, generated-token consumption by 31%, and end-to-end latency by 38% relative to regeneration, while retaining comparable task utility and coherence.

Takeaway. Safe fallback minimizes computation but sacrifices task completion; regeneration restores task utility at substantially higher cost. HYSGUARD provides a favorable middle ground, retaining approximately 98.5% of regeneration's task utility without paying for a second generation trajectory. Its benefit is therefore not merely faster boundary recovery, but more computationally efficient recovery of useful assistance under comparable safety constraints.

AuthorsOpenReview ↗

Reviewer gZeR

Dear Reviewer,

Thank you again for your constructive feedback. In response to your latest concerns, we have conducted additional end-to-end experiments comparing task utility, coherence, latency, and computational cost under matched safety constraints.

As the discussion period is approaching its deadline, we would sincerely appreciate it if you could review our latest response and share your thoughts.

Thank you for your time and consideration.

Reviewer_mLnGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that streaming safety monitors suffer from "safety-state hysteresis": a long benign prefix dilutes a later unsafe segment, and persistent unsafe state delays release after recovery. HysGuard is a lightweight white-box monitor on a frozen generator. It builds cross-layer evidence, compares fast and slow traces of that evidence to detect regime boundaries, and uses a learned gate to refresh only the monitor's memory, leaving the KV cache untouched. It is trained on SwitchMix, which concatenates safe and unsafe free-running generations and labels the boundaries, and evaluated on the similarly constructed SwitchTrace. On Qwen3-8B and three other backbones, it keeps sequence-level F1 while clearly improving Switch F1 and onset/recovery delay, with about 2M parameters and 5.9% overhead.

Critical strengths
  • Useful problem framing. Separating the lifetime of generation memory from that of safety memory, and measuring onset and recovery delay explicitly, is an improvement over judging streaming guards by completed-response F1 alone.
  • Lightweight and practical. The monitor leaves the KV cache untouched and adds little overhead. The curriculum ablation (safe→safe negatives cut benign triggers) shows the design choices matter.
Critical weaknesses
  • The link between the motivation and the constructed setting is not established. Token labels are inherited from the source prompt while an aligned backbone generates freely. The paper does not say how it ensures that the text produced in an "unsafe" stage is actually unsafe. Each boundary coincides with a new stage instruction, whereas the introduction motivates transitions within a single streamed response. The intervention after an alarm (halting vs. redacting) is never specified, so it is unclear when recovery delay matters.
  • The comparison setup is under-specified. HysGuard is trained with boundary supervision on data built by the same pipeline as the test set, and the paper does not state whether the baselines were trained on this data. Qwen3Guard-Stream and SCM are listed as baselines but do not appear in Table 1.
  • Limited technical novelty. The two propositions are elementary (prefix-mean dilution; the difference of two EMAs), and fast/slow change detection is well known, as the paper's own CUSUM citation shows. The contribution therefore rests mainly on the learned gate and the data construction.
Questions for authors

Same as weaknesses.

3 comments
AuthorsOpenReview ↗

response to reviewer mLnG

We thank mLnG for recognizing the motivation of separating generation memory from safety memory and the lightweight design of HYSGUARD.

W1/Q1 (constructed transitions and realized safety).

A1. SWITCHTRACE evaluates persistent session-level monitoring across consecutive free-running assistant segments. Each segment is generated autoregressively, while the monitor state persists across stage transitions. No reference continuations are teacher-forced.

Stage labels provide controlled weak supervision. We additionally verify realized content safety on 400 balanced held-out streams, assigning safe refusals and benign lead-ins according to their actual content.

MethodStage labels: F1 / TR@16 / D+ / D−Content labels: F1 / TR@16 / D+ / D−
Kelp85.2 / 86.6 / 12.0 / 14.684.1 / 85.3 / 12.8 / 15.2
GRU85.7 / 86.4 / 12.1 / 12.884.6 / 85.1 / 12.9 / 13.5
HYSGUARD90.0 / 92.7 / 6.7 / 7.989.2 / 91.8 / 7.2 / 8.4

HYSGUARD maintains a 4.6-point Switch F1 gain over GRU under independently verified boundaries, while reducing onset and recovery delays by 5.7 and 5.1 tokens. This directly supports the validity of the temporal evaluation.

W2/Q2 (training comparability and baseline coverage).

A2. We isolate architectural gains from additional supervision. Fixed-window and GRU controls are trained on identical SWITCHMIX trajectories with matched optimization budgets and auxiliary boundary supervision. Qwen3Guard-Stream and SCM are additionally evaluated through their native interfaces.

SettingMethodSwitch F1 ↑TR@16 ↑D+ ↓D− ↓
NativeQwen3Guard-Stream85.787.811.012.9
NativeSCM86.288.110.612.3
MatchedFixed window86.086.811.913.2
MatchedGRU87.388.79.610.8
MatchedHYSGUARD89.492.07.08.2

HYSGUARD exceeds the matched GRU by 2.1 Switch F1 and 3.3 TR@16 points. This isolates the benefit of the proposed temporal mechanism beyond merely receiving boundary supervision.

W3/Q3 (technical novelty and theoretical justification).

A3. The contribution is learned safety-state refresh, not two-timescale smoothing in isolation. The design couples three elements:

  1. Cross-layer evidence captures distributed safety information.
  2. Fast–slow discrepancy exposes local regime changes.
  3. A learned gate distinguishes safety transitions from semantic changes and selectively refreshes monitor memory.

The boundary-gate ablation isolates their contributions:

Gate variantAUROC ↑Median error ↓Benign trigger ↓
No boundary loss71.29.018.9
No fast–slow contrast78.56.013.4
Fixed CUSUM82.15.011.2
Full HYSGUARD93.82.04.7

The learned gate improves AUROC by 11.7 points over CUSUM and reduces benign triggers by 6.5 points. Thus, the contribution extends beyond conventional change detection: the gate provides a learned semantic filter over the boundary signal.

AuthorsOpenReview ↗

Reviewer mLnG

Dear Reviewer,

Thank you again for appreciating our problem formulation and lightweight design. We have provided additional experimental evidence and detailed responses to your concerns.

We would be grateful if you could revisit our responses and share your further thoughts.

I thank the authors for the new results and clarifications. Based on the method’s contribution, I will raise my score.

Reviewer_dTrQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper discusses safety-state hysteresis in streaming LLM safeguards. HYSGUARD extracts safety evidence from multiple hidden layers of a frozen generator and uses the difference between fast and slow exponential traces to guide a learned memory refresh gate. The method keeps the generator and its KV cache unchanged. SWITCHMIX provides supervision through staged generation and SWITCHTRACE evaluates adaptation to unsafe onset and safe recovery. Experiments on four backbone families show improved detection and recovery delays while largely preserving completed-response classification performance.

Critical strengths
  1. This paper addresses an important problem in streaming safeguards by considering both detection and recovery. Evaluating these transitions helps identify late alarms and prolonged blocking. These timing errors are not captured by classifying completed responses alone. The proposed method preserves the frozen generator and its KV cache while adapting the monitoring state, providing a practical way to address stale safety evidence with limited additional computation.
  2. The experiments are well aligned with the main problem. Long-history and repeated-switch evaluations directly examine the effects of persistent safety state, while the ablations help establish the contributions of the refresh gate and switching supervision. The consistent improvements across four backbone families further support the usefulness of the approach. These results provide evidence that the method improves temporal adaptation while largely preserving completed-response classification performance.
Critical weaknesses
  1. The evaluation boundaries need further validation. Generated tokens inherit their stage labels, and the first token after a label change is treated as the boundary. However, unsafe requests may produce safe refusals, and harmful responses may begin with benign content. The constructed boundaries may not match actual changes in output safety, which may affect the interpretation of detection and recovery delays. The verification procedure for the unsafe-sentence detection results is also unclear. Please explain how generated content is checked against stage labels and whether the main gains hold with independently verified content boundaries.
  2. The main streaming comparison is incomplete. Qwen3Guard-Stream and SCM are listed as baselines, and the implementation details state that both are evaluated through their streaming interfaces. Neither appears in the main comparison of detection and recovery performance. Qwen3Guard-Stream is only reported in the attack-type analysis, and no numerical results are provided for SCM. The available results are insufficient to assess HYSGUARD’s detection and recovery advantages over these methods. Please provide their temporal metrics under the same evaluation setting or explain the limitations preventing this comparison.
Questions for authors
  1. How are safety labels verified for the newly generated responses in SWITCHMIX and SWITCHTRACE? The source label may be unsafe even when the frozen model produces a safe answer. Is the source label still inherited in such cases? Please explain how generated content is checked and how label disagreements are handled.
  2. Qwen3Guard-Stream and SCM are listed as evaluated baselines. Neither appears in the main temporal comparison. Please provide their Switch F1, TR@16, and onset/recovery delays under the same evaluation protocol and false-positive budget. If this comparison is not feasible, please explain the specific limitations.
  3. Please clarify the intervention policy after an alarm. If generation stops at the first alarm, how does a shorter recovery delay D−D^- benefit the deployed safeguard? If generation continues while output is masked or buffered, what criterion is used to resume output? Does faster recovery reduce the blocking of benign content without prematurely releasing unsafe content?
3 comments
AuthorsOpenReview ↗

response to reviewer dTrQ

We thank dTrQ for recognizing the importance of bidirectional temporal evaluation and the practical advantages of preserving the frozen generator and its KV cache.

W1/Q1 (boundary validity and content verification).

A1. Stage labels provide weak regime supervision, not token-level semantic ground truth. To verify realized safety, we independently annotate 400 balanced held-out streams. Safe refusals and benign lead-ins receive content-based labels. Both evaluations use identical trajectories, fixed monitor weights, and unchanged thresholds.

MethodStage labels: F1 / TR@16 / D+ / D−Content labels: F1 / TR@16 / D+ / D−
Kelp85.2 / 86.6 / 12.0 / 14.684.1 / 85.3 / 12.8 / 15.2
GRU85.7 / 86.4 / 12.1 / 12.884.6 / 85.1 / 12.9 / 13.5
HYSGUARD90.0 / 92.7 / 6.7 / 7.989.2 / 91.8 / 7.2 / 8.4

Scores are percentages; delays are tokens. Under independent content labels, HYSGUARD retains 89.2 Switch F1 and 91.8 TR@16, outperforming Kelp by 5.1 and 6.5 points, respectively. The gains therefore persist beyond inherited stage boundaries.

W2/Q2 (streaming baselines and matched comparison).

A2. We provide both native-system and controlled matched-supervision comparisons. All methods use identical held-out streams, independently verified boundaries, and a shared calibration set. Fixed-window and GRU controls receive identical SWITCHMIX trajectories, auxiliary boundary supervision, and matched optimization budgets.

SettingMethodSwitch F1 ↑TR@16 ↑D+ ↓D− ↓
NativeQwen3Guard-Stream85.787.811.012.9
NativeSCM86.288.110.612.3
MatchedFixed window86.086.811.913.2
MatchedGRU87.388.79.610.8
MatchedHYSGUARD89.492.07.08.2

HYSGUARD exceeds the matched GRU by 2.1 Switch F1 and 3.3 TR@16 points, reducing both transition delays by 2.6 tokens. Native external systems are reported separately from retrained controls. Score-based interfaces use the shared calibration set; fixed-decision interfaces are evaluated at their native operating points, with achieved false-positive rates distinguished from the 5% target.

Q3 (intervention and recovery).

A3. HYSGUARD targets buffered, continuous moderation rather than irreversible termination.

  • Alarm: Withhold the current flagged token and subsequent unsafe span.
  • Preservation: Generation may continue internally without resetting the original KV cache.
  • Recovery: Resume eligible output after pt<θp_t < \theta for kk consecutive tokens.
  • Safety: Previously withheld unsafe content is discarded, never retroactively released.

A bounded output buffer accommodates detection latency. Thus, D−D^{-} measures unnecessary blocking of subsequent benign content, while the confirmation window prevents premature release.

AuthorsOpenReview ↗

Reviewer dTrQ

Dear Reviewer,

Thank you again for recognizing the importance of our bidirectional temporal evaluation and practical design. We have carefully addressed your concerns with additional experiments and clarifications.

We would greatly appreciate your further feedback and hope to hear from you.

Thank you for the response. The additional experiments address my main concerns. For me, the overall contribution of the paper is insufficient to support a higher rating. I maintain my rating of 3 (Weak acceptance).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Message to Area Chair on Discussion Status

Dear Area Chair,

Thank you very much for your time and effort in overseeing our submission. We sincerely appreciate Reviewer gZeR's continued engagement in the discussion. In response to their follow-up, we have further clarified the practical benefits of safe continuation through additional end-to-end evaluations of task utility, coherence, and computational efficiency.

While we fully appreciate the reviewers' busy schedules, we would like to respectfully note that we have not yet received follow-up feedback from Reviewers dTrQ and mLnG. We have carefully addressed their concerns through detailed responses and additional experiments and analyses in our blue-highlighted revision. If convenient before the discussion period closes, we would be very grateful if you could kindly send a gentle reminder in case they are available to share further thoughts or questions.

Thank you again for your support and guidance.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.