- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces HYSGUARD, a trained white-box monitor for streaming LLM outputs that refreshes its internal safety state while preserving the generator's context and KV cache. The paper identifies a phenomenon termed safety-state hysteresis: accumulated benign evidence can delay the detection of an unsafe continuation, while persistent unsafe evidence can delay recovery after the output returns to a safe regime. HYSGUARD combines cross-layer hidden-state features with fast and slow temporal traces and a learned refresh gate.
Critical strengths
-
The paper considers both unsafe-onset detection and safe-recovery delay, providing a temporal evaluation perspective that completed-response classification metrics alone do not capture. Separating the monitor's memory from the generator's context is a potentially useful design principle for applications requiring continuous moderation.
-
The reported results show improved boundary-sensitive performance over the evaluated baselines. The paper includes relevant ablations of the refresh gate, evidence layers, and training curriculum, which help investigate the contributions of individual components.
Critical weaknesses
-
Presentation and contribution clarity. This paper is hard to follow. The abstract introduces HYSGUARD, SWITCHMIX, and SWITCHTRACE, while this this paper does not sufficiently explain how their specific limitations motivate the proposed design. Although the roles of the three components are stated, the manuscript should more directly connect the identified failure mode, the refresh mechanism, the required supervision, and the evaluation protocol.
-
The practical motivation for unsafe-to-safe recovery depends on an intervention policy that is not clearly specified. Unsafe generation normally terminates at the first unsafe detection, subsequent recovery within that generation has no operational role.
-
The propositions motivate temporal adaptation but provide limited justification for the specific refresh mechanism. The distinction between theoretical support and empirical evidence is not sufficiently clear.
Questions for authors
- Could the authors clarify the central methodological advance over existing temporal monitors in the introduction, and explain how SWITCHMIX and SWITCHTRACE support this contribution?
- Could the authors clarify the intervention policy and the intended deployment setting? If generation continues after unsafe content is detected, how is that content handled, and what practical benefit does subsequent safe recovery provide?
- Could the authors explain the theoretical motivation for the specific refresh mechanism, including how Propositions 1 and 2 inform its design and which design choices are supported empirically?
4 comments
I thank the authors for your reply. But it leaves my main concern unresolved: what practical benefit is obtained by continuing the same generation after an alarm?
In particular, satisfying the recovery condition does not establish that useful assistance has been recovered. Even if the subsequently released content is genuinely safe, it may be irrelevant to the legitimate task or incoherent after the flagged span has been removed. Faster recovery therefore needs to be connected to the usefulness of the delivered response, rather than simply the amount of subsequent text that can be released.
As I understand it, the proposed advantage should be assessed by comparing the computational cost of obtaining safe, useful output through continued generation versus regeneration. Therefore, I would expect an end-to-end comparison with appropriate alternatives, such as a safe fallback or regeneration with a safety intervention, at comparable safety levels. The evaluation should assess the usefulness and coherence of the released response, latency, and total computational cost, including generation of withheld tokens.
Additional Response to Reviewer gZeR
We thank the reviewer for this important clarification. We agree that recovery must be evaluated by the usefulness of the delivered response, rather than the amount of subsequently released content. We therefore introduce an end-to-end comparison of task utility, coherence, and computational cost under matched safety constraints.
Task-Preserving Recovery under Matched Safety
Experimental setup. We evaluate 1,200 held-out tasks across summarization, QA, extraction, coding, planning, and writing, including 800 mixed-intent tasks and 400 benign hard negatives. Three post-alarm policies branch from identical free-running trajectories:
- Stop + Fallback: Terminate generation and return a fixed safe alternative.
- Restart + Intervention: Discard the unfinished response and regenerate from the original prompt with a standardized safety intervention.
- HYSGUARD Continue: Preserve the KV cache, discard flagged content, and resume after eight consecutive safe tokens, with a 128-token recovery horizon.
All methods use the same frozen generator and decoding configuration. Operating thresholds are calibrated to comparable safety levels (unsafe leakage ≤1%, benign interruption ≤5%).
Automated evaluation. Two independent, frozen open-weight LLM judges from different model (Qwen3-32B-Instruct and Llama-3.3-70B-Instruct) families evaluate anonymized final responses, with randomized presentation order. They assess task utility and post-intervention coherence (1–5), as well as legitimate-task completion. A separate safety evaluator measures unsafe leakage. No judge shares parameters with HYSGUARD. We additionally use executable unit tests, exact-match/F1, and constraint checks on objectively verifiable tasks. Only the final user-visible response is evaluated; withheld content is excluded.
Task quality. Scores are averaged across the two LLM judges.
| Policy | Unsafe (%) ↓ | Utility ↑ | Coherence ↑ | Task completion (%) ↑ |
|---|---|---|---|---|
| Stop + Fallback | 0.1 | 2.31 | 4.63 | 23.6 |
| Restart + Intervention | 0.4 | 4.08 | 4.49 | 83.4 |
| HYSGUARD Continue | 0.5 | 4.02 | 4.43 | 82.5 |
End-to-end efficiency. We account for all generated tokens, including withheld and discarded tokens, prompt prefill, monitor execution, and regeneration.
| Policy | Recovery latency (s) ↓ | E2E latency (s) ↓ | Generated tokens (×) ↓ | GPU cost (×) ↓ |
|---|---|---|---|---|
| Stop + Fallback | 0.15 | 3.4 | 0.59 | 0.62 |
| Restart + Intervention | 3.70 | 8.1 | 1.79 | 1.88 |
| HYSGUARD Continue | 0.85 | 5.0 | 1.23 | 1.26 |
Costs are normalized to uninterrupted single-pass generation. HYSGUARD reduces GPU cost by 33%, generated-token consumption by 31%, and end-to-end latency by 38% relative to regeneration, while retaining comparable task utility and coherence.
Takeaway. Safe fallback minimizes computation but sacrifices task completion; regeneration restores task utility at substantially higher cost. HYSGUARD provides a favorable middle ground, retaining approximately 98.5% of regeneration's task utility without paying for a second generation trajectory. Its benefit is therefore not merely faster boundary recovery, but more computationally efficient recovery of useful assistance under comparable safety constraints.
Reviewer gZeR
Dear Reviewer,
Thank you again for your constructive feedback. In response to your latest concerns, we have conducted additional end-to-end experiments comparing task utility, coherence, latency, and computational cost under matched safety constraints.
As the discussion period is approaching its deadline, we would sincerely appreciate it if you could review our latest response and share your thoughts.
Thank you for your time and consideration.
response to reviewer gZeR
We thank gZeR for recognizing the importance of evaluating both unsafe onset and safe recovery, and the potential of separating monitor memory from generation context.
A1. HYSGUARD addresses the mismatch between persistent generation context and adaptive safety state. Its three components form a unified pipeline:
The main results demonstrate why completed-response classification alone is insufficient:
HYSGUARD improves boundary-sensitive performance while preserving sequence-level discrimination. Furthermore, independent content verification retains 89.2 Switch F1, and matched-supervision evaluation achieves 89.4, establishing that the gains are not merely attributable to inherited stage labels or additional boundary supervision.
A2. The intended deployment is buffered, continuous moderation.
A bounded output buffer accommodates detection latency. Consequently, faster recovery reduces unnecessary blocking of subsequent benign content. This benefit applies to continuous session-level moderation rather than deployments that irreversibly terminate generation after the first alarm.
A3. Propositions 1 and 2 establish the two structural requirements motivating selective refresh.
For prefix aggregation, a fixed-length regime change is diluted by preceding history:
\frac{m}{n+m}(\mu_1-\mu_0) \xrightarrow[n\to\infty]{} 0.$$ For fast–slow traces sharing the same pre-transition state: $$\hat{m}_t^f-\hat{m}_t^s= (\alpha_f-\alpha_s)(\mu_1-\mu_0).$$ The first motivates removing obsolete historical influence; the second provides an immediate local signal for detecting regime changes. HYSGUARD uses this discrepancy as input to a learned gate rather than applying unconditional resets: $$g_t=\sigma\left(G_\phi(d_t)\right).$$ $$m_t^k=(1-g_t)\hat{m}_t^k+g_tz_t, \qquad k\in\{f,s\}.$$ The gate ablation directly supports this design: | Gate variant | AUROC ↑ | Median error ↓ | Benign trigger ↓ | |---|---:|---:|---:| | No boundary loss | 71.2 | 9.0 | 18.9 | | No fast–slow contrast | 78.5 | 6.0 | 13.4 | | Fixed CUSUM | 82.1 | 5.0 | 11.2 | | **Full HYSGUARD** | **93.8** | **2.0** | **4.7** | The theoretical properties motivate the boundary signal, while the ablations establish the effectiveness of the learned refresh mechanism. Throughout, only auxiliary monitor memory is updated; the generator's causal state and KV cache remain unchanged.