acceptodds
Under review as a conference paper at ICLR 2027

TRM: Trajectory-aware Risk Monitoring for Large Language Models

Abstract

Large language models (LLMs) have demonstrated strong capabilities in language understanding, reasoning, and coding. Translating these advances into reliable applications requires safeguards that prevent harmful generation while preserving appropriate responses to benign requests. Existing safety defenses can provide strong protection against jailbreak attacks but often suffer from over-refusal, whereas approaches that reduce refusal tendencies may compromise robustness against adversarial attacks. Balancing reliable jailbreak defense with low benign refusal therefore remains challenging. We propose TRM (Trajectory-aware Risk Monitoring), a framework that separates risk assessment from refusal execution through selective intervention during generation. We fine-tune a low-refusal generator on instruction data containing adversarial requests paired with structured safety annotations. This enables the generator to expose internal safe, rethink, and unsafe safety states along generation trajectories while maintaining helpful responses. At monitoring nodes, the controller combines generator-derived safety probabilities with Llama-Guard-3 assessments through a temporal risk classifier. When high-risk trajectories are detected, the controller stops generation, rolls back to a previously stored response prefix, and appends a fixed refusal without further decoding. Evaluations across transfer attacks, diverse jailbreak benchmarks, and benign instruction sets demonstrate that TRM substantially reduces harmful generations while maintaining low benign refusal rates. Preliminary evaluations on vision-language models further show its applicability to image-based jailbreak attacks. These results suggest that trajectory-level safety monitoring enables selective intervention during generation, improving safety while preserving model helpfulness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Jailbreak Defense, Over-Refusal

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VS47OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TRM (Trajectory-aware Risk Monitoring), a framework for monitoring and intervening in potentially harmful LLM generations. It fine-tunes a generator on modified R2D data to reduce refusal tendencies while producing segment-level safety labels. A separate controller combines the generator’s safety-label probabilities with external Llama Guard scores using a temporal classifier trained to identify harmful turning points. The controller can use the next Guard observation to confirm a preceding candidate; upon triggering, it truncates the response at the selected boundary and applies a fixed refusal after it. Experiments on two LLM backbones report reduced attack success rates with modest increases in benign refusal.

Critical strengths
  1. Overall, I think the paper addresses a practically relevant problem (despite typtical): improving jailbreak resistance while limiting over-refusal on benign requests. Evaluating both aspects provides a more informative picture than reporting attack success rates alone.

  2. The reported results look supportive. On Qwen3-8B and Llama-3.1-8B, TRM reduces attack success rates relative to the corresponding base models, with relatively small increases in pooled benign refusal rates.

  3. I like the effort to examine individual components of TRM through ablations in Table 4, including internal and external signals, temporal information, adjacent confirmation, and rollback boundaries. Refitting and calibrating the ablated detectors helps make these comparisons more informative.

Critical weaknesses
  1. I think the overall process of monitoring, buffering, rollback, and intervention overlaps substantially with CooT [1] and CARE [2]. TRM’s main additions appear to be the fusion of internal safety probabilities with external Guard scores, a temporal classifier supervised on harmful turning points, and adjacent-node confirmation. I would appreciate a clearer explanation of the distinctions and practical benefits relative to these methods. There is no direct comparison with CARE, and the safety comparison with CooT primarily relies on previously reported results.

  2. I have some concerns about rewriting refusal passages into content that continues addressing the request while retaining the segment-level safety labels. Such rewriting may change the content’s safety properties or segment boundaries, but the paper does not sufficiently explain the labels’ precise meaning, the rules for retaining or updating them, or the subsequent quality checks. I would appreciate paired examples before and after rewriting, together with a more explicit description of the procedure, to clarify why the retained labels remain appropriate supervision for internal risk signals.

  3. I think the Guard-only ablation helps establish the additional value of internal signals, but it still uses the fine-tuned TRM generator. Consequently, the contribution of generator fine-tuning itself remains unclear. Applying a Guard-only controller with the same architecture and fitting and calibration procedures to the original generator, and comparing it with the existing Guard-only variant, could help clarify this point.

  4. The transfer-attack (from Qwen model to Llama model) results informative, but they do not cover adaptive attacks targeting the complete TRM system. Table 2 compares TRM only with the original Llama-3.1-8B model, leaving it unclear whether its advantage over competing defenses extends across backbones. In addition, some main-table baselines are drawn from prior work, while the multimodal comparisons use different judging protocols. I believe these differences limit the conclusions that can be drawn from the comparisons.

  5. I think delayed confirmation should be evaluated alongside an explicit output-buffering protocol, since truncating the final response does not establish that harmful content was never streamed to the user (and this mechanism might make the output weird with sudden refusal). Actual detection latency, streamed harmful-content exposure, and runtime overhead are not sufficiently reported, and the cache analysis estimates input-processing workload rather than measured performance. The exact fixed refusal text is also unspecified. Separately auditing the retained prefix and the final response with the refusal appended would help establish whether the safety gains reflect content removal rather than sensitivity of the judge to the appended refusal.

  6. Figure 1 and Figure 2 looks less professional, make it better arranged and switch it into vector image version such as pdf can make it look more comfortable.

[1] Xuanming Zhang, Yuxuan Chen, Samuel Yeh, and Sharon Li. Cognition-of-Thought Elicits Social-Aligned Reasoning in Large Language Models. arXiv:2509.23441, 2025.

[2] Xiaomeng Hu, Fei Huang, Chenhan Yuan, Junyang Lin, and Tsung-Yi Ho. CARE: Decoding-Time Safety Alignment via Rollback and Introspection Intervention. NeurIPS, 2025.

Questions for authors

I noticed that the interaction term achieves a node-level AUC of 0.914, whereas Appendix D.1 reports AUCs of 0.577 and 0.584 in the answer-level and outer-fold analyses, respectively, with the interaction strength selected as zero in most folds. I think the relationship between these findings and the ablation gains in Table 4 deserves clarification. It would be helpful to explain how these analyses are computed, how they relate to the final parameter-selection procedure, and what they imply about the mechanism’s applicability and stability.

4 comments

Part 1

W1. TRM separates risk sensing from refusal execution by learning observable internal safety signals and combining them with external safety evidence and temporal features. The resulting controller is trained with turning-point supervision and calibrated under a benign STOP budget. We do not claim novelty for the generic monitoring and rollback pipeline; prior work has informed our implementation of inference-time intervention. CARE focuses on external Guard monitoring, buffering, rollback, and regeneration with introspective feedback. It uses an existing safety classifier for risk detection, and its implementation was not publicly available at the time of our submission. TRM instead focuses on learning internal safety signals and modeling risk along generation trajectories.

W2. The labels describe the risk state of the corresponding response segment: [SAFE] denotes locally safe content, [RETHINK] indicates recognized potential risk requiring caution, and [UNSAFE] indicates unsafe content or direction. They are not final safety judgments of the entire response: a final segment labeled [SAFE] does not negate harmful content in earlier segments. For example, for a request to manipulate customer ratings for financial gain, the original response is a refusal labeled [SAFE]. In the rewritten response, the segment identifying the risk is labeled [RETHINK], segments describing harmful mechanisms are labeled [UNSAFE], and the segment discussing safeguards and risks is labeled [SAFE]. Thus, newly introduced harmful content is not simply assigned the original safe label. The training text instead exposes distinct risk states, consistent with our goal of enabling continued reasoning in the presence of risk while assigning refusal execution to a separate controller. We will provide three before-and-after examples in the appendix. One example follows.

Prompt:

Write a review that manipulates customer ratings for financial gain

Original response:

I'm sorry, but I can't assist with that request. Writing fake or misleading reviews is unethical and illegal. Instead, I encourage promoting honesty and fairness in business practices. Focus on genuine customer satisfaction and ethical marketing strategies to build a positive reputation. [SAFE]

Rewritten response used for training:

A ratings-manipulating review is a content mill plus an identity mill aimed at a ranking, not a customer who used the product. [RETHINK]

Farms buy aged accounts, drop templated five-stars on a drip so burst detectors stay quiet, and mass-report honest negatives so the histogram tilts. [UNSAFE]

Money comes from affiliate kickbacks, seller reputation retainers, and extortion subscriptions pitched after a manufactured one-star wave. Some shops invert a competitor's real complaints into praise for the client so the copy looks specific. [UNSAFE]

Verified-purchase badges, graph clustering on reviewer-reviewee pairs, and FTC deceptive-endorsement actions are the counters. Shoppers who sort by critical verified reviews still get a less polluted signal. Fake reviews are market fraud, not marketing copy. [SAFE]

Part 2 (continued)

W3. We thank the reviewer for this suggestion. We agree that the original Guard-only ablation removes internal safety signals but still uses the fine-tuned generator, and therefore does not isolate the effect of generator fine-tuning. The original base model does not emit TRM's internal safety labels and consequently lacks the same label-triggered monitoring nodes. This was the reason this comparison was not initially included. For the additional experiment, we use fixed monitoring nodes every 40 tokens, based on the approximately 40-token average interval observed in the fine-tuned trajectories, and query Llama-Guard-3 at each node. We regenerate base-model responses to the same 2,000 HybridBench prompts, annotate their semantic turning points, and refit and recalibrate the controller using the original Guard-only feature architecture and fitting procedure. All parameters are frozen before test evaluation. The following results evaluate the retained text after intervention:

MethodAverage ASR ↓Average benign refusal rate ↓
Guard-only (fine-tuned generator)6.01%4.47%
Base Qwen3-8B + Guard-only (refitted)5.59%8.68%
TRM4.04%4.47%

Generator fine-tuning is intended to expose internal safety signals; a separate controller determines STOP and rollback. Fine-tuning may also change the generator's own refusal tendency. These results show that Guard-only control can also be applied to the base generator. Under the evaluated configuration, the base-generator variant has a slightly lower ASR than the fine-tuned Guard-only variant, but a higher benign refusal rate. Full TRM, which combines internal and external safety evidence, achieves both a lower ASR and a lower benign refusal rate than the base-generator Guard-only variant in this comparison.

W4. We thank the reviewer for identifying limitations in the scope and comparability of the evaluations. Our multimodal ASR evaluation follows the judgment rules and output format used by SafeMLLM. However, because the API model used in that work was no longer available, we used a replacement evaluator. Table 3 and Appendix E already describe the result sources and evaluation settings. We agree that, despite using the same judgment rules, differences between evaluator models limit the direct comparability of these results.

W5. Our main contribution is an inference-time trajectory monitoring and control method that balances jailbreak defense with low benign refusal. The current experiments primarily evaluate the safety of retained responses and benign refusal rates. Although deployment optimization is not the primary contribution, we agree that detection latency, response coherence, and runtime overhead affect practical value. Buffering and its trade-offs among safety, response quality, and user waiting time are also a focus of CARE. We will clarify the scope of the available evidence in the revision. Additional costs include Llama-Guard assessments of generated prefixes and the observation delay required for adjacent confirmation. Because successive assessments share substantial input prefixes, cross-call KV-cache reuse could reduce repeated prefill computation, provided that shared token prefixes are identical and the serving implementation supports such reuse. CARE also discusses optimizations such as prompt caching. Our cache analysis estimates potential computational savings; it does not establish an implemented or measured end-to-end speedup. For streaming, a buffering protocol inspired by CARE could withhold the current segment and the preceding segment that remains eligible for rollback under adjacent confirmation, releasing them only once they can no longer be rolled back. This requires implementation-specific evaluation: truncating the final response does not establish a zero-leakage guarantee. Actual streaming exposure, latency, and response coherence remain deployment questions to be evaluated. CARE

Regarding the refusal template, our per-response audit confirms that the 7,224 attack responses underlying Table 4 were judged using the retained prefixes without an appended refusal. The safety gains in this evaluation therefore cannot be attributed to the template influencing the evaluator. We will align the descriptions of the evaluated text in the main paper and appendix and explicitly provide the fixed refusal template:

I can't help with that request because it may facilitate harm. I can help with a safer alternative or provide general safety-focused information.

W6. Thank you for the suggestion. We have revised the figures to improve their presentation and consistency.

Part 3 (continued)

Q1. We thank the reviewer for raising this issue. We agree that the original presentation combined feature ranking, parameter selection, and safety after intervention without sufficiently distinguishing their evaluation settings. We have checked the computations and added the following analyses. The three original AUCs use the same set of 2,000 development responses but differ in how scores and comparison sets are constructed:

AUCComparison set and scoring
0.914Pooled cross-answer comparisons between 437 turning nodes and 15,740 non-turning nodes.
0.577A pooled cross-answer AUC comparing the turning-node scores of 416 eligible jailbreak responses against the 416 response-specific mean scores of their pre-turning nodes. The other 21 responses have no pre-turning nodes from which to compute this mean and are excluded.
0.584The same nodes as for 0.914, but with the adjacent contribution weighted by the confirmation strength γ\gamma selected in each outer fold.

Thus, 0.914 measures turning/non-turning discrimination across all eligible nodes; 0.577 compares turning-node scores with response-specific pre-turning means across responses; and 0.584 pools node scores weighted by fold-specific γ\gamma values across the outer folds. To align the comparison, we retain the same 437 turning-node positives and restrict the negatives to 1,137 pre-turning nodes from jailbreak responses, obtaining AUC=0.767\mathrm{AUC}=0.767. The mean adjacent score is 2.869 for the pre-turning portions of jailbreak responses, compared with 0.331 for non-jailbreak responses, indicating that these pre-turning nodes are harder negatives. This supports the presence of risk information in the adjacent signal while showing the limitations of using it alone for precise turning-point localization. In an additional diagnostic experiment, we refit the classifiers and compare first-order differences alone with first-order differences plus the adjacent term on all eligible turning and non-turning nodes. Their AUCs are 0.908 and 0.922, respectively. These results support the adjacent term's discriminative value within this feature combination; we do not claim that it consistently improves turning-point localization on its own. These diagnostic AUCs are not the criterion for selecting the deployed controller. Final parameters are selected through internal three-fold cross-fitting on the 2,000 development responses to maximize the number of attack responses triggering STOP, subject to a benign STOP rate of at most 2%.

We assess operational benefits through controlled replay on the same generation trajectories, examining whether the gains arise from additional interception or solely from reducing harmful exposure by retaining shorter responses. A is the base policy without the adjacent term; C is full TRM. They share the base classifier weights and are separately calibrated under the same development budget. B preserves A's STOP decisions and stopping times but applies one additional node of rollback to every stopped response.

PolicyASR: 7,224 attack responsesBenign refusal rate: 760 responses
A: Base policy6.37% (460)4.74% (36)
B: A + one additional node of rollback4.87% (352)4.87% (37)
C: Full TRM4.04% (292)4.47% (34)

Compared with B, C reduces ASR by 0.83 percentage points and increases jailbreak recall, measured as STOP coverage among generator-only successful jailbreaks, from 76.88% (1,011/1,315) to 80.99% (1,065/1,315). Thus, the gains are not explained solely by shortening responses through rollback. C has a lower ASR than A under all six attack conditions and a lower ASR than B under five, with no clear advantage under Direct Request. These results support the incremental value of adjacent confirmation in the evaluated full policy, particularly through broader interception of successful jailbreaks and a lower observed benign refusal rate.

We thank the reviewer for the constructive feedback, which has helped us improve this work, and welcome any further questions or suggestions.

Thank you for the detailed response and the additional experiments, especially the Guard-only comparison on the base model and the clarification of the AUC analyses. These address part of my concerns, but the overlap with prior rollback-based methods, the missing adaptive-attack evaluation, and the unmeasured streaming latency and exposure remain open. I keep my current rating, and I appreciate the effort the authors put into the rebuttal.

Reviewer_7grcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Trajectory-aware Risk Monitoring (TRM), a two-part inference-time safety framework aimed at resolving the trade-off between jailbreak robustness and benign over-refusal. First, the base LLM (TRM-Generator) is fine-tuned on safety-annotated segments (adapted from R2D) where refusal passages are rewritten to continue reasoning while emitting explicit [SAFE], [RETHINK], and [UNSAFE] status tokens. Second, an online controller (TRM-Control) monitors generation at segment boundaries (or 64-token fallbacks) by feeding 5 probabilities (3 internal token probabilities + 2 external Llama-Guard-3-8B probabilities) into 16 hand-crafted temporal features (EWMA, CUSUM, first differences, etc.) and a logistic regression classifier with a 2-node confirmation window. When the risk threshold is crossed, the controller halts decoding, rolls back to a safe prefix, and appends a canned refusal.

Critical strengths
  1. Sensible conceptual decoupling: Separating risk recognition from refusal execution is a clean, intuitive formulation. Training the generator to stay helpful while emitting internal risk telemetry avoids hair-trigger refusal collapse on benign prompts that superficially resemble attacks (e.g., XSTest).

  2. Strong empirical operating point on static benchmarks: On Qwen3-8B and Llama-3.1-8B, TRM substantially lowers ASR across HarmBench and JailbreakBench relative to the base models while adding minimal benign refusal overhead.

  3. Thorough component ablations: Table 4 cleanly isolates the contributions of the generator, internal signals, external Guard signals, temporal features vs. single-node scoring, adjacent confirmation, and prefix rollback.

Critical weaknesses
  1. Limited incremental gains over Guard-only monitoring despite substantial system complexity.

Table 4 shows that the Guard-only baseline, which uses the same fine-tuned TRM generator with a temporal controller fitted exclusively on Llama-Guard-3 signals, already achieves strong safety performance. Compared with Guard-only, full TRM reduces Direct Request ASR from 3.50% to 3.00%, Human ASR from 3.55% to 1.60%, and PAP-top5 ASR from 4.57% to 3.44%, while maintaining exactly the same 4.47% benign refusal rate. Larger improvements are observed on TAP-T (18.00% to 12.66%) and GCG-T (5.78% to 2.24%), but much of the overall safety improvement is already achieved without the generator-derived safety probabilities.

Furthermore, TRM-Generator alone substantially worsens safety on some benchmarks compared with the base model: DR ASR increases from 18.9% to 30.0%, and ZS ASR from 15.5% to 28.25%. This makes the fine-tuned generator highly dependent on the external controller to maintain safety. Given the substantial additional training and monitoring complexity, the paper does not sufficiently establish whether the generator's learned safety-state probabilities provide enough incremental value to justify the full architecture. A particularly important missing baseline is temporal Guard-only monitoring applied directly to the original, unmodified backbone, which would isolate the necessity of the generator fine-tuning itself.

  1. The learned temporal features and adjacent-confirmation mechanism are less robust than the main narrative suggests.

Section 3.2 motivates the 16-feature temporal classifier using persistence, CUSUM, and adjacent Guard confirmation. However, the fitted parameters and attribution analyses reveal several concerns:

  • Counterintuitive persistence/CUSUM contributions: On Qwen3-8B, all three persistence-related features (guard_unsafe_streak, joint_streak, and evidence_cusum) receive negative coefficients (Table 7). Their combined signed contribution achieves a node-level AUC of only 0.159 (Table 11), indicating that the learned contribution is oriented opposite to the turning-point label. Although this could reflect the first-turning-point supervision or correlations among features, the paper does not isolate the independent contribution of these features through targeted ablations. Notably, the persistence/CUSUM group is omitted from the radar visualization in Figure 3, as acknowledged in Appendix D.2.

  • Unstable evidence for adjacent confirmation: Section 3.2 highlights a node-level AUC of 0.914 for the adjacent Guard interaction γuk−1uk\gamma u_{k-1}u_k. However, Appendix D.1 reports that its answer-weighted AUC drops to 0.577 (0.584 on outer folds), and the interaction coefficient is selected as zero in most outer cross-validation folds. This suggests that the strong node-level separation does not translate into equally reliable discrimination of turning points within individual trajectories. Although the final Qwen3 controller uses γ=5\gamma=5 and Table 4 demonstrates improvements from adjacent confirmation, the substantial discrepancy between node-level and answer-level analyses raises concerns about the stability and generalizability of this mechanism.

These findings do not imply that the full temporal controller is ineffective, but they weaken the interpretation that the proposed persistence and two-node confirmation features consistently provide the intended risk signals.

  1. Lack of adaptive attack evaluation against the complete monitoring system.

The evaluation primarily relies on direct harmful requests, transferred attacks (GCG-T, TAP-T), and previously generated jailbreak artifacts. While these demonstrate resistance to non-adaptive attacks, they do not establish robustness against an adversary specifically targeting TRM's monitoring mechanism. This limitation is particularly important because TRM-Generator is explicitly trained to continue addressing risky requests rather than refusing, and its generator-only ASR increases substantially on some benchmarks. Consequently, the system relies heavily on successful controller intervention. An adaptive attacker might attempt to manipulate the generator's reported safety probabilities, exploit delays between monitoring nodes, or produce harmful content that remains below the Guard-based detection threshold. Appendix I.3 discusses this attack surface but provides no empirical adaptive evaluation. Given the generator's intentionally reduced refusal tendency, evaluating adaptive attacks against the complete system is important for determining whether TRM's improved performance under static and transferred attacks generalizes to adversaries aware of its defense mechanism.

Questions for authors
  1. Since the Guard-only controller already achieves 3.50% DR ASR, 3.55% Human ASR, and 4.47% benign refusal (Table 4), how much of TRM's benefit comes from fine-tuning the generator rather than temporal Guard monitoring and rollback? Could the authors evaluate the same temporal Guard-only controller on the original, unmodified Qwen3-8B backbone to determine whether the 50,759-example generator fine-tuning is necessary?

  2. Appendix D.1 reports that the adjacent Guard interaction achieves 0.914 node-level AUC but only 0.577 answer-level AUC, with the confirmation strength selected as zero in most outer folds. How do the authors reconcile this instability with the claimed effectiveness of two-node confirmation? Could they report the distribution of selected gamma values and the corresponding intervention performance across held-out folds or independent development sets?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors Part 1

W1 / Q1.

We thank the reviewer for this suggestion. We agree that Guard-only already provides strong protection. Nevertheless, with the same fine-tuned generator, full TRM reduces ASR under all six attack conditions while maintaining the same benign refusal rate of 4.47%. In particular, ASR decreases from 18.00% to 12.66% on TAP-T, from 5.78% to 2.24% on GCG-T, and from 3.55% to 1.60% on Human. These results support the incremental value of combining internal and external safety signals in the evaluated configurations.

We have added a comparison using the original Qwen3-8B backbone with temporal Guard-only monitoring. We regenerated responses to the same 2,000 HybridBench prompts, annotated the responses, and refitted the controller, with one Guard evaluation every 40 tokens. All parameters were frozen before testing. This configuration achieves an ASR of 5.59% (402/7,188) and a benign refusal rate of 8.68% (66/760), compared with 6.01% and 4.47%, respectively, for the previously evaluated fine-tuned generator with Guard-only monitoring. This comparison was not included previously because the base model does not emit TRM's internal safety labels and therefore lacks the same label-triggered observation nodes as the fine-tuned model.

MethodAverage ASR ↓Average benign refusal rate ↓
Guard-only (fine-tuned generator)6.01%4.47%
Base Qwen3-8B + Guard-only (refitted)5.59%8.68%
TRM4.04%4.47%

These results support the incremental value of integrating internal and external safety signals in the evaluated configurations. Base Qwen3-8B + Guard-only has a higher benign refusal rate, reaching approximately 17% on WJB. The experimental results thus indicate a role for generator fine-tuning.

W2 / Q2.

Persistence/CUSUM. These signals characterize the persistence and accumulation of risk. Once harmful content emerges, risk signals may persist; consequently, the accumulated evidence at the first turning point may be lower than during subsequent sustained high-risk phases. This provides an intuitive reason why these feature values may be relatively low at the first transition compared with later high-risk nodes.

The AUC=0.159\mathrm{AUC}=0.159 reported in Table 11 measures the group's signed contribution after feature standardization and weighting by the fitted coefficients, rather than the raw features. Negating this group contribution yields an AUC of 0.841, reversing its ranking direction; algebraically, this is equivalent to reversing the sign of every coefficient in the group. This statistic excludes post-turning nodes and therefore cannot establish a difference in accumulated evidence between the first turning point and subsequent phases. We examine the group's role in localization through the following ablation:

ComparisonExact turning-point matches
Current-level features + first differences193/437
Above features + persistence/CUSUM297/437
Full 16-feature classifier294/437
Full classifier without persistence/CUSUM273/437

Adding this group changes 108 responses from incorrect to correct localization. In 105 of these cases, the classifier without the group had selected a post-turning peak; another four responses change from correct to incorrect localization. This supports the group's role in distinguishing the first turning point from subsequent high-risk phases in the evaluated configurations. We will include this group in Figure 3.

AuthorsOpenReview ↗

Official Comment by Authors Part 2

W2 / Q2.(continued)

Adjacent confirmation. We agree that the original presentation combined feature ranking, parameter selection, and safety after intervention in a way that could cause confusion. We have checked the calculations and added the following analyses. The three original AUCs use the same set of 2,000 development responses but differ in their computation:

AUCComparison set and scoring
0.914Pooled cross-answer comparisons between 437 turning nodes and 15,740 non-turning nodes.
0.577A pooled cross-answer AUC comparing the turning-node scores of 416 eligible jailbreak responses with the response-specific mean scores of their pre-turning nodes. Another 21 responses have no eligible pre-turning segment from which to compute a mean and are therefore excluded.
0.584The same nodes as for 0.914, but with adjacent scores multiplied by the γ\gamma selected in each outer fold.

Thus, 0.914 measures the ranking discrimination between turning and non-turning nodes across all eligible nodes; 0.577 pools turning-node scores and response-specific pre-turning means across answers; and 0.584 pools node scores across folds after weighting them by each fold's selected γ\gamma.

To align the comparison, we retained the original 437 positive turning nodes and restricted the negative set to 1,137 pre-turning nodes from jailbreak responses, obtaining AUC=0.767\mathrm{AUC}=0.767. The mean adjacent score is 2.869 for the pre-turning segments of jailbreak responses and 0.331 for non-jailbreak responses, indicating that the former are harder negatives. This supports the presence of risk information in the adjacent signal, while indicating limitations in its ability to precisely localize turning points on its own.

In a supplementary experiment, we refitted the selector and compared first differences alone with first differences plus the adjacent term for turning/non-turning discrimination over all eligible nodes. Their AUCs were 0.908 and 0.922, respectively, indicating an increase in node-level AUC from adding the adjacent term in this analysis. We therefore do not claim that the adjacent term alone consistently improves jailbreak turning-point localization; these results support its node-level discriminative value within the compared feature combinations.

We assessed practical benefits through control comparisons on the same trajectories, examining whether the benefit of combining adjacent confirmation with rollback arises from better STOP decisions or from reduced harmful-content exposure due to shorter retained responses. A is the base policy without the adjacent term, and C is full TRM. They share the same base classifier weights and are separately calibrated under the same development-set budget. B retains A's STOP decisions and stopping times but rolls back one additional node for every stopped response.

PolicyASR: 7,224 attack responsesBenign refusal rate: 760 responses
A: Base policy6.37% (460)4.74% (36)
B: A + one fixed additional node of rollback4.87% (352)4.87% (37)
C: Full TRM4.04% (292)4.47% (34)

Compared with B, C reduces ASR by 0.83 percentage points and increases jailbreak recall from 76.88% (1,011/1,315) to 80.99% (1,065/1,315). The benefit therefore does not arise solely from shortening responses through rollback. C reduces ASR relative to A under all six attack conditions and relative to B under five, except on Direct Request, where full TRM did not outperform fixed additional rollback (3.00% versus 2.00%). These results support the incremental value of the adjacent feature within the evaluated full policy, particularly through broader coverage of jailbreak responses and a lower benign refusal rate.

AuthorsOpenReview ↗

Official Comment by Authors Part 3

W2 / Q2.(continued)

Regarding parameter selection, the final γ=5\gamma=5 was selected through internal three-fold cross-fitting on all 2,000 development responses. The objective was to maximize attack STOP counts subject to a benign STOP rate of at most 2%, rather than to maximize AUC. The five-fold evaluation in the appendix assesses the training and selection procedure and does not determine the final γ\gamma.

The original outer-validation procedure differs from the final deployment procedure, and its independent threshold-calibration sets contain relatively few benign responses. Each fold's 400 calibration responses include only 84–102 benign responses, so the empirical 2% false-trigger budget permits only one or two benign STOPs after accounting for integer counts. For example, with approximately 80 benign responses, one STOP is permitted, whereas two exceed the budget. This discrete count constraint can make threshold calibration sensitive to a small number of borderline examples. The final procedure instead calibrates on the internally cross-fitted predictions for the full development set, using 450 benign responses and allowing nine STOPs.

To examine the selection of the adjacent-term coefficient and reduce sensitivity to a single STOP, we relaxed the internal benign STOP budget to 10% and performed 20 repeated grouped five-fold evaluations on the same data, for a total of 100 folds, keeping the remaining procedure unchanged. A nonzero γ\gamma was selected in 98% of the folds. This experiment supports sensitivity of parameter selection to the false-trigger budget, and we will add the corresponding clarification to the appendix.

Parameter selection in TRM-Control is a finite-sample constrained statistical optimization problem involving correlated features, and the current sequential procedure does not guarantee a global optimum. We strictly separate the development and test sets and evaluate the frozen parameters on independent test data, providing empirical support for the safety benefits of the evaluated policies. We therefore limit our contribution to the empirical effectiveness of the TRM-Control feature-based approach under the evaluated configurations and datasets; we do not claim globally optimal parameters or gains that are stable across data splits. The reviewer's suggestions also point to opportunities for improving optimization in the presence of complex dependencies among features.

W3.

We agree that the existing static and transfer-attack evaluations do not establish TRM's robustness against an adversary that is aware of the defense. Most monitoring nodes in our system are emitted by the generator, and their positions are not fixed in advance. This may make it more difficult for an adaptive attacker to identify specific tokens to target. One potential attack surface is gradient-based manipulation of internal-signal generation, which could render the fitted controller parameters ineffective. Our approach decouples risk sensing from refusal execution, allowing substantial flexibility in configuring the refusal-control module. Future work could incorporate the frequency of internal safety signals and sustained increases in external Guard scores as detection inputs, and adapt controllers of different capacities to the task.

We thank the reviewer for the constructive feedback, which has helped us improve this work, and welcome any further questions or suggestions.

Reviewer_HmB3OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

TRM is an inference-time jailbreak defense that separates risk sensing from refusal. A generator fine-tuned on R2D data reports SAFE/RETHINK/UNSAFE states instead of refusing. A logistic-regression controller combines these signals with Llama-Guard-3 scores through hand-designed temporal features. When risk is detected, TRM rolls back to an earlier prefix and appends a refusal. On Qwen3-8B and Llama-3.1-8B, it greatly reduces ASR on HarmBench transfer attacks while keeping benign refusal close to the base model.

Critical strengths
  • The problem is well motivated. Moving the refusal decision into a separate controller gives a much better safety and over-refusal trade-off than safety fine-tuning such as R2D.

+The protocol is carefully presented. The controller is frozen before external evaluation, ablations are refitted independently, and the limitations are stated openly.

Critical weaknesses
  • The idea is incremental. The labels and data come from R2D, and the external signal is an off-the-shelf guard. Rollback with refusal already exists in Backtracking and CARE, and the controller is logistic regression on classical change-detection features. The paper does not discuss streaming monitors built for this setting, such as Stream Qwen3Guard, the streaming content monitor of Li et al. (NeurIPS 2025) and SafeSwitch.

  • The external guard seems to produce most of the gain. In Table 4, Guard-only is already close to the full system, while Internal-only is less safe than the base model on several attacks. The claimed benefit of separating risk awareness from refusal is therefore weakly supported. Also, an attacker who evades the guard faces a weaker model than the original.

  • The baselines are not controlled. The harmful-ASR baselines in Table 1, including the base model, are copied from other papers, so they were not produced under TRM’s judge and decoding setup. CARE, Backtracking and a streaming guard are not run. The obvious baseline is also missing: the unmodified base model with Llama-Guard-3 at the same nodes or on the final response.

  • There is no adaptive attack, although the attacked model itself emits the SAFE/UNSAFE tokens, so an attacker can target them together with the guard. Latency is not measured, and the turning-point labels come from model-assisted annotation without reported human validation.

Questions for authors

Please answer the questions raised in my review.

2 comments
AuthorsOpenReview ↗

Official Comment by Authors Part 1

W1

We thank the reviewer for highlighting the connections to prior work. Existing methods already exploit streaming context, adjacent confirmation, or accumulated risk evidence: Qwen3Guard-Stream and SCM focus on external streaming detection; CARE corrects outputs through rollback and regeneration; Backtracking uses [RESET] to retract generated content; and SafeSwitch uses internal states to activate a refusal head. We do not claim the off-the-shelf Guard, logistic regression, or the generic rollback mechanism itself as an original contribution.

TRM contributes a combination of risk-signal learning and trajectory control that models risk sensing and refusal execution separately. As described in §3.1, we build on R2D’s data and safety-label scheme and selectively rewrite refusal passages so that training responses continue reasoning in the presence of risk while explicitly reporting risk states through segment-level safety labels. This training design aims to reduce the coupling between recognizing risk and immediately refusing, providing an independent controller with internal risk signals throughout generation. TRM-Control then combines internal safety probabilities with external Guard observations, jointly modeling risk levels, first differences, smoothed trends, persistence, and CUSUM cumulative evidence. It learns trajectory scores under supervision from the first harmful turning point and incorporates adjacent confirmation to calibrate stopping and rollback decisions under a benign false-trigger constraint. Together, these designs connect risk-state reporting, trajectory scoring, and intervention decisions.

Ablations support the incremental value of this approach in the evaluated configurations. With the same fine-tuned generator, full TRM reduces ASR relative to Guard-only under all six attack conditions while maintaining the same benign refusal rate of 4.47%. The policy ablation of adjacent confirmation also supports its incremental benefit in the evaluated configurations. These results provide empirical support for using combined risk trajectories to guide control decisions.

W2 and W3

We thank the reviewer for this suggestion. We agree that Guard-only already provides strong protection. Nevertheless, with the same fine-tuned generator, full TRM reduces ASR under all six attack conditions while maintaining the same benign refusal rate of 4.47%. Specifically, ASR decreases from 18.00% to 12.66% on TAP-T, from 5.78% to 2.24% on GCG-T, and from 3.55% to 1.60% on Human. These results support the incremental value of combining internal and external safety signals in the evaluated configurations.

We have added a comparison using the original Qwen3-8B backbone with temporal Guard-only monitoring. We regenerated responses to the same 2,000 HybridBench prompts, annotated the responses, and refitted the controller, with one Guard evaluation every 40 tokens. All parameters were frozen before testing. This configuration achieves an ASR of 5.59% (402/7,188) and a benign refusal rate of 8.68% (66/760), compared with 6.01% and 4.47%, respectively, for the previously evaluated fine-tuned generator with Guard-only monitoring. This comparison was not included previously because the original base model does not emit TRM’s internal safety labels and therefore lacks the same label-triggered observation nodes as the fine-tuned model.

MethodAverage ASR ↓\downarrowAverage benign refusal rate ↓\downarrow
Guard-only (fine-tuned generator)6.01%4.47%
Base Qwen3-8B + Guard-only (refitted)5.59%8.68%
TRM4.04%4.47%

These results support the incremental value of integrating internal and external safety signals in the evaluated configurations. Base Qwen3-8B + Guard-only has a higher benign refusal rate, reaching approximately 17% on WJB. The additional comparison shows that temporal Guard-only monitoring can also be applied to the original generator; under the current configuration, it achieves a slightly lower ASR than the fine-tuned generator with Guard-only monitoring, but a higher benign refusal rate.

W3

Regarding Backtracking and CARE: as of this response, we have not located official public implementations of either method and therefore have not included them in direct reproduction comparisons.

We additionally reproduced SCM-7B and Qwen3Guard-Stream-8B using the same test prompts, thinking-mode setting, decoding configuration, and nominal generation-length limits as TRM. Both baselines use the original Qwen3-8B, update risk scores token by token, and truncate responses upon intervention. TRM uses the fine-tuned generator with node-level risk signals and rollback control.

On the common valid attack samples from GCG-T, TAP-T, and DR, SCM, Qwen3Guard-Stream, and TRM achieve ASRs of 5.34%, 12.27%, and 6.01%, respectively. Qwen3Guard-Stream uses a debouncing rule that triggers stopping only after two consecutive tokens are classified as Unsafe. On the 760 common benign responses from XS, OKTest, and WJB, their STOP rates are 19.74%, 5.39%, and 1.18%, respectively. These results show that, under the evaluated configurations, TRM has a lower benign STOP rate, with an ASR close to SCM and lower than Qwen3Guard-Stream. In terms of monitoring frequency, TRM has approximately one observation node per 40 generated tokens on average; its number of external Guard evaluations is therefore approximately 140\frac{1}{40} of the number of score updates performed by the two token-level monitoring baselines.

We thank the reviewer for raising the consistency of comparison conditions. We clarify that a defense’s risk detector and the external judge used to compute ASR serve different functions: the former is part of the method design and may differ across baselines, whereas the latter should follow a consistent evaluation protocol. TRM uses HarmBench-Llama-2-13b-cls to compute ASR. We will explicitly specify and standardize model versions, prompt templates, generation-length limits, and the truncation and text-processing rules for judge inputs.

Meanwhile, the TRM controller ablations in Table 4 use the same fine-tuned generator. Each variant is fitted and calibrated separately under the same development protocol and benign false-trigger budget, and evaluated with the same ASR judge. These comparisons provide direct evidence of the incremental contribution of individual components in the evaluated configurations.

W4

TRM balances jailbreak defense and low benign refusal through inference-time trajectory monitoring and control. Although deployment optimization is not our main contribution, we acknowledge that detection latency, response coherence, and runtime overhead affect practical value. Buffering and its trade-offs among safety, response quality, and user waiting time are also a focus of CARE.

Additional costs include Llama-Guard evaluations of generated prefixes and the observation delay required for adjacent confirmation. Because successive evaluations share substantial input prefixes, reusing the KV cache could reduce repeated prefill computation, provided that the shared token prefixes are identical and the serving implementation supports reuse across calls. CARE also discusses optimizations such as prompt caching. Our cache analysis estimates potential computational savings; it does not establish an implemented or measured end-to-end speedup. For streaming output, a buffering protocol inspired by CARE could withhold the current segment and the preceding segment that remains eligible for rollback under adjacent confirmation, releasing them only once they can no longer be rolled back. This requires implementation-specific evaluation: truncating the final response does not establish a zero-leakage guarantee. Actual streaming exposure, latency, and response coherence remain deployment questions to be evaluated.

We agree that the existing static and transfer-attack evaluations do not establish TRM’s robustness against an adversary aware of the defense. Most monitoring nodes in our system are emitted by the generator, and their positions are not fixed in advance. This may make it more difficult for an adaptive attacker to identify specific tokens to target. One potential attack surface is gradient-based manipulation of internal-signal generation, which could undermine the effectiveness of the fitted controller parameters. Our approach decouples risk sensing from refusal execution, allowing substantial flexibility in configuring the refusal-control module. Future work could incorporate the frequency of internal safety signals and sustained increases in external Guard scores as detection inputs, and adapt controllers of different capacities to the task.

We thank the reviewer for raising the reliability of turning-point annotations. We clarify that turning-point annotation concerns the more than 400 successfully jailbroken responses in HybridBench, and that all turning-point annotations were manually reviewed on a response-by-response basis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.