- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TRM (Trajectory-aware Risk Monitoring), a framework for monitoring and intervening in potentially harmful LLM generations. It fine-tunes a generator on modified R2D data to reduce refusal tendencies while producing segment-level safety labels. A separate controller combines the generator’s safety-label probabilities with external Llama Guard scores using a temporal classifier trained to identify harmful turning points. The controller can use the next Guard observation to confirm a preceding candidate; upon triggering, it truncates the response at the selected boundary and applies a fixed refusal after it. Experiments on two LLM backbones report reduced attack success rates with modest increases in benign refusal.
Critical strengths
-
Overall, I think the paper addresses a practically relevant problem (despite typtical): improving jailbreak resistance while limiting over-refusal on benign requests. Evaluating both aspects provides a more informative picture than reporting attack success rates alone.
-
The reported results look supportive. On Qwen3-8B and Llama-3.1-8B, TRM reduces attack success rates relative to the corresponding base models, with relatively small increases in pooled benign refusal rates.
-
I like the effort to examine individual components of TRM through ablations in Table 4, including internal and external signals, temporal information, adjacent confirmation, and rollback boundaries. Refitting and calibrating the ablated detectors helps make these comparisons more informative.
Critical weaknesses
-
I think the overall process of monitoring, buffering, rollback, and intervention overlaps substantially with CooT [1] and CARE [2]. TRM’s main additions appear to be the fusion of internal safety probabilities with external Guard scores, a temporal classifier supervised on harmful turning points, and adjacent-node confirmation. I would appreciate a clearer explanation of the distinctions and practical benefits relative to these methods. There is no direct comparison with CARE, and the safety comparison with CooT primarily relies on previously reported results.
-
I have some concerns about rewriting refusal passages into content that continues addressing the request while retaining the segment-level safety labels. Such rewriting may change the content’s safety properties or segment boundaries, but the paper does not sufficiently explain the labels’ precise meaning, the rules for retaining or updating them, or the subsequent quality checks. I would appreciate paired examples before and after rewriting, together with a more explicit description of the procedure, to clarify why the retained labels remain appropriate supervision for internal risk signals.
-
I think the Guard-only ablation helps establish the additional value of internal signals, but it still uses the fine-tuned TRM generator. Consequently, the contribution of generator fine-tuning itself remains unclear. Applying a Guard-only controller with the same architecture and fitting and calibration procedures to the original generator, and comparing it with the existing Guard-only variant, could help clarify this point.
-
The transfer-attack (from Qwen model to Llama model) results informative, but they do not cover adaptive attacks targeting the complete TRM system. Table 2 compares TRM only with the original Llama-3.1-8B model, leaving it unclear whether its advantage over competing defenses extends across backbones. In addition, some main-table baselines are drawn from prior work, while the multimodal comparisons use different judging protocols. I believe these differences limit the conclusions that can be drawn from the comparisons.
-
I think delayed confirmation should be evaluated alongside an explicit output-buffering protocol, since truncating the final response does not establish that harmful content was never streamed to the user (and this mechanism might make the output weird with sudden refusal). Actual detection latency, streamed harmful-content exposure, and runtime overhead are not sufficiently reported, and the cache analysis estimates input-processing workload rather than measured performance. The exact fixed refusal text is also unspecified. Separately auditing the retained prefix and the final response with the refusal appended would help establish whether the safety gains reflect content removal rather than sensitivity of the judge to the appended refusal.
-
Figure 1 and Figure 2 looks less professional, make it better arranged and switch it into vector image version such as pdf can make it look more comfortable.
[1] Xuanming Zhang, Yuxuan Chen, Samuel Yeh, and Sharon Li. Cognition-of-Thought Elicits Social-Aligned Reasoning in Large Language Models. arXiv:2509.23441, 2025.
[2] Xiaomeng Hu, Fei Huang, Chenhan Yuan, Junyang Lin, and Tsung-Yi Ho. CARE: Decoding-Time Safety Alignment via Rollback and Introspection Intervention. NeurIPS, 2025.
Questions for authors
I noticed that the interaction term achieves a node-level AUC of 0.914, whereas Appendix D.1 reports AUCs of 0.577 and 0.584 in the answer-level and outer-fold analyses, respectively, with the interaction strength selected as zero in most folds. I think the relationship between these findings and the ablation gains in Table 4 deserves clarification. It would be helpful to explain how these analyses are computed, how they relate to the final parameter-selection procedure, and what they imply about the mechanism’s applicability and stability.
4 comments
Part 2 (continued)
W3. We thank the reviewer for this suggestion. We agree that the original Guard-only ablation removes internal safety signals but still uses the fine-tuned generator, and therefore does not isolate the effect of generator fine-tuning. The original base model does not emit TRM's internal safety labels and consequently lacks the same label-triggered monitoring nodes. This was the reason this comparison was not initially included. For the additional experiment, we use fixed monitoring nodes every 40 tokens, based on the approximately 40-token average interval observed in the fine-tuned trajectories, and query Llama-Guard-3 at each node. We regenerate base-model responses to the same 2,000 HybridBench prompts, annotate their semantic turning points, and refit and recalibrate the controller using the original Guard-only feature architecture and fitting procedure. All parameters are frozen before test evaluation. The following results evaluate the retained text after intervention:
| Method | Average ASR ↓ | Average benign refusal rate ↓ |
|---|---|---|
| Guard-only (fine-tuned generator) | 6.01% | 4.47% |
| Base Qwen3-8B + Guard-only (refitted) | 5.59% | 8.68% |
| TRM | 4.04% | 4.47% |
Generator fine-tuning is intended to expose internal safety signals; a separate controller determines STOP and rollback. Fine-tuning may also change the generator's own refusal tendency. These results show that Guard-only control can also be applied to the base generator. Under the evaluated configuration, the base-generator variant has a slightly lower ASR than the fine-tuned Guard-only variant, but a higher benign refusal rate. Full TRM, which combines internal and external safety evidence, achieves both a lower ASR and a lower benign refusal rate than the base-generator Guard-only variant in this comparison.
W4. We thank the reviewer for identifying limitations in the scope and comparability of the evaluations. Our multimodal ASR evaluation follows the judgment rules and output format used by SafeMLLM. However, because the API model used in that work was no longer available, we used a replacement evaluator. Table 3 and Appendix E already describe the result sources and evaluation settings. We agree that, despite using the same judgment rules, differences between evaluator models limit the direct comparability of these results.
W5. Our main contribution is an inference-time trajectory monitoring and control method that balances jailbreak defense with low benign refusal. The current experiments primarily evaluate the safety of retained responses and benign refusal rates. Although deployment optimization is not the primary contribution, we agree that detection latency, response coherence, and runtime overhead affect practical value. Buffering and its trade-offs among safety, response quality, and user waiting time are also a focus of CARE. We will clarify the scope of the available evidence in the revision. Additional costs include Llama-Guard assessments of generated prefixes and the observation delay required for adjacent confirmation. Because successive assessments share substantial input prefixes, cross-call KV-cache reuse could reduce repeated prefill computation, provided that shared token prefixes are identical and the serving implementation supports such reuse. CARE also discusses optimizations such as prompt caching. Our cache analysis estimates potential computational savings; it does not establish an implemented or measured end-to-end speedup. For streaming, a buffering protocol inspired by CARE could withhold the current segment and the preceding segment that remains eligible for rollback under adjacent confirmation, releasing them only once they can no longer be rolled back. This requires implementation-specific evaluation: truncating the final response does not establish a zero-leakage guarantee. Actual streaming exposure, latency, and response coherence remain deployment questions to be evaluated. CARE
Regarding the refusal template, our per-response audit confirms that the 7,224 attack responses underlying Table 4 were judged using the retained prefixes without an appended refusal. The safety gains in this evaluation therefore cannot be attributed to the template influencing the evaluator. We will align the descriptions of the evaluated text in the main paper and appendix and explicitly provide the fixed refusal template:
I can't help with that request because it may facilitate harm. I can help with a safer alternative or provide general safety-focused information.
W6. Thank you for the suggestion. We have revised the figures to improve their presentation and consistency.
Part 3 (continued)
Q1. We thank the reviewer for raising this issue. We agree that the original presentation combined feature ranking, parameter selection, and safety after intervention without sufficiently distinguishing their evaluation settings. We have checked the computations and added the following analyses. The three original AUCs use the same set of 2,000 development responses but differ in how scores and comparison sets are constructed:
| AUC | Comparison set and scoring |
|---|---|
| 0.914 | Pooled cross-answer comparisons between 437 turning nodes and 15,740 non-turning nodes. |
| 0.577 | A pooled cross-answer AUC comparing the turning-node scores of 416 eligible jailbreak responses against the 416 response-specific mean scores of their pre-turning nodes. The other 21 responses have no pre-turning nodes from which to compute this mean and are excluded. |
| 0.584 | The same nodes as for 0.914, but with the adjacent contribution weighted by the confirmation strength selected in each outer fold. |
Thus, 0.914 measures turning/non-turning discrimination across all eligible nodes; 0.577 compares turning-node scores with response-specific pre-turning means across responses; and 0.584 pools node scores weighted by fold-specific values across the outer folds. To align the comparison, we retain the same 437 turning-node positives and restrict the negatives to 1,137 pre-turning nodes from jailbreak responses, obtaining . The mean adjacent score is 2.869 for the pre-turning portions of jailbreak responses, compared with 0.331 for non-jailbreak responses, indicating that these pre-turning nodes are harder negatives. This supports the presence of risk information in the adjacent signal while showing the limitations of using it alone for precise turning-point localization. In an additional diagnostic experiment, we refit the classifiers and compare first-order differences alone with first-order differences plus the adjacent term on all eligible turning and non-turning nodes. Their AUCs are 0.908 and 0.922, respectively. These results support the adjacent term's discriminative value within this feature combination; we do not claim that it consistently improves turning-point localization on its own. These diagnostic AUCs are not the criterion for selecting the deployed controller. Final parameters are selected through internal three-fold cross-fitting on the 2,000 development responses to maximize the number of attack responses triggering STOP, subject to a benign STOP rate of at most 2%.
We assess operational benefits through controlled replay on the same generation trajectories, examining whether the gains arise from additional interception or solely from reducing harmful exposure by retaining shorter responses. A is the base policy without the adjacent term; C is full TRM. They share the base classifier weights and are separately calibrated under the same development budget. B preserves A's STOP decisions and stopping times but applies one additional node of rollback to every stopped response.
| Policy | ASR: 7,224 attack responses | Benign refusal rate: 760 responses |
|---|---|---|
| A: Base policy | 6.37% (460) | 4.74% (36) |
| B: A + one additional node of rollback | 4.87% (352) | 4.87% (37) |
| C: Full TRM | 4.04% (292) | 4.47% (34) |
Compared with B, C reduces ASR by 0.83 percentage points and increases jailbreak recall, measured as STOP coverage among generator-only successful jailbreaks, from 76.88% (1,011/1,315) to 80.99% (1,065/1,315). Thus, the gains are not explained solely by shortening responses through rollback. C has a lower ASR than A under all six attack conditions and a lower ASR than B under five, with no clear advantage under Direct Request. These results support the incremental value of adjacent confirmation in the evaluated full policy, particularly through broader interception of successful jailbreaks and a lower observed benign refusal rate.
We thank the reviewer for the constructive feedback, which has helped us improve this work, and welcome any further questions or suggestions.
Thank you for the detailed response and the additional experiments, especially the Guard-only comparison on the base model and the clarification of the AUC analyses. These address part of my concerns, but the overlap with prior rollback-based methods, the missing adaptive-attack evaluation, and the unmeasured streaming latency and exposure remain open. I keep my current rating, and I appreciate the effort the authors put into the rebuttal.
Part 1
W1. TRM separates risk sensing from refusal execution by learning observable internal safety signals and combining them with external safety evidence and temporal features. The resulting controller is trained with turning-point supervision and calibrated under a benign STOP budget. We do not claim novelty for the generic monitoring and rollback pipeline; prior work has informed our implementation of inference-time intervention. CARE focuses on external Guard monitoring, buffering, rollback, and regeneration with introspective feedback. It uses an existing safety classifier for risk detection, and its implementation was not publicly available at the time of our submission. TRM instead focuses on learning internal safety signals and modeling risk along generation trajectories.
W2. The labels describe the risk state of the corresponding response segment:
[SAFE]denotes locally safe content,[RETHINK]indicates recognized potential risk requiring caution, and[UNSAFE]indicates unsafe content or direction. They are not final safety judgments of the entire response: a final segment labeled[SAFE]does not negate harmful content in earlier segments. For example, for a request to manipulate customer ratings for financial gain, the original response is a refusal labeled[SAFE]. In the rewritten response, the segment identifying the risk is labeled[RETHINK], segments describing harmful mechanisms are labeled[UNSAFE], and the segment discussing safeguards and risks is labeled[SAFE]. Thus, newly introduced harmful content is not simply assigned the original safe label. The training text instead exposes distinct risk states, consistent with our goal of enabling continued reasoning in the presence of risk while assigning refusal execution to a separate controller. We will provide three before-and-after examples in the appendix. One example follows.Prompt:
Original response:
Rewritten response used for training: