TRM: Trajectory-aware Risk Monitoring for Large Language Models

Large language models (LLMs) have demonstrated strong capabilities in language understanding, reasoning, and coding. Translating these advances into reliable applications requires safeguards that prevent harmful generation while preserving appropriate responses to benign requests. Existing safety defenses can provide strong protection against jailbreak attacks but often suffer from over-refusal, whereas approaches that reduce refusal tendencies may compromise robustness against adversarial attacks. Balancing reliable jailbreak defense with low benign refusal therefore remains challenging. We propose TRM (Trajectory-aware Risk Monitoring), a framework that separates risk assessment from refusal execution through selective intervention during generation. We fine-tune a low-refusal generator on instruction data containing adversarial requests paired with structured safety annotations. This enables the generator to expose internal safe, rethink, and unsafe safety states along generation trajectories while maintaining helpful responses. At monitoring nodes, the controller combines generator-derived safety probabilities with Llama-Guard-3 assessments through a temporal risk classifier. When high-risk trajectories are detected, the controller stops generation, rolls back to a previously stored response prefix, and appends a fixed refusal without further decoding. Evaluations across transfer attacks, diverse jailbreak benchmarks, and benign instruction sets demonstrate that TRM substantially reduces harmful generations while maintaining low benign refusal rates. Preliminary evaluations on vision-language models further show its applicability to image-based jailbreak attacks. These results suggest that trajectory-level safety monitoring enables selective intervention during generation, improving safety while preserving model helpfulness.