- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a lightweight supervised fine-tuning approach that trains LLMs to explicitly infer the underlying intent of an instruction before generating a response. The authors construct intention-aware training data using adversarial instructions paired with their corresponding harmful instructions, and fine-tune the model to generate an intent analysis followed by a safe or benign response. Experiments across Llama, Qwen, and GPT-4.1-mini show improved robustness against several jailbreak attacks while largely preserving utility and reducing over-refusal.
Critical strengths
-
The motivation and method are clear, and the visualization is helpful.
-
This paper is well-written, making it easy to follow.
Critical weaknesses
-
Limited methodological novelty and insufficient comparison with closely related work. The central idea of identifying the underlying intent before responding has already been explored by several jailbreak defences, including Intention Analysis and SENTINEL [1]. Moreover, recent training-based safety-reasoning approaches [2-5] similarly train models to perform safety/intent reasoning before response generation. The main distinction of INTENT-FT, therefore, appears to be a relatively direct SFT-based instantiation of intention deduction. Stronger differentiation and direct comparisons with these closely related methods are needed to substantiate the methodological novelty.
-
The supervision may make it unclear whether the model learns general intent understanding. The harmful training labels are generated by a teacher that is explicitly given both the original harmful instruction and its adversarial version. Thus, the model is trained on relatively direct mappings from adversarial prompts to known harmful intents, rather than independently learning to infer intent. The paper would benefit from stronger evaluation on unseen attack constructions and more challenging intent obfuscation.
-
The empirical evaluation could be strengthened in several concrete aspects. First, the open-source evaluation is limited to 7–8B models; experiments on larger and more recent models (e.g., Qwen 3.5 27B, Llama-3.3-70B and MOE gpt-oss-20b) would help establish scalability. Second, evaluation is conducted on only 100 harmful instructions from JailbreakBench; additional benchmarks such as HarmBench and WildJailbreak would provide substantially broader coverage of harmful behaviors and naturally occurring jailbreaks. Third, several strong and closely related training-based defenses are missing, particularly Reasoning-to-Defend and Safety Chain-of-Thought, which are important baselines given that they similarly introduce safety reasoning into model training.
-
The claimed cross-model generalisation is only partially supported. The paper shows that generated intentions can transfer to other vanilla models, but the transfer is notably weaker for Qwen. This raises questions about whether the learned intention representation is genuinely model-independent or is partly tied to the model that generated the reasoning.
-
The inference procedure introduces an additional reasoning step, and its cost is not fully characterised. INTENT-FT requires the model to generate an explicit intention analysis before producing the final response. Although the paper argues that this improves robustness, the additional tokens and latency, as well as the effect of intention-generation errors on downstream responses, deserve more systematic analysis.
[1] Intention Matching Stops Jailbreaks, https://openreview.net/forum?id=vKHCM9O25U
[2] Reasoning-to-defend: Safety-aware reasoning can defend large language models from jailbreaking
[3] Safechain: Safety of language models with long chain-of-thought reasoning capabilities
[4] SafeBehavior: Simulating human-like multistage reasoning to mitigate jailbreak attacks in large language models
[5] CoT defender: Preemptive chain-of-thought occupation for jailbreak attack mitigation
Questions for authors
Refer to weaknesses.