- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the joint optimization problem of reducing over-refusal while preserving safety in safety-aligned LLMs. This paper observes that repeated sampling from the same frozen model on a borderline prompt can produce refusals, unsafe responses, and safe non-refusals, and uses the resulting safe–unsafe responses to construct model-local preference pairs. Based on this observation, this paper proposes RESINJECT, a state-dependent low-rank residual adapter that conditions its correction on the current hidden state and is trained with a reference-adjusted preference objective together with a chosen-response likelihood term. Experiments across three model families show that RESINJECT achieves a balanced trade-off between reducing benign over-refusal and harmful unsafe generation.
Critical strengths
1. Clear and meaningful problem formulation. This paper explicitly considers the joint objective of reducing benign over-refusal and harmful unsafe generation, rather than optimizing either aspect in isolation. This provides a practical perspective on the trade-off between safety and model usability.
2. Novel model-local preference construction. RESINJECT constructs safe–unsafe preference pairs from different responses generated by the same frozen model for the same prompt, without relying on external answers or treating refusal as the preferred target. This provides a simple way to exploit the model’s own stochastic behavior as supervision.
3. Simple and well-motivated method design. RESINJECT introduces a state-dependent low-rank residual correction that conditions the intervention on the current hidden state, allowing different inputs to receive different corrections while keeping the number of trainable parameters relatively small.
Critical weaknesses
1. The contribution of state dependence is not sufficiently validated through ablation. The paper attributes the effectiveness of RESINJECT partly to its state-dependent residual adaptation. However, the representation controls in Table 2 do not provide a fully controlled comparison: the fixed-vector and static-residual baselines contain only 4,096 trainable parameters, whereas RESINJECT contains 65,536. Therefore, it remains unclear whether the performance improvement comes from the increased number of trainable parameters or from state dependence itself.
2. Insufficient safety evaluation. The other primary concern is that the paper’s safety evaluation is limited to direct and overtly malicious queries, which may not adequately reflect more realistic adversarial threats. Recent jailbreak attacks, such as AutoDAN [1] and PAIR [2], are specifically designed to bypass the model’s safety alignment. This issue is particularly important for RESINJECT because its correction depends on the current hidden state and is learned from safe–unsafe response pairs sampled from the frozen model itself. Jailbreak prompts may induce hidden state distributions that differ substantially from those observed during training, potentially weakening the learned state-dependent correction.
3. The discussion of related work is incomplete. In particular, the paper does not sufficiently discuss more recent methods such as AlphaSteer [3] or AdaCD [4]. These approaches similarly introduce adaptive intervention or decoding mechanisms to better balance safety and over-refusal, and are closely related to the paper’s motivation of moving beyond fixed global steering directions. A comparison or clearer discussion of how RESINJECT differs from these recent methods would help better establish the novelty of RESINJECT .
[1] Xiaogeng Liu et al., “AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models”, ICLR 2024.
[2] Patrick Chao et al., “Jailbreaking Black Box Large Language Models in Twenty Queries”, SaTML 2025.
[3] Leheng Sheng et al., “AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint”, ICLR 2026.
[4] Yupeng Qi et al., “Please Refuse to Answer Me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding”, ACL 2026.
Questions for authors
-
The authors should include a parameter-matched state-independent residual baseline to clarify whether the performance improvement comes from state dependence itself or simply from having more trainable parameters.
-
The authors should further evaluate RESINJECT under representative jailbreak attacks, such as AutoDAN and PAIR, to verify whether reducing over-refusal can still preserve safety under jailbreak attack prompts.
-
The authors should consider providing a clearer comparison with recent related methods, such as AlphaSteer or AdaCD, to better clarify the novelty and differences of RESINJECT.