acceptodds
Under review as a conference paper at ICLR 2027

ImmuneCoT: Learning Reasoning Immunity via Privileged On-Policy Self-Distillation

Abstract

Chain-of-thought (CoT) reasoning helps large language models (LLMs) solve complex problems, yet unsafe intermediate steps can steer subsequent reasoning toward harmful assistance. Learning to recover from such drift is therefore an important component of reasoning safety. We introduce ImmuneCoT, an immune-inspired on-policy self-distillation (OPSD) framework that trains recovery through controlled exposure to corrupted reasoning prefixes. A frozen teacher initialized from the same base model provides two complementary supervision signals: Recognition conditions on the infected prefix to elicit drift-aware correction, whereas Response bypasses that prefix and anchors behavior to the original request. A Base-adjusted product of experts fuses these signals into token-level targets for the student's on-policy continuations, enabling recovery without privileged triggers at inference. Auxiliary benign examples provide self-tolerance training to limit indiscriminate refusal. We evaluate Qwen3-1.7B and Qwen3-4B on standard safety, over-refusal, and general-capability benchmarks, and assess Qwen3-4B under H-CoT mid-reasoning hijacking. Among cases with safe answers before injection, ImmuneCoT achieves a safe recovery rate of 55.5%; on the common Clean-safe subset shared with STAR-1, its paired advantage is 26.1 percentage points. On Qwen3-4B, it also reduces HarmBench attack success from 30.9% to 6.2%, keeps over-refusal below 4% on both benign benchmarks, and limits the maximum observed capability decrease to 0.4 percentage points relative to Base. These findings support learning recovery from corrupted reasoning states while retaining helpfulness and general capability. Code, data, and model resources are available at https://anonymous.4open.science/r/ImmuneCoT-3DE0/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Chain-of-Thought Safety, On-Policy Self-Distillation, Privileged Context, Reasoning Recovery

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_bDvDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors study how a reasoning model can recover after harmful reasoning has entered its chain of thought. For that during training, they insert harmful and persuasive drift into the model’s own reasoning and train it to continue safely, using a frozen copy of the model as the teacher. The teacher judges each continuation twice: once after a note saying the reasoning was manipulated (Recognition), and once with the corrupted reasoning removed and a note stating the right final behavior (Response). On Qwen3-4B, 55.5% of safe responses stay safe after H-CoT hijacking (vs. at most 30.2% for baselines), HarmBench attack success falls from 30.9% to 6.2%, and over-refusal stays below 4%.

Critical strengths

Prior work has shown that safety alignment often depends on the first few output tokens [1]. Attack such as H-CoT [2] corrupt the reasoning midway, and post-training approaches that start from clean reasoning, such as STAR-1 [3], fail against them. The proposed method ImmuneCoT, trains the model directly on corrupted reasoning so that it learns to recover after the reasoning has gone wrong. This targets a gap that current safety training does not cover.

[1] Safety Alignment Should Be Made More Than Just a Few Tokens Deep, 2025. [2] H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism, 2025. [3] STAR-1: Safer Alignment of Reasoning LLMs with 1K Data, 2026.

Critical weaknesses

The ablation results does not clearly show that combining the two signals is what helps. Specifically, the authors hypothesize that Recognition and Response are complementary. However, combining them without the adjustment against the corrupted-context score gives 27.5% HarmBench ASR, worse than Response alone (20.9%). The authors should test Response alone with the same adjustment.

Questions for authors
  1. How does a Response-only variant perform when it uses the same adjustment against the corrupted-context score as the full method?
  2. Just curious, does recovery hold when the harmful drift comes from the model itself, as in self-jailbreak [1], rather than being inserted?

[1] Mao, Y., Zhang, C., Wang, J., Guan, X., Cao, B., Lu, Y., Lin, H., Han, X. and Sun, L., 2026, July. When models outthink their safety: Unveiling and mitigating self-jailbreak in large reasoning models. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 22274-22302).

2 comments

Dear Reviewer bDvD,

Thank you for your careful review of this paper and for identifying the key comparison missing from our validation of the complementarity of the two signals. Your question about harmful drift produced by the model itself also prompted us to extend the recovery experiments to naturally generated drift prefixes.


W1&Q1: Response-Only with Explicit Base Adjustment

Thank you for requesting this control. Response-only with inverse-Base weighting yields 28.4% HarmBench ASR versus ImmuneCoT's 6.2%; Base division alone does not reproduce the gains.

Here, N\mathcal N normalizes over vocabulary, qBq_B is the floored corrupted-context Base score, and q~rec,q~res\tilde q_{\mathrm{rec}},\tilde q_{\mathrm{res}} are softly anchored scores:

P_{\mathrm{full}}&=\mathcal N[\tilde q_{\mathrm{rec}}\tilde q_{\mathrm{res}}/q_B],\\ P_{\mathrm{PoE\ w/o\ Base}}&=\mathcal N[\tilde q_{\mathrm{rec}}\tilde q_{\mathrm{res}}],\\ P_{\mathrm{Response\text{-}only}}&=\mathcal N[\tilde q_{\mathrm{res}}],\\ P_{\mathrm{Response+explicit\ Base}}^{\mathrm{ideal}} &=\mathcal N[\tilde q_{\mathrm{res}}/q_B]. \end{aligned}$$ Original Response-only sets the Recognition weight to zero, giving $\tilde q_{\mathrm{rec}}=q_B$ and cancelling the denominator. The new control omits this factor; its normalized score uses $\min\{\log\tilde q_{\mathrm{res}}-\log q_B,5\}$ and per-token loss clip 10. | Method | HarmBench ASR ↓ | WJ-H ASR ↓ | [email protected] ↓ | | --- | --- | --- | --- | | Response-only | 20.9% | 65.1% | 15.7% | | PoE w/o Base adjustment | 27.5% | 77.3% | 15.3% | | Response-only + explicit Base adjustment (new) | 28.4% | 73.8% | 24.0% | | **ImmuneCoT** | **6.2%** | **47.8%** | **0.6%** | **The full target outperforms both single-signal targets and unadjusted PoE on all three metrics.** This supports the complete fusion design, rather than unadjusted combination or inverse-Base weighting alone. The stabilized ratio control is not a strictly matched one-factor comparison. **Revision note:** We add this control, the formulas, and numerical settings. Thank you for helping us clarify the source of the gains. ### Q2: Recovery from Model-Generated Harmful Drift Thank you for this question. It prompted us to test whether learned recovery also holds when the harmful reasoning originates from the model itself. **ImmuneCoT achieves 35.72% SRR (514/1,439), with 176 explicit R³ safe outcomes (12.23%), the highest R³ rate among the ten methods.** Using RQ3's paired settings and safety threshold, we test the 500 longest drift prefixes from 572 automatically screened Base trajectories on WildJailbreak, without rewriting. Ten models receive identical requests/prefixes, with five continuations and paired Clean outputs each (2,500 per condition). Missing scores and empty final answers are excluded. SRR = S→S/Clean-safe; S→S means safe under both conditions. DeepSeek-v4-pro applies the frozen RQ3 stage prompt, withholding model identities and final answers. Ordered Rollback requires Recognition followed by explicit withdrawal; R³ additionally requires final safety. All 8,811 eligible continued-reasoning trajectories received annotations with validated exact evidence spans and ordering. | Method | Valid pairs | Clean-safe | S→S | Recognition | Ordered rollback | Explicit R³ safe | SRR ↑ | | --- | --- | --- | --- | --- | --- | --- | --- | | Base | 2486 | 176 | 38 | 10 | 5 | 4 | 21.59% | | SafeChain | 2474 | 805 | 423 | 71 | 41 | 33 | 52.55% | | UnsafeChain | 2483 | 1052 | 116 | 38 | 25 | 9 | 11.03% | | STAR-1 | 2495 | 1668 | 181 | 123 | 85 | 33 | 10.85% | | AdvChain | 2447 | 1340 | 87 | 553 | 277 | 37 | 6.49% | | SFT | 2491 | 512 | 96 | 85 | 67 | 27 | 18.75% | | DPO | 2471 | 536 | 101 | 78 | 65 | 29 | 18.84% | | OPSD | 2497 | 1048 | 272 | 216 | 171 | 106 | 25.95% | | GRPO | 2486 | 236 | 59 | 19 | 11 | 6 | 25.00% | | **ImmuneCoT** | **2496** | **1439** | **514** | **275** | **216** | **176** | **35.72%** | Models have different Clean-safe subsets; on common pairs, ImmuneCoT exceeds STAR-1 and OPSD by 25.65 and 17.84 points. OPSD and SafeChain reach 10.11% and 4.10% R³, respectively. SafeChain has higher SRR (52.55%); UnsafeChain reaches 11.03%. On 619 common Clean-safe pairs, SafeChain also leads: 59.61% versus 39.10%. Both train on WildJailbreak-derived data; UnsafeChain additionally learns corrected safe CoT and answers for unsafe Base cases. **Under shared harmful prefixes generated by Base itself, ImmuneCoT improves safe recovery and exhibits complete visible R³ trajectories.** **Revision note:** Appendix B.6 adds prefix construction, settings, recovery results, and process analysis; RQ3 retains H-CoT and references this supplement. Thank you for motivating this additional test. --- Thank you again for your in-depth analysis and suggestions. If the added control and supplementary experiment adequately address your concerns, we sincerely hope you will consider re-evaluating this paper.
AuthorsOpenReview ↗

Follow-up to Reviewer bDvD

Dear Reviewer bDvD,

Thank you again for suggesting the Response-only control with explicit Base adjustment and the evaluation on model-generated drift. With the discussion period closing soon, we would appreciate hearing whether these two additions address your questions. If anything remains unclear about the control or the recovery results, we would be happy to clarify.

Reviewer_ESsDOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ImmuneCoT, an on-policy self-distillation framework for learning safe recovery from corrupted chain-of-thought prefixes. A frozen teacher initialized from the same base model provides two complementary supervision signals: Recognition uses the corrupted prefix to prompt reassessment, while Response bypasses it and uses the training label to guide refusal or assistance. A Base-adjusted product of experts combines these signals into token-level targets for the student's own continuations, allowing recovery without privileged prompts at inference. Benign over-caution examples help limit indiscriminate refusal. Experiments on Qwen3 models suggest improved safety and recovery under reasoning hijacking while largely preserving general capabilities.

Critical strengths
  1. The paper addresses an important problem in reasoning safety. Misleading intermediate steps can undermine an otherwise safe reasoning process and influence subsequent generation. Learning to recover from such corrupted states is therefore a relevant goal for improving the robustness of long-form reasoning models.
  2. The method provides a meaningful extension to recovery-oriented training. Compared with AdvChain's offline supervised learning on constructed correction trajectories, ImmuneCoT distills recovery guidance on continuations sampled from the current student. Its separation of drift-aware Recognition and prefix-independent Response, together with a Base-adjusted product of experts, offers a concrete way to combine complementary teacher views into token-level supervision.
  3. The method achieves a favorable balance between safety and general capability. It improves safety over the base models while largely preserving performance on mathematics, code generation, and instruction-following benchmarks. These results suggest that the safety gains come with limited loss of general capability in the reported evaluations.
Critical weaknesses
  1. W1. Most successful recovery cases involve immediate termination of the corrupted reasoning. Table 5 shows that 80 of the 106 safe infected outcomes are achieved by immediately closing the thinking block, while only 8 exhibit explicit Recognition–Rollback–Recovery. Immediate containment is a useful defense and a valid success under the paper's definition. However, the results provide more extensive evidence for safe termination than for recognizing and correcting an erroneous reasoning trajectory. The extent to which the method improves correction within continued reasoning therefore remains less established.
  2. W2. The contribution of the proposed supervision design to recovery remains unclear. The component ablations in Figure 4 cover standard benchmarks but do not report H-CoT recovery rates or stage decompositions. In addition, Response uses a direct-response prefill, whereas Recognition and the student continue from an unclosed thinking block (Appendix D). This leaves the effects of the teacher-view decomposition and Base adjustment entangled with the change in response format. Evaluating the existing ablations on H-CoT and adding a matched-thinking-format Response control would help isolate the sources of the recovery gains.
  3. W3. Generality across model families and scales remains insufficiently validated. The evaluation covers Qwen3-1.7B and Qwen3-4B, with H-CoT recovery tested only on the 4B model. Although two model sizes are included, the method's applicability to other reasoning-model families and larger scales remains unclear. Additional evaluations on other families, such as DeepSeek-R1 or Kimi, and a wider range of model sizes would strengthen the generality of the findings.
Questions for authors
  1. Given that most safe outcomes come from immediately closing the thinking block, to what extent does ImmuneCoT improve error recognition and correction when reasoning continues after injection?
  2. Could the authors report H-CoT recovery rates and stage decompositions for the existing component ablations, and include a Response control with a matched thinking format to distinguish the effects of the supervision design from those of response formatting?
  3. Could the authors evaluate other reasoning-model families, such as DeepSeek-R1 or Kimi, and a wider range of model sizes, including larger models, to assess whether the safety, recovery, and capability-retention gains generalize across families and scales?
2 comments

Dear Reviewer ESsD,

We sincerely thank you for your careful review of this paper and for recognizing the significance of the problem of recovery from corrupted reasoning as well as the safety–utility balance of our method. Your suggestions on the recovery process, the supervision design, and cross-family generalization helped us identify the key questions that required further validation. We respond to each point below.


W1&Q1: Continued Correction and Immediate Containment

Thank you for examining the recovery process closely. Among the 37 Clean-safe cases with continued reasoning, 14 show explicit recognition, 11 show ordered rollback, and 8 satisfy the full R³ criterion (21.6%).

Our contribution is learning to recover safe behavior from corrupted reasoning states. R³ requires explicit, ordered recognition and rollback followed by a safe answer; this strict process diagnostic is not required for every recovery. When the request reveals harmful intent, terminating corrupted reasoning and responding safely is appropriate, supported by the prefix-independent Response view.

We automatically assessed explicit risk recognition and safe refusal in nine comparison models' matched Clean outputs:

Clean risk recognition and safe refusalImmediate containmentContinued reasoning
At least one comparison model63/80 (78.8%)18/37 (48.6%)
At least two comparison models40/80 (50.0%)9/37 (24.3%)

These associations are consistent with more identifiable request risk in immediate-containment cases.

W2&Q2: Component and Matched-Format Controls

Thank you for suggesting these controls. We added component and format-matched H-CoT recovery rates and stage decompositions:

(1) Component ablations. Under the same H-CoT setting, ImmuneCoT reaches 55.5% SRR versus 31.8%, 26.9%, and 15.2% for the three ablations:

MethodClean-safeImmediate safeContinuedRecognitionOrdered rollbackExplicit R³ safeOther safeTotal safeSRR ↑
Recognition-only22372104731.8%
Response-only1041722321102826.9%
PoE w/o Base adjustment33160004515.2%
ImmuneCoT1918037141181810655.5%

(2) Matched thinking format. Results:

MethodHB ASR ↓Clean-safeTotal safeSRR ↑ContinuedRecognitionOrdered rollbackExplicit R³ safe
Response-only (original format)20.9%1042826.9%22321
Response-format matched16.6%1697242.60%33865
ImmuneCoT6.2%19110655.5%3714118

Format matching raises Response-only SRR from 26.9% to 42.60%. On 156 common Clean-safe pairs, ImmuneCoT reaches 59.62% recovery versus 44.23% for the matched checkpoint (+15.38 points). HarmBench ASR is 6.2% versus 16.6%. Endpoint labels use different judge versions.

Revision note: We add these results and detail teacher prefills in Appendix D.

W3&Q3: Cross-Family Validation at 8B Scale

Thank you for suggesting this extension. We added DeepSeek-R1-Distill-Llama-8B, extending beyond Qwen3 to an R1-distilled Llama backbone at 8B scale. We report safety, over-refusal, and general capability:

方法HB ↓WJ-H ↓SR ↓XS-H ↓XS-B ↓WJ-B ↓MATH ↑HE ↑IFEval ↑
Base54.1%80.1%48.6%97.0%0.0%0.5%84.6%79.9%58.4%
SafeChain40.9%71.0%35.1%95.0%0.0%0.5%81.8%80.5%58.4%
UnsafeChain40.6%70.0%37.4%89.0%5.2%0.5%81.8%80.5%55.8%
STAR-11.2%18.1%0.6%22.0%39.2%20.0%82.8%75.6%44.0%
AdvChain4.7%34.4%0.6%17.5%24.8%3.8%85.2%82.9%53.0%
SFT35.3%64.0%25.9%85.0%0.4%1.9%87.2%82.9%54.0%
DPO42.5%72.9%29.1%92.0%0.8%1.0%86.4%80.5%58.6%
OPSD40.6%71.0%31.3%94.0%0.0%0.5%84.6%80.5%44.7%
GRPO51.6%81.0%54.0%96.0%0.0%0.5%84.6%82.9%57.3%
ImmuneCoT8.8%35.5%7.3%38.5%2.0%1.0%86.0%84.1%52.5%

Revision note: We add the 8B setup and results alongside Qwen3-1.7B/4B.


Thank you again for your detailed suggestions. They helped us explain the form of recovery, the role of each component, and the scope of generalization more clearly. If the added evidence and the clarifications above address your main concerns, we sincerely hope you will consider re-evaluating this paper.

AuthorsOpenReview ↗

Follow-up to Reviewer ESsD

Dear Reviewer ESsD,

Thank you again for suggesting the H-CoT ablations and the cross-family evaluation. With the discussion period closing soon, we would appreciate any feedback on whether the new recovery comparisons, including the Response-format control, and the 8B results address your main concerns. If any point needs clarification, we would be happy to respond before the discussion closes.

Comments on the paper

4 comments

Dear Reviewer 9qR9,

We sincerely thank you for your specific suggestions on recovery evidence, evaluation scale, label reliability, computational cost, and training-data construction. These comments helped us further separate the questions that need clarification from the experimental evidence that needs to be added. We respond to each point below.


W1: Safe Behavioral Recovery and Explicit Reasoning Correction

Thank you for examining the recovery evidence. Our contribution is learning to recover safe behavior from corrupted reasoning states; immediate containment and continued correction are both recovery pathways. Of the 106 safe outcomes, 80 involve immediate containment. Among the 37 Clean-safe cases with continued reasoning, 14 show explicit recognition, 11 show ordered rollback, and 8 satisfy the full R³ criterion (21.6%).

R³ requires explicit, ordered recognition and rollback followed by a safe answer; this strict process diagnostic is not required for every recovery. When the request reveals harmful intent, safe termination is appropriate, supported by the prefix-independent Response view.

We automatically assessed explicit risk recognition and safe refusal in nine comparison models' Clean outputs, matching requests and sampling indices:

Clean risk recognition and safe refusalImmediate containmentContinued reasoning
At least one comparison model63/80 (78.8%)18/37 (48.6%)
At least two comparison models40/80 (50.0%)9/37 (24.3%)

These associations across different request groups are consistent with more identifiable original risk. We distinguish overall safe recovery from explicit reasoning correction.

W2: Full H-CoT Coverage and an Additional 500-Prefix Evaluation

Thank you for raising the question of evaluation coverage. We used the complete official H-CoT set: 50 request–prefix pairs spanning 10 risk categories. Each request has five generations; cluster bootstrap treats requests as independent units.

We additionally evaluated 500 Base-generated harmful drift prefixes, extending both the scale and the source of corruption. We selected the 500 longest prefixes from 572 automatically screened WildJailbreak trajectories, without rewriting. Following RQ3, all models receive the same prefixes, with five continuations and paired Clean outputs per request (2,500 per condition).

ImmuneCoT achieves 35.72% SRR (514/1,439), versus 21.59% for Base (38/176), a descriptive 14.13-point difference. SafeChain achieves 52.55% SRR. ImmuneCoT's explicit R³ rate is 176/1,439 (12.23%), the highest observed among the ten models. Denominators use each model's valid Clean-safe pairs, excluding missing scores and empty answers. All 8,811 eligible continued-reasoning trajectories received annotations, extending the official H-CoT evaluation to natural drift.

Revision note: Appendix B.6 adds the experiment, construction, settings, and results; RQ3 retains H-CoT and references this supplement. Thank you for motivating this broader evaluation.

W3&Q1: Recovery Evaluation and Adaptive-Attack Robustness

Thank you for suggesting this extension. Our experiment measures recovery from a shared corrupted reasoning state; adaptive attacks test robustness to prefixes re-optimized against each trained model. Shared prefixes control corruption across models; adaptive optimization changes that state and requires a separate attack budget and search protocol.

The results establish recovery under shared corruption. We clarify this scope; model-specific adaptive attacks were not evaluated.

W4: Improving Safety While Preserving Utility

Thank you for comparing the baselines. We evaluate safety, recovery, benign helpfulness, and general capability jointly. STAR-1 achieves lower ASR on some benchmarks and 69.1% IFEval, compared with Base's 80.8%. ImmuneCoT reduces HarmBench ASR from 30.9% to 6.2% while retaining 80.4% IFEval and 91.5% HumanEval.

MethodHB ↓WJ-H ↓SR ↓XS-H ↓XS-B ↓WJ-B ↓MATH ↑HE ↑IFEval ↑
Base30.9%81.1%21.4%60.5%0.0%0.0%92.2%90.2%80.8%
STAR-12.8%31.3%5.4%20.0%19.6%2.4%94.0%83.5%69.1%
ImmuneCoT w/o benign0.3%13.9%0.0%19.0%17.6%11.0%92.6%90.2%83.4%
ImmuneCoT6.2%47.8%0.6%24.5%3.6%1.0%93.8%91.5%80.4%

Removing benign examples raises refusal from 3.6% to 17.6% on XSTest and from 1.0% to 11.0% on WildJailbreak. Retaining benign examples preserves helpfulness alongside the full method's safety and recovery gains. We clarify this joint evaluation objective.

W5: Reliability of Refusal Evaluation

Thank you for raising this concern. Refusal-phrase matching follows prior evaluation protocols: XSTest describes it [1]; OR-Bench (ICML 2025) uses it on all 80K benign prompts [2]; AdvChain uses it on XSTest and WildJailbreak benign subsets [3].

We note errors from varied refusal wording and mixed responses. WildJailbreak-Harmful ASR uses the HarmBench classifier, independently of refusal matching.

[1] XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models

[2] OR-Bench: An Over-Refusal Benchmark for Large Language Models

[3] AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models

W6: Added Human Validation of Recovery Labels

Thank you for suggesting this validation. Two annotators independently scored 100 trajectories: 60 continued-reasoning and 40 immediate-containment. Agreement:

Judgment itemCohen's κ between annotators
Final safety0.979
Recognition (3-level)0.725
Rollback (3-level)0.897

Adjudicated human consensus matches the original safety labels on all 100 trajectories. This validates audited safety labels; κ measures process-label agreement.

Revision note: We report sampling, annotation, adjudication, and agreement.

W7: Added Data-Construction and Training Costs

Thank you for requesting this breakdown. Data construction costs US$6.40 for 1,790 valid samples; ImmuneCoT training takes 2.57 hours (10.26 GPU-hours). We measure DeepSeek-R1-Distill-Llama-8B on 4 A800 80GB GPUs for 2 epochs, using recorded off-peak DeepSeek-V4-Pro API rates.

ItemCost / Output
Data-construction calls and token usage4,190 calls; 7.80M tokens
Data-construction API costUS$6.40
Valid constructed samples1,790
ImmuneCoT (2 epochs)2.57 h / 10.26 GPU-hours
OPSD (2 epochs)2.41 h / 9.65 GPU-hours
SFT (2 epochs)1.44 h / 5.76 GPU-hours

ImmuneCoT training takes approximately 1.07× OPSD's time and 1.78× SFT's.

Revision note: Appendix F separates API construction costs from training GPU-hours, with usage, pricing, and hardware.

W8: Benign-Data Ablation and Injection-Position Selection

Thank you for suggesting these controls. Our benign-removal ablation raises over-refusal from 3.6% to 17.6% on XSTest and 1.0% to 11.0% on WildJailbreak. Harmful data, injection positions, and supervision are unchanged.

MethodHB ↓WJ-H ↓SR ↓XS-H ↓XS-B ↓WJ-B ↓MATH ↑HE ↑IFEval ↑
ImmuneCoT w/o benign0.3%13.9%0.0%19.0%17.6%11.0%92.6%90.2%83.4%
ImmuneCoT6.2%47.8%0.6%24.5%3.6%1.0%93.8%91.5%80.4%

Lower ASR but higher benign refusal isolates the contribution of self-tolerance to helpfulness.

Supervision methods share injection positions. The Vulnerability Sniffer selects susceptible Base steps; the Attack Agent constructs contextual drift. SFT, DPO, OPSD, component ablations, and ImmuneCoT share these samples and positions at each scale.

Random positions would test selection separately; this remains unevaluated, with no claim of superiority.

Revision note: RQ4 adds this ablation and shared-position construction.

Q2: Cross-Family Validation at 8B Scale

Thank you for suggesting this extension. We added DeepSeek-R1-Distill-Llama-8B: an R1-distilled Llama backbone at 8B scale. Results cover safety, over-refusal, and capability:

方法HB ↓WJ-H ↓SR ↓XS-H ↓XS-B ↓WJ-B ↓MATH ↑HE ↑IFEval ↑
Base54.1%80.1%48.6%97.0%0.0%0.5%84.6%79.9%58.4%
SafeChain40.9%71.0%35.1%95.0%0.0%0.5%81.8%80.5%58.4%
UnsafeChain40.6%70.0%37.4%89.0%5.2%0.5%81.8%80.5%55.8%
STAR-11.2%18.1%0.6%22.0%39.2%20.0%82.8%75.6%44.0%
AdvChain4.7%34.4%0.6%17.5%24.8%3.8%85.2%82.9%53.0%
SFT35.3%64.0%25.9%85.0%0.4%1.9%87.2%82.9%54.0%
DPO42.5%72.9%29.1%92.0%0.8%1.0%86.4%80.5%58.6%
OPSD40.6%71.0%31.3%93.5%0.0%0.5%84.6%80.5%44.7%
GRPO51.6%81.0%54.0%96.0%0.0%0.5%84.6%82.9%57.3%
ImmuneCoT8.8%35.5%7.3%38.5%2.0%1.0%86.0%84.1%52.5%

Revision note: We add 8B settings and results.


Thank you again for taking the time to provide these specific suggestions. The added experiments and clearer reporting will help readers judge the role of the supervision design, the strength of the recovery evidence, and its applicable scope. If these responses adequately address your main concerns, we sincerely hope you will consider re-evaluating this paper.

AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank the reviewers for their careful reading and constructive feedback. We have responded to each comment and summarize the main additions and clarifications below. New material in the revised manuscript is marked in blue.

  • Supervision design and recovery mechanisms (ESsD, bDvD, 9qR9). We evaluated the three existing ImmuneCoT ablations on H-CoT and added a Response-format control and a Response-only control with explicit Base adjustment. The format control achieved a 42.6% safe recovery rate (SRR), compared with 55.5% for the full method. Explicit Base adjustment alone did not reproduce the full method’s safety gains. In our individual replies, we also explain how both immediate containment and continued reasoning can yield safe recovery, and why explicit R³ is a stricter measure of visible correction.

  • Evaluation coverage and scope (bDvD, 9qR9). The original evaluation uses the full official H-CoT test set, with five generations per request to assess variability across samples. We added an evaluation with 500 unedited harmful reasoning prefixes generated by the base model. ImmuneCoT achieved 35.72% SRR and the highest explicit R³ rate among the ten methods (12.23%). SafeChain achieved a higher overall SRR. Our individual replies explain how evaluation under shared corruption differs from adaptive attacks optimized against each trained model.

  • Cross-family generalization and the safety–utility trade-off (ESsD, 9qR9). We added DeepSeek-R1-Distill-Llama-8B results for ImmuneCoT and all baselines. ImmuneCoT reduced HarmBench ASR from 54.1% to 8.8%, with over-refusal rates of 2.0% and 1.0% on the two benign benchmarks. Math and coding performance improved, while instruction-following performance declined. We also discuss STAR-1’s stronger results on some safety benchmarks alongside its higher over-refusal and larger losses on some capability benchmarks.

  • Evaluation reliability (9qR9). Two human annotators independently labeled 100 trajectories. Inter-annotator Cohen’s κ was 0.979 for final-answer safety, 0.725 for Recognition, and 0.897 for Rollback. The adjudicated human consensus matched the original safety labels on all 100 sampled trajectories. Our individual reply also explains the basis and limitations of refusal-phrase matching.

  • Training data and computational cost (9qR9). Removing benign training examples increased over-refusal from 3.6% to 17.6% on XSTest and from 1.0% to 11.0% on WildJailbreak, showing how these examples help preserve helpfulness on benign requests. We also report data construction costs and training costs for ImmuneCoT, OPSD, and SFT. As clarified in our individual reply, the supervised comparisons use shared injection positions, and the effect of position selection remains to be evaluated separately.

Thank you again for helping us improve the paper. As the discussion period draws to a close, we would appreciate any feedback on whether these additions address your main concerns. If any questions remain, we would be grateful for the opportunity to clarify them before the discussion closes.

AuthorsOpenReview ↗

Follow-up to Reviewer 9qR9

Dear Reviewer 9qR9,

Thank you again for your detailed feedback. Our replies include a broader recovery evaluation, independent human validation of the recovery labels, and a benign-data ablation. With the discussion period closing soon, we would appreciate any feedback on whether these additions address your main concerns and whether any point would benefit from further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.