- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a safety mid-training method that aims to learn safety as a foundational capability of language models rather than relying solely on safety alignment at the post-training stage. Specifically, the method uses a base model to generate structured knowledge documents and reasoning traces, which, after quality filtering, are used for continual pre-training. Results show that the method improves the trade-off between safety and general capability and maintains relatively good safety performance in subsequent training that contains no safety data.
Critical strengths
The paper investigates a practically meaningful problem: how to keep safety behavior stable during subsequent training while improving general capabilities. The experimental evaluation is fairly comprehensive, covering not only different model scales and multiple safety benchmarks but also different safety data ratios, training stages, and the effects of subsequent SFT and reinforcement learning.
Critical weaknesses
The main problem is that the experimental results are not yet sufficient to support the core mechanistic claim the authors put forward. The authors argue that safety reasoning and general reasoning share some underlying structure, but the existing experiments mainly demonstrate the effectiveness of jointly training on safety and general data, without adequately distinguishing the benefits of a shared reasoning mechanism from those of ordinary multi-task training. In addition, the method involves additional continual pre-training, and the baselines differ in training objectives, data structure, and training budget, so the specific source of the performance gains remains unclear. Although the authors conduct ablations on training stages and structured fields, these experiments cannot fully rule out the influence of additional training compute or additional training information. Therefore, the paper's conclusion that safety is internalized as a foundational capability still lacks sufficiently direct evidence.
Questions for authors
Can the authors further explain how the existing experiments demonstrate that safety and general capability are indeed learned jointly through a shared reasoning structure, rather than merely as an effect of jointly training on the two types of data? In addition, can they provide a simple continual pre-training baseline with strictly controlled training data, token budget, and training objective, so as to more clearly identify the independent contribution of the proposed data structure? Finally, how do the authors distinguish between the model genuinely acquiring more stable safety reasoning capability and the model merely learning more persistent safety response patterns?
4 comments
Thank you again for your thoughtful comments and suggestions. We have carefully addressed each point in our response and would be grateful if you could let us know whether the clarifications resolve your concerns. We look forward to receiving any further feedback from you.
Thank you for the detailed response to my review. In general, the rebuttal has addressed most of my questions, hence, I'm increasing the overall score.
Thank you for your encouraging feedback and for taking the time to review our response. We truly appreciate your thoughtful engagement and valuable comments.
Weakness 1 Evidence for shared reasoning and the source of gains
Thank you for this comment. We add a controlled comparison with ordinary multi-task CPT to examine the contribution of the unified reasoning structure under matched training conditions. Both conditions use the same source data and sampling procedure, with the response portion kept identical and the training objective and token budget matched. The ordinary multi-task baseline generates safety and general data naturally for each task, while our method constructs reasoning traces for both through a common structure. Table R1 combines the new ordinary multi-task CPT results with the ASR and utility results for our method reported in Tables 1 and 2. Before downstream SFT, our method reduces ASR from 9.3% to 6.6% and improves utility from 70.6% to 73.1% under the matched training conditions. The improvement in Table R1 therefore supports the contribution of this shared organization when both models already receive safety and general data. The difference becomes more pronounced during subsequent general-only SFT, where the ordinary CPT model's ASR increases from 9.3% to 36.2%, while ours changes from 6.6% to 7.6%. This shows that the safety learned through our method remains effective when subsequent training contains no safety examples, supporting the stability of the resulting capability during general-task adaptation. The SP and AP results in Table 2 provide further evidence concerning training-data scale. Their training corpora contain approximately 2.7B and 1.3B tokens, respectively, and their Qwen3-4B models reach 17.3% and 14.5% ASR after general-only SFT, compared with 7.6% for ours. Our stronger safety performance with a smaller training corpus further supports the importance of the reasoning data design. These results support unified reasoning as an effective approach to learning safety alongside general capabilities, with safety gains that remain stable during later general-only training.
Question 1 Shared reasoning and stable safety behavior
Thank you for this helpful suggestion. The controlled CPT comparison above addresses the contribution of the shared reasoning structure, and we further examine whether the resulting safety gains accompany appropriate responses to benign requests and generalize across evaluation conditions. Before downstream SFT, Table R1 shows that our method reduces XSTest over-refusal from 24.6% to 9.7%, alongside the reduction in harmful-request ASR from 9.3% to 6.6%. Utility also increases from 70.6% to 73.1%. The model therefore handles harmful requests more safely while answering benign requests more reliably, supporting greater selectivity in its safety judgments. The main safety averages in Section 4.1 exclude benchmarks that share sources with the midtraining data, so the gains in these averages also provide evidence of generalization across benchmark sources. We additionally evaluate five StrongREJECT jailbreak methods to examine whether the safety advantage persists when the instructions surrounding a harmful request change. Table R2 shows lower ASR for our method under all five evaluated methods. Under refusal suppression, our model reaches 3.8% ASR compared with 47.3% for ordinary CPT, and under short style injection it reaches 0.3% compared with 35.8%. These results show that the safety advantage remains effective under instructions that attempt to alter the model's response behavior. During general-only SFT, our model also maintains low ASR while reaching 74.9% utility, compared with 62.4% for ordinary CPT, showing that continued learning on general tasks preserves useful safety behavior alongside general capability. These findings support the value of unified reasoning for learning selective safety behavior that generalizes across prompt changes and remains effective during subsequent training.
Table R1 Controlled multi-task CPT comparison on Qwen3-4B
Table R2 ASR (%) under five StrongREJECT jailbreak methods