- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces a post-training method to improve the security of code generated by LLMs. The method co-trains two adapters, one for secure code generation, and one for insecure code generation. First, using sec-insec code data pairs from prior datasets, these two adapters are cold-started. Then, a two step loop is repeated: 1) sec-insec pairs are generated from the two adapters and preference learning is applied (sec preference for the sec adapter, and vice-versa); 2) GRPO RL is done on each adapter separately. The evaluation across 5 models shows some improvements over the base model and prior methods.
Critical strengths
- Important topic as coding models and agents become ever more prevalent.
- Interesting idea to maintain an opposite policy to aid the training of the target policy.
Critical weaknesses
- I am not convinced about the effectiveness of the full method based on the results. Note that here I am referring to the, in my view, main metric: C&S. From the five models used in the evaluation, only in the case of Granite 4.1 is the full method clearly above any alternative. For Qwen 2.5 Coder 7B, an ablated version of the method is better + the improvement is minor compared to the best prior method; for Qwen 2.5 Coder 14B, even two ablated versions are better; for Phi-4-Mini, the full method does not improve over the base, but ablated versions do and prior work is better; and for CodeLlama, the improvement over SafeCoder is minor despite the large amount of additional data, complexity, and training.
- The used models and resulting coding settings are somewhat outdated. Current coding is mostly focused on agentic coding driven by reasoning LLMs. While training frontier agentic models is admittedly not in scope for academic/research works; I do believe that there are already smaller scale capable reasoning code models (such as Qwen 3 or 3.5 series, for instance), that could be trained and evaluated in some simpler harnesses, making a more compelling case for any proposed method that its insights could extrapolate to frontier, or at least, practically relevant coding tasks and usecases.
Questions for authors
See weaknesses.
4 comments
[W2] Applicability To Reasoning Models
To examine applicability to reasoning models, we conducted additional experiments with Qwen3.5-4B in thinking-ON mode using three seeds (42/43/44).
To train reasoning, we added an auxiliary reasoning loss (weight 0.05) to both training stages and trained for up to five rounds. Due to training-time constraints, thinking and final code were each limited to 1,024 tokens. We followed the round-selection procedure in the main paper. Full reports a three-seed mean, while Base and RESCUE are existing single-evaluation results under the same thinking/code budgets. SafeCoder, DiSCo-LPO, and similar methods were excluded because their original SFT or preference-optimization training data do not include reasoning.
| Model/method | Test C&S | EvalPlus functionality |
|---|---|---|
| Qwen3.5-4B Base (thinking ON) | 14.49% | 57.20% |
| Qwen3.5-4B Full (thinking ON, R5) | 21.26% | 75.83% |
| Qwen3.5-4B RESCUE (thinking ON) | 13.04% | 54.61% |
Compared with thinking-ON Base, Full improved mean test C&S from 14.49% to 21.26% and EvalPlus functionality from 57.20% to 75.83%. Our method improved both C&S and functionality in this setting, demonstrating its applicability to a reasoning model. In particular, compared with the average changes from Base across the original five backbones, this reasoning model showed more pronounced improvements in both C&S and functionality.
Future work will evaluate our method on a broader range of reasoning models using agentic coding benchmarks and suitable evaluation harnesses.
Thank you for the rebuttal. I believe the method is showing promise in timelier settings. However, these results are still preliminary. A paper focusing more on these results and settings would be a better fit for the conference. Therefore, I maintain my score.
Thank you for your response and thoughtful feedback! We’ll take your suggestions into account as we continue to improve the paper.
[W1] Performance Gains With Limited Data
We thank the reviewer for their thoughtful feedback. We address the main concerns below.
A. Dataset
Our method uses 1,136 SafeCoder pairs and 885 SeCodePLT items for bootstrap, and 119 CWEval items for replay. After deduplicating the SeCodePLT items reused across bootstrap and replay, there are 2,140 original training items. In comparison, SafeCoder uses 46,719 general instructions and 1,141 security pairs, while DiSCo-LPO uses 9,489 pairs. TSP's actual training pairs contain 360 unique original instructions, and RESCUE requires no additional training. These counts refer to original items, excluding generated candidates and repeated training. Our method continually generates and refreshes training examples from this limited set of original tasks.
B. Performance
We agree that Full does not always achieve the highest score on each individual backbone. However, according to Table 1 in the main paper, Full's five-model mean test C&S is 23.3%, 3.3pp higher than the strongest external baseline, SafeCoder, at 20.0%. EvalPlus functionality is also higher for Full: 64.9% versus SafeCoder's 62.3%. DiSCo-LPO achieves 19.1% mean C&S and 60.5% EvalPlus. Thus, even these baselines with relatively close C&S results incur greater functionality losses than Full, and Full's advantage is clearer when both metrics are considered together.
To further substantiate our performance gains from limited original training data, we conducted additional experiments extending the training of both baselines and ablations. The results show that these alternatives did not reach Full's mean C&S even with additional training, supporting the value of continually generating and refreshing verified examples through co-evolution.
We extended the baselines whose measured training costs were at least 10% below Full's to bring their training time closer to Full's, retaining their data, learning rates, and loss settings. This covered DiSCo-LPO on all five base models and SafeCoder on Qwen-14B and Phi Mini. TSP already had similar training time and was not extended. The extensions used seed 42; Full reports the main paper's three-seed means. Cells report test C&S / EvalPlus functionality (%); N/A denotes a setting not extended.
Test C&S decreased relative to the original baseline results in all seven extended settings, with substantial functionality losses in some cases. Thus, simply increasing training time on their fixed data did not improve these baselines or close the gap to Full. In contrast, our method obtains gains by continually generating verified training examples from a limited set of original tasks.
We also extended No-B to R28 and Frozen-B to R15, approximately matching Full R10's total training budget based on estimated FLOPs. These experiments retained the original settings and seed-matched bootstrap, used three seeds (42/43/44), and followed the main paper's round-selection procedure. Cells report test C&S / EvalPlus functionality (%), with selected rounds in parentheses.
Even with these additional budgets, Full retained mean C&S advantages of 2.71pp over No-B and 1.45pp over Frozen-B, with similar EvalPlus functionality. Only No-B on Qwen-7B and Qwen-14B selected rounds beyond R10, and both remained below Full's C&S on those backbones. Although individual ablations still perform better on some backbones, longer training did not eliminate Full's average advantage. These results further support our central contribution: obtaining performance gains from limited original data through co-evolution, which the evaluated alternatives did not match on average by extending training alone.