- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RMC-Sec trains a code model to recognize security risks, propose mitigations, and implement those mitigations consistently. After SFT on teacher-generated structured responses, GRPO mixes security and functional coding tasks. For security tasks, an LLM evaluator builds a response-independent reference risk inventory and scores risk recognition (R), mitigation planning (M), and implementation consistency (C). A hierarchical formula lets risk recognition modulate downstream rewards and ties implementation credit to mitigation quality. Format checks and selected CodeQL checks act as hard gates; failing candidates receive zero security reward. On Qwen2.5-14B-Instruct-1M, the paper reports gains across secure generation, repair, and detection, including benchmarks jointly testing functionality and security, while largely preserving general coding performance.
Critical strengths
-
The reward design explicitly connects security analysis to executable controls. The reference risk inventory is constructed from the task before inspecting candidate responses, reducing response-dependent shifts in the scoring target. Recognition balances coverage and precision, mitigation is judged for identified risks, and consistency checks require actual implementation rather than comments or imported APIs. Hard gates and these explicit criteria provide a more structured signal than a single unconstrained holistic judgment.
-
The experiments include a meaningful matched reward baseline and several complementary evaluations. Full RMC-Sec and the binary-CodeQL variant share the SFT initialization, training tasks, functionality reward, and other RL settings. Their comparison supports a benefit from the added process assessment. BaxBench and CWEval jointly evaluate functionality and security, while repair, detection, general coding, and cross-language results provide evidence beyond the training-time static check. The paper also reports cases where another method or the binary-reward variant performs better.
Critical weaknesses
-
The experiments do not establish the value of the particular hierarchical decomposition. Removing all RMC scoring shows that richer feedback helps, but does not distinguish the proposed dependencies from an additive combination of R/M/C, a simpler holistic LLM security reward, or one of the components alone. The fixed weights and recognition-gate floor are also not examined. Since the manuscript acknowledges earlier work combining reasoning rewards with outcome gates, a matched comparison isolating its security-specific hierarchy is important for assessing the methodological contribution.
-
The reliability of the process evaluator remains insufficiently validated. A single LLM supplies the reference risks and process judgments, and reference-list omissions can cause valid risk statements to be penalized or missed risks to escape the recall denominator. The implementation includes useful safeguards, but no independent expert audit, inter-evaluator agreement, or controlled tests of misleading reasoning and incomplete implementations are reported. The independent downstream tests support practical gains, yet do not establish that the intermediate scores reliably measure the intended stages. Evidence from one backbone and apparently one run per training condition also limits confidence in small performance differences.
Questions for authors
- Missing simpler reward baselines using the same LLM evaluator limits attribution of the proposed structure’s benefits. RMC-Sec uses DeepSeek V4 Flash to provide detailed security judgments, but its main reward ablation compares only against binary CodeQL feedback. Consequently, the results do not distinguish the benefit of additional LLM feedback from that of the R/M/C decomposition and hierarchical gating. Relevant baselines would use the same DeepSeek V4 Flash to directly assign either an overall security-reasoning score or a holistic security-quality score to the complete response, under matched SFT initialization, training data, format and CodeQL gates, and GRPO budgets. A simple weighted sum of the same R/M/C scores would further help establish the additional value of the hierarchical dependencies.
- The process evaluator is not sufficiently validated. A single LLM provides both the reference risks and the process judgments. If the reference list omits a risk, a valid risk statement can be penalized, or a missed risk can fall outside the recall denominator. The implementation includes useful safeguards. However, the manuscript reports no independent expert audit, no inter-evaluator agreement, and no controlled tests with misleading reasoning or incomplete implementations. The independent downstream tests show practical gains, but they do not show that the intermediate scores measure the intended stages. The experiments also use one backbone and, apparently, one run per training condition, so small performance differences are hard to interpret.
We thank the reviewer for recognizing the connection between security reasoning and executable controls. To address the attribution of structural benefits and the reliability of scoring, we have added three ablations and an analysis of agreement between reference-risk and R/M/C judgments and expert annotations across 50 tasks.
1. Does the hierarchical structure provide additional benefits?
The new additive baseline retains the R/M/C scores, original weights, and format and CodeQL gates, but changes the reward to G(0.45R+0.35M+0.20C), comparing different aggregation methods for the same process scores. We also add w/o C, which removes the C evaluation and renormalizes the remaining weights, and w/o M&C, which removes both the M and C evaluations.
The full model outperforms all three variants on all four secure-generation metrics. Adding M improves BaxBench, CyberSecEval-SCG, and CoV-Eval completion and repair; adding C further improves the four generation metrics, supporting both stages' contributions. The full model also exceeds the additive baseline on detection and classification, although the additive reward performs better on repair. These results support dependency-aware aggregation. On functionality, the full model leads on MBPP+ and remains comparable on HumanEval+, preserving general programming ability.
Our work focuses further on dependencies between stages of security reasoning. In this round, we prioritized the additive comparison targeting hierarchical aggregation and have not conducted an experiment using the same evaluator to assign a holistic security score. Holistic scoring would require changes to the prompt and scoring criteria, so its results could be affected by those design choices. The additive baseline retains the same R/M/C judgments, weights, and outer gates, making it easier to isolate the contribution of the aggregation structure.
2. Agreement between the process evaluator and experts
We randomly sampled 50 distinct security tasks from training responses that passed the format and CodeQL gates and had complete scoring records. Experts with code-security expertise reviewed each item, and we computed binary agreement metrics using human judgments as the reference:
The reference inventory achieves a recall of 84.51% and an F1 score of 82.05%, providing direct evidence of risk coverage. The F1 scores for R, M, and C are 88.68%, 97.83%, and 95.00%, respectively, supporting the ability of stage-specific scores to reflect expert judgments of risk recognition, mitigation effectiveness, and control implementation.
3. Weights and the recognition-gate floor
The weights (0.45,0.35,0.20) and gate floor δ=0.35 were not chosen arbitrarily. During early development, we performed a grid search over weight settings on 50 representative samples and selected the setting that best distinguished high-quality from low-quality reasoning processes in terms of reward.
4. Repeated experiments
We agree that multi-seed and cross-backbone validation would strengthen statistical reliability. The existing evidence also suggests that the results are not sensitive to training randomness: RL starts from the same SFT checkpoint; process rewards are computed using a temperature-0 evaluator, deterministic static analysis, and fixed scoring rules; and checkpoints are selected on a validation set. This limits variance in the reward signal and model selection. More importantly, improvements are consistent across benchmarks using different evaluation methods—execution tests, static analysis, and LLM judges—and across eight programming languages. Such consistency is difficult to explain through chance fluctuations in a single run. We will add multi-seed and cross-backbone experiments in future work to further quantify the effect of training variance on our conclusions.