acceptodds
Under review as a conference paper at ICLR 2027

RMC-Sec: A Process-Gated Reward Framework for Secure Code Generation

Abstract

Large language models are widely used to assist software development, yet vulnerabilities in generated code remain a major obstacle to their reliable use. Prior work has explored reinforcement learning (RL) to improve code security using feedback from functional tests or static analysis results. However, these outcome-based rewards provide little direct feedback on security reasoning and assign the same reward to responses with the same evaluation outcome, regardless of differences in reasoning quality. To address these limitations, we propose RMC-Sec, a process-gated reward framework for secure code generation. Starting from a policy initialized via supervised fine-tuning, RMC-Sec evaluates three sequential stages—Risk Recognition (R), Mitigation Planning (M), and Implementation Consistency (C)—and integrates the resulting scores into a hierarchical gating reward function. To mitigate reward hacking, we check the final code with a static analyzer and use the binary pass/fail result as a hard gate for reward assignment. We also mix security tasks with functional code generation tasks during RL to balance code security and functional correctness. Experiments show that, with 7,768 RL training examples, the 14B RMC-Sec model achieves the best or joint-best performance across five code generation and vulnerability repair metrics. On BaxBench, it improves the joint functional–security pass rate by 68.0% relative to the strongest baseline, while preserving functional performance on the evaluated general-purpose coding tasks. We further demonstrate that the learned security reasoning generalizes to vulnerability analysis and repair.

Primary area:
reinforcement learning
Keywords:
Secure Code Generation, Large Language Models, Reinforcement Learning, Security Reasoning, Reward Design, Static Analysis.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sAfPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RMC-Sec trains a code model to recognize security risks, propose mitigations, and implement those mitigations consistently. After SFT on teacher-generated structured responses, GRPO mixes security and functional coding tasks. For security tasks, an LLM evaluator builds a response-independent reference risk inventory and scores risk recognition (R), mitigation planning (M), and implementation consistency (C). A hierarchical formula lets risk recognition modulate downstream rewards and ties implementation credit to mitigation quality. Format checks and selected CodeQL checks act as hard gates; failing candidates receive zero security reward. On Qwen2.5-14B-Instruct-1M, the paper reports gains across secure generation, repair, and detection, including benchmarks jointly testing functionality and security, while largely preserving general coding performance.

Critical strengths
  1. The reward design explicitly connects security analysis to executable controls. The reference risk inventory is constructed from the task before inspecting candidate responses, reducing response-dependent shifts in the scoring target. Recognition balances coverage and precision, mitigation is judged for identified risks, and consistency checks require actual implementation rather than comments or imported APIs. Hard gates and these explicit criteria provide a more structured signal than a single unconstrained holistic judgment.

  2. The experiments include a meaningful matched reward baseline and several complementary evaluations. Full RMC-Sec and the binary-CodeQL variant share the SFT initialization, training tasks, functionality reward, and other RL settings. Their comparison supports a benefit from the added process assessment. BaxBench and CWEval jointly evaluate functionality and security, while repair, detection, general coding, and cross-language results provide evidence beyond the training-time static check. The paper also reports cases where another method or the binary-reward variant performs better.

Critical weaknesses
  1. The experiments do not establish the value of the particular hierarchical decomposition. Removing all RMC scoring shows that richer feedback helps, but does not distinguish the proposed dependencies from an additive combination of R/M/C, a simpler holistic LLM security reward, or one of the components alone. The fixed weights and recognition-gate floor are also not examined. Since the manuscript acknowledges earlier work combining reasoning rewards with outcome gates, a matched comparison isolating its security-specific hierarchy is important for assessing the methodological contribution.

  2. The reliability of the process evaluator remains insufficiently validated. A single LLM supplies the reference risks and process judgments, and reference-list omissions can cause valid risk statements to be penalized or missed risks to escape the recall denominator. The implementation includes useful safeguards, but no independent expert audit, inter-evaluator agreement, or controlled tests of misleading reasoning and incomplete implementations are reported. The independent downstream tests support practical gains, yet do not establish that the intermediate scores reliably measure the intended stages. Evidence from one backbone and apparently one run per training condition also limits confidence in small performance differences.

Questions for authors
  1. Missing simpler reward baselines using the same LLM evaluator limits attribution of the proposed structure’s benefits. RMC-Sec uses DeepSeek V4 Flash to provide detailed security judgments, but its main reward ablation compares only against binary CodeQL feedback. Consequently, the results do not distinguish the benefit of additional LLM feedback from that of the R/M/C decomposition and hierarchical gating. Relevant baselines would use the same DeepSeek V4 Flash to directly assign either an overall security-reasoning score or a holistic security-quality score to the complete response, under matched SFT initialization, training data, format and CodeQL gates, and GRPO budgets. A simple weighted sum of the same R/M/C scores would further help establish the additional value of the hierarchical dependencies.
  2. The process evaluator is not sufficiently validated. A single LLM provides both the reference risks and the process judgments. If the reference list omits a risk, a valid risk statement can be penalized, or a missed risk can fall outside the recall denominator. The implementation includes useful safeguards. However, the manuscript reports no independent expert audit, no inter-evaluator agreement, and no controlled tests with misleading reasoning or incomplete implementations. The independent downstream tests show practical gains, but they do not show that the intermediate scores measure the intended stages. The experiments also use one backbone and, apparently, one run per training condition, so small performance differences are hard to interpret.
1 comment

We thank the reviewer for recognizing the connection between security reasoning and executable controls. To address the attribution of structural benefits and the reliability of scoring, we have added three ablations and an analysis of agreement between reference-risk and R/M/C judgments and expert annotations across 50 tasks.

1. Does the hierarchical structure provide additional benefits?

The new additive baseline retains the R/M/C scores, original weights, and format and CodeQL gates, but changes the reward to G(0.45R+0.35M+0.20C)G(0.45R+0.35M+0.20C), comparing different aggregation methods for the same process scores. We also add w/o C, which removes the C evaluation and renormalizes the remaining weights, and w/o M&C, which removes both the M and C evaluations.

MetricRMC-SecAdditivew/o Cw/o M&C
BaxBench sec_pass@1 (%)10.718.937.405.61
CWEval func-sec@1 (%)42.0239.5037.8238.66
CyberSecEval-SCG security rate (%)78.7676.3875.7974.06
CoV-Eval completion security rate (%)68.5259.2661.1159.26
CoV-Eval repair security rate (%)77.3681.7679.2575.05
HumanEval+ pass@1 (%)78.6678.6679.8881.10
MBPP+ pass@1 (%)70.3769.3169.3167.46

The full model outperforms all three variants on all four secure-generation metrics. Adding M improves BaxBench, CyberSecEval-SCG, and CoV-Eval completion and repair; adding C further improves the four generation metrics, supporting both stages' contributions. The full model also exceeds the additive baseline on detection and classification, although the additive reward performs better on repair. These results support dependency-aware aggregation. On functionality, the full model leads on MBPP+ and remains comparable on HumanEval+, preserving general programming ability.

Our work focuses further on dependencies between stages of security reasoning. In this round, we prioritized the additive comparison targeting hierarchical aggregation and have not conducted an experiment using the same evaluator to assign a holistic security score. Holistic scoring would require changes to the prompt and scoring criteria, so its results could be affected by those design choices. The additive baseline retains the same R/M/C judgments, weights, and outer gates, making it easier to isolate the contribution of the aggregation structure.

2. Agreement between the process evaluator and experts

We randomly sampled 50 distinct security tasks from training responses that passed the format and CodeQL gates and had complete scoring records. Experts with code-security expertise reviewed each item, and we computed binary agreement metrics using human judgments as the reference:

Review itemPrecisionRecallF1
Risk inventory agreement79.73%84.51%82.05%
R risk-recognition agreement79.66%100.00%88.68%
M mitigation-effectiveness agreement95.74%100.00%97.83%
C implementation-consistency agreement97.44%92.68%95.00%

The reference inventory achieves a recall of 84.51% and an F1 score of 82.05%, providing direct evidence of risk coverage. The F1 scores for R, M, and C are 88.68%, 97.83%, and 95.00%, respectively, supporting the ability of stage-specific scores to reflect expert judgments of risk recognition, mitigation effectiveness, and control implementation.

3. Weights and the recognition-gate floor

The weights (0.45,0.35,0.20)(0.45,0.35,0.20) and gate floor δ=0.35\delta=0.35 were not chosen arbitrarily. During early development, we performed a grid search over weight settings on 50 representative samples and selected the setting that best distinguished high-quality from low-quality reasoning processes in terms of reward.

4. Repeated experiments

We agree that multi-seed and cross-backbone validation would strengthen statistical reliability. The existing evidence also suggests that the results are not sensitive to training randomness: RL starts from the same SFT checkpoint; process rewards are computed using a temperature-0 evaluator, deterministic static analysis, and fixed scoring rules; and checkpoints are selected on a validation set. This limits variance in the reward signal and model selection. More importantly, improvements are consistent across benchmarks using different evaluation methods—execution tests, static analysis, and LLM judges—and across eight programming languages. Such consistency is difficult to explain through chance fluctuations in a single run. We will add multi-seed and cross-backbone experiments in future work to further quantify the effect of training variance on our conclusions.

Reviewer_tBLZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an RL reward framework (RMC-Sec) for secure code generation. The reward combines risk recognition, mitigation planning and implementation consistency through process gating. The paper first applies SFT to Qwen2.5-14B-Instruct-1M using 21,024 teacher trajectories, then performs GRPO training on 5,054 cyber tasks and 2,714 functional programing tasks. Results show that posttrained model using RMC-Sec gains 4.33% improvement on BaxBench, compared to the strongest baseline.

Critical strengths

Overall, this paper is well motivated and novel on the reward design. Compared to binary reward, a reward signal consists of risk recognition, mitigation planning and implementation consistency is a reasonable framework for improving models' security reasoning and secure code generating. Besides, this paper presents detailed technique and experiments, which makes the reward design easy to follow.

Critical weaknesses

My main concern is the reliability of the reference ground truth and reward judgments used during training. There are two potential problems in the ground truth method. The first one is that CodeQL as a static analysis tool can generate false positives while missing true positives. The paper gives zero reward for CodeQL reports without re-validating them, so false positives can lead to wrong rewards. Also, the post-analysis model may help find vulnerabilities missed by CodeQL, but it can make mistakes too, especially without prompting the model/agent to generate a working PoC. An agentic re-validation and re-auditing workflow, to my knowledge, is a better method on detecting the true positive vulnerabilities.

While I liked the idea of separate the security reward into three gated score, I think the paper does not strongly present the evidence for the proposed reward design. Though table 3 compares binary CodeQL rewards with the complete RMC framework, an ablation study isolating risk recognition, mitigation plan, and code implementation would help show how much each component contributes and whether all three are needed.

Questions for authors

None.

1 comment

We thank the reviewer for recognizing the motivation and the reward design. Regarding the concerns about the reliability of ground-truth construction and reward judgments, we first acknowledge that CodeQL produces false positives and false negatives, explain how the current design handles both, and respond to the agentic re-validation suggestion; we then present a 50-task expert review as evidence for the reliability of the reward judgments; finally, two stage ablations examine each stage's contribution.

1. Reliability of ground-truth construction and reward judgments

We acknowledge that CodeQL, as a static analysis tool, can produce false positives and can miss real vulnerabilities, and we agree that this affects the accuracy of zero-reward decisions. The current design handles the two issues as follows.

False positives. The gate uses only security alerts of medium or higher severity with valid CWE identifiers and excludes alerts that concern code quality alone, reducing the chance that false positives enter zero-reward decisions.

False negatives. To our knowledge, no existing method detects vulnerabilities with perfect recall. Our design goal is not a perfect vulnerability ground truth, but to shrink the reward-hacking space through a hard static-check gate: even if a response earns a high process score through seemingly complete risk recognition and mitigation planning, its security reward remains zero as long as the final code triggers a security alert. As a reference point, PurpCode likewise uses static-analyzer results as its security reward in RL; our gate follows the same practice, adds the process scores on top, and does not relax the static check.

On agentic re-validation. We agree that re-validation is a valuable direction, but its form matters. If the agent only reads and analyzes the code, its conclusion is still an LLM judgment with no essential gain in reliability—the same problem our current LLM evaluator faces. If the agent writes and runs PoCs for dynamic verification, it can indeed prove the presence of vulnerabilities effectively, but this requires constructing and executing exploits for every candidate response during training, which would greatly increase the time cost of RL training and is hard to afford under the current training budget. We consider it a promising direction for future improvement.

Expert review of reward judgments. The reviewer also notes that the LLM used to supplement vulnerability analysis can itself make mistakes. To quantify the reliability of the reward judgments, we randomly sampled 50 distinct security tasks from training responses that passed the format and CodeQL gates and had complete scoring records; experts with code-security expertise reviewed each item, and we computed binary agreement with human judgments as the reference:

Review itemPrecisionRecallF1
Risk inventory agreement79.73%84.51%82.05%
R risk-recognition agreement79.66%100.00%88.68%
M mitigation-effectiveness agreement95.74%100.00%97.83%
C implementation-consistency agreement97.44%92.68%95.00%

2. Contribution of each stage

We add two ablations: w/o M&C keeps only the risk-recognition reward and removes the M and C evaluations; w/o C removes the C evaluation, renormalizes the remaining weights, and keeps the gating of M by R. Both variants keep the format and CodeQL hard gates.

MetricRMC-Secw/o Cw/o M&C
BaxBench sec_pass@1 (%)10.717.405.61
CWEval func-sec@1 (%)42.0237.8238.66
CyberSecEval-SCG security rate (%)78.7675.7974.06
CoV-Eval completion security rate (%)68.5261.1159.26
CoV-Eval repair security rate (%)77.3679.2575.05
HumanEval+ pass@1 (%)78.6679.8881.10
MBPP+ pass@1 (%)70.3769.3167.46

Adjacent comparisons separate the role of each stage. Adding the M evaluation on top of w/o M&C improves BaxBench, CyberSecEval-SCG, CoV-Eval completion, and repair, showing an independent benefit of the mitigation-planning evaluation. Adding the C evaluation on top of that (the full model) further improves BaxBench, CWEval, CyberSecEval-SCG, and CoV-Eval completion, and the full model outperforms both stage-ablated variants on all four secure-generation metrics. This shows that M and C each contribute an independent gain, supporting the design that links risk recognition, mitigation planning, and control implementation. On repair, w/o C is slightly higher than the full model; the advantage of the complete framework mainly appears in secure generation. On functionality, the full model outperforms both ablated variants on MBPP+ and is close to them on HumanEval+, indicating that adding the M and C evaluations preserves general programming ability while improving security.

Reviewer_4U8rOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RMC-Sec, a reward framework for secure code generation that evaluates risk recognition, mitigation planning, and implementation consistency. These assessments are combined through a hierarchical reward function, with format and CodeQL checks acting as hard gates. The authors apply supervised fine-tuning followed by GRPO on a mixture of security and functional coding tasks. Experiments on a 14B model show improvements across several security benchmarks while largely maintaining general coding performance.

Critical strengths
  • The framework addresses a limitation of outcome-only security rewards. Distinguishing risk identification, mitigation quality, and actual implementation provides an interpretable way to evaluate security reasoning. The scoring criteria include useful safeguards: reference risks are constructed independently of candidate responses, and implementation consistency requires executable controls rather than comments or superficial API mentions.
  • The experiments provide evidence for the complete training approach. In Table 3, RMC-Sec and the binary-CodeQL variant share the SFT initialization, training data, functionality reward, and main RL settings. RMC-Sec improves all five reported security generation and repair metrics over this comparison. Including BaxBench and CWEval, which jointly assess functionality and security, strengthens the evaluation beyond static-analysis pass rates. The general coding results also suggest that the security improvements do not come with a substantial loss of functionality on the evaluated benchmarks.
Critical weaknesses
  • No ablation tests the hierarchical aggregation. The main ablation replaces binary CodeQL feedback with the entire RMC evaluation pipeline. This introduces additional LLM supervision, more differentiated rewards, and the proposed decomposition and aggregation simultaneously. Consequently, the comparison supports the complete approach but does not establish the incremental value of its hierarchical structure. This distinction is important because the paper acknowledges prior work combining reasoning rewards with outcome-based gates. Its main methodological contribution therefore rests on the security-specific R/M/C assessment and their dependency-aware aggregation. A simple weighted sum of the same R/M/C scores would directly test the aggregation, while an overall security-quality score from the same evaluator would help assess the value of the decomposition. These comparisons should retain the same initialization, training data, outer gates, and comparable training budgets.
  • The reliability of the process supervision is insufficiently validated. An LLM constructs the reference risk inventory and supplies the judgments used to compute R, M, and C, but the paper does not report an independent assessment of their accuracy. This matters directly for the training objective: an incomplete reference inventory can penalize a valid risk statement or exclude an important risk from the coverage calculation. Errors in judging mitigation effectiveness or implementation can similarly reward responses that do not satisfy the intended security criteria. Detailed prompts tell us what the judge is asked to do, not whether it does it correctly. Better benchmark scores show that the training helps, but not that the R/M/C scores are accurate. A targeted expert audit and controlled tests involving ineffective mitigations or incomplete implementations would substantially strengthen this central part of the paper.
Questions for authors
  • Under matched training conditions, how does the proposed reward compare with a simple weighted sum of the same R/M/C scores and with an overall security score produced by the same LLM evaluator? Do these comparisons demonstrate an additional benefit from the proposed decomposition and hierarchical dependencies?
  • Can the authors independently validate a representative sample of reference inventories and R/M/C judgments? Particularly useful measurements would include reference-risk coverage, agreement with expert assessments, and sensitivity to deliberately ineffective mitigations or missing implementations.
1 comment

We thank the reviewer for the positive assessment of RMC-Sec's design and experiments. Following your suggestions, we have added three reward ablations and an expert review of 50 tasks to quantify reference-risk coverage and the agreement of R/M/C judgments with expert annotations.

1. Contribution of hierarchical aggregation

The new additive baseline retains the same R/M/C scores, weights, and format and CodeQL gates, but changes the reward to G(0.45R+0.35M+0.20C)G(0.45R+0.35M+0.20C), directly comparing additive and hierarchical aggregation. We also add two variants: w/o C, which removes the C evaluation and renormalizes the remaining weights, and w/o M&C, which removes both the M and C evaluations.

MetricRMC-SecAdditivew/o Cw/o M&C
BaxBench sec_pass@1 (%)10.718.937.405.61
CWEval func-sec@1 (%)42.0239.5037.8238.66
CyberSecEval-SCG security rate (%)78.7676.3875.7974.06
CoV-Eval completion security rate (%)68.5259.2661.1159.26
CoV-Eval repair security rate (%)77.3681.7679.2575.05
HumanEval+ pass@1 (%)78.6678.6679.8881.10
MBPP+ pass@1 (%)70.3769.3169.3167.46

The full model outperforms all three variants on the security-related metrics of BaxBench, CWEval, CyberSecEval-SCG, and CoV-Eval completion. Comparisons between adjacent stages distinguish the contributions of the individual modules: adding the M evaluation to w/o M&C improves BaxBench, CyberSecEval-SCG, and CoV-Eval completion and repair, indicating an independent contribution from mitigation-planning evaluation; adding the C evaluation further improves BaxBench, CWEval, CyberSecEval-SCG, and CoV-Eval completion. On functionality, the full model outperforms all three variants on MBPP+, while the variants perform similarly on HumanEval+. These results support hierarchical aggregation as a way to improve secure generation while preserving general programming ability. The additive reward performs better on CoV-Eval repair, with the full hierarchy's advantage appearing primarily in secure generation.

Given the time and resource constraints, and our focus on dependencies between stages of security reasoning, we prioritized the additive comparison targeting hierarchical aggregation and have not conducted an experiment using the same evaluator to assign a holistic security score. Holistic scoring would also require changes to the prompt and scoring criteria, so its results could be affected by those design choices. The additive baseline retains the same R/M/C judgments, weights, and outer gates, making it easier to isolate the contribution of the aggregation structure.

2. Expert validation of reference-risk coverage and process judgments

We randomly sampled 50 distinct security tasks from training responses that passed the format and CodeQL gates and had complete scoring records. Experts with code-security expertise reviewed each item, and we computed binary agreement metrics using human judgments as the reference:

Review itemPrecisionRecallF1
Risk inventory agreement79.73%84.51%82.05%
R risk-recognition agreement79.66%100.00%88.68%
M mitigation-effectiveness agreement95.74%100.00%97.83%
C implementation-consistency agreement97.44%92.68%95.00%

For R (risk recognition), the evaluator assigns an exact, partial, or missed label to each reference risk; exact and partial are merged as "identified," and missed is treated as "not identified." For M (mitigation planning), evaluation is restricted to risks judged as identified by both the evaluator and the experts; adequate and partial are merged as "effective measure," while ineffective and the absence of a relevant measure are merged as "no effective measure."

The reference inventory covers 84.51% of the risks in the expert inventory, directly addressing your concern about reference-risk coverage. M and C achieve F1 scores of 97.83% and 95.00%, respectively, providing direct evidence of agreement with experts on the two key judgments of mitigation effectiveness and control implementation.

Comments on the paper

1 comment

We thank the reviewer for the positive assessment of the reference-risk design and the matched reward comparison. We have added three reward ablations, and we validated risk-inventory coverage and the agreement of the process judgments against expert annotations. We answer the four questions in turn.

1. Hierarchical structure and stage ablations

The additive baseline keeps the R/M/C scores, the original weights, and the outer hard gates, and changes the reward to G(0.45R+0.35M+0.20C)G(0.45R+0.35M+0.20C); w/o C removes the C evaluation and renormalizes the remaining weights; w/o M&C removes both the M and C evaluations.

MetricRMC-SecAdditivew/o Cw/o M&C
BaxBench sec_pass@1 (%)10.718.937.405.61
CWEval func-sec@1 (%)42.0239.5037.8238.66
CyberSecEval-SCG security rate (%)78.7676.3875.7974.06
CoV-Eval completion security rate (%)68.5259.2661.1159.26
CoV-Eval repair security rate (%)77.3681.7679.2575.05
HumanEval+ pass@1 (%)78.6678.6679.8881.10
MBPP+ pass@1 (%)70.3769.3169.3167.46

The full model outperforms all three variants on the four secure-generation metrics, supporting the hierarchical aggregation and the complete evaluation chain. Adjacent comparisons in the stage ablation isolate each module's contribution: adding the M evaluation on top of w/o M&C improves BaxBench, CyberSecEval-SCG, CoV-Eval completion, and repair, indicating that the mitigation-planning evaluation has an independent effect; adding the C evaluation further improves BaxBench, CWEval, CyberSecEval-SCG, and CoV-Eval completion. Relative to the additive reward, the full model is also higher on detection and classification. The additive reward is higher on repair, so the structural benefit mainly shows in secure generation. On functionality, the full model outperforms all three variants on MBPP+ and is close to them on HumanEval+, indicating that the complete evaluation chain does not harm general programming ability.

2. Expert review and reference-risk omission

We randomly sampled 50 distinct security tasks from training responses that passed the format and CodeQL gates and had complete scoring records. Experts with code-security expertise reviewed them item by item, and we report binary agreement with human judgments as the reference:

Review itemPrecisionRecallF1
Risk inventory agreement79.73%84.51%82.05%
R risk-recognition agreement79.66%100.00%88.68%
M mitigation-effectiveness agreement95.74%100.00%97.83%
C implementation-consistency agreement97.44%92.68%95.00%

The reference inventory covers 84.51% of the expert risk list. The recall of R is 100%, meaning that in this review sample every risk the expert judged as identified was also judged as identified by the evaluator; the F1 scores of M and C are 97.83% and 95.00%. These results directly support the agreement between the process judgments and expert judgments.

3. Random seeds

We agree that multi-seed and cross-backbone validation would strengthen statistical reliability. We would also note that the existing evidence suggests the results are not sensitive to training randomness: RL starts from the same SFT checkpoint; the process reward is computed by a temperature-0 evaluator, deterministic static analysis, and fixed scoring rules, and the checkpoint is selected by validation performance, so the variance of the reward signal and of model selection itself is limited. More importantly, the improvements are directionally consistent across multiple benchmarks with heterogeneous evaluation stacks (execution tests, static analysis, LLM judging) and across eight programming languages—a consistency that is hard to explain by the chance fluctuation of a single run. We will add multi-seed and cross-backbone experiments in future work to further quantify the impact of training variance on the conclusions.

4. CoV-Eval repair evaluation protocol

LLM-as-judge is the evaluation logic of the official CoV-Eval protocol itself, not an additional design of ours. To avoid same-model preference, we deliberately chose GPT-4o as the judge, which comes from a different model family than the DeepSeek evaluator used in training. In addition, the repair metric of this benchmark measures the security rate of the repaired code; its protocol does not require verifying whether functionality is preserved after repair. For evidence that the model improves security while preserving functionality, we point to BaxBench and CWEval: both take the joint functionality–security pass rate as their core metric, and RMC-Sec achieves the best or joint-best performance on these two joint evaluations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.