- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose SecureForge, a pipeline that automatically optimizes system prompts to improve models’ ability to generate secure code. SecureForge consists of two stages: first, it uses MCMC to construct a diverse set of tasks that are likely to elicit security failures; second, it uses GEPA to optimize the system prompt on these tasks. Experiments on an author-constructed dataset and SWE-chat show some improvement over baselines that use simple prompts.
Critical strengths
- The authors conduct extensive experiments across multiple models. The inclusion of realistic coding tasks such as SWE-chat strengthens the evaluation’s practical relevance.
- The authors assess security using Semgrep, CodeQL, and red teaming. These complementary approaches provide a broader assessment than relying on a single evaluation method.
- The approach is lightweight and broadly applicable.
Critical weaknesses
- Limited improvements: The gains are modest. For example, on SWE-chat, the improvements in P(secure, tests pass) over the Secure baseline are only around 5% (Figure 5). The gains in the red-teaming setting shown in Figure 9 are even smaller; moreover, the authors compare only against the Baseline, omitting the Secure/CWE-aware baseline in the red-teaming setting.
- Insufficient evidence of method effectiveness: The current results do not isolate incremental contributions of SecureForge. First, the baselines are relatively weak: simply adding actionable mitigation instructions might yield improvements over these baselines. Second, the authors do not provide sufficient ablation studies to demonstrate that the proposed components meaningfully contribute to the final performance.
Questions for authors
-
Contribution of actionable mitigation instructions. Does the system prompt optimized by SecureForge contain a set of actionable mitigation instructions? If so, I would encourage the authors to compare SecureForge against a baseline that simply incorporates readily available mitigation guidance. For example, MITRE CWE entries typically include a “Potential Mitigations” section [1]. Could comparable improvements be achieved by appending these mitigations for all studied CWEs to a fixed system prompt and using it for code generation?
This comparison would help clarify whether SecureForge’s gains over the Secure and CWE-aware baselines stem primarily from providing explicit, actionable mitigation instructions, or from its ability to develop more effective instructions through the proposed optimization procedure.
-
Further ablation analysis. The current MCMC comparison shows that it elicits more vulnerable outputs than simple rephrasing, but does not establish that it leads to a better final system prompt. I encourage the authors to compare simple rephrasing + GEPA with MCMC amplification + GEPA, evaluating the resulting optimized system prompts to demonstrate MCMC’s contribution to the effectiveness of the optimization process.
-
Fairness of the comparison. Figure 10 aggregates 27,660 samples for Secure and 25,880 for SecureForge. Could the authors explain why the sample counts differ and whether both methods were evaluated on the same set of tasks? Given the modest gains reported for SecureForge, differences in evaluation coverage could affect the fairness of the comparison and the reliability of the conclusions.
6 comments
Mitigation Instructions
Does the system prompt optimized by SecureForge contain a set of actionable mitigation instructions?
Yes, but we note that in “CWE” based prompting baseline, we also apply the entire CWE page of the CWEs we are looking for as a part of the prompt. This includes mitigation instructions whenever they are available.
Our method outperforms this approach because it produces actionable instructions that are model-specific, e.g., in the form of structured few-shot examples for failure. This allows us to mitigate CWE type failures even when generic mitigations fail.
Effect Size
Agreed that the joint passage rate gains are smaller than the mere vulnerability reduction. However, as seen in Figure 4, the rates of joint test pass rate and vulnerability reduction is dominated primarily due to the difficulty of keeping test passage rates high. As an example, GPT-6-terra achieved 39% reduction of vulnerability (5 pp) compared to baseline but only 5% (4 pp) joint rate improvement.
For Figure 9, we will note that red-teaming is a zero-shot transfer setting where we optimize our method on CWEs and then apply them using full red-teaming exercises. If our method were optimized on red-teaming, we expect to see additional improvements.
Further Ablation on MCMC
To validate the necessity of our MCMC setup, we perform an ablation whereby we perform the same optimization scheme and subsequent rollout from the seed failure prompts directly.
We were surprised by the low test passage rate, and investigated some excerpts of generated prompts. The prompts form direct optimization appears to be overfit to the specific frameworks used during the initial seed generation. As such, there are a series of key differences:
Security Direct optimization uses explicit construction of rules not to do (very similar to mitigation rules) instead of higher-level guidance as in MCMC outputs.
Test Passage Direct optimization prompts prescribes the test boundaries and how to monkeypatch (we believe this is the reason why test pass rates drop precipitously), instead of recommending higher level patterns of passing tests.
To illustrate this, here are the first two paragraphs of an optimized system prompts from each of the optimization schemes.
Direct Optimization
MCMC