- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PACO, a property-aware diffusion decoding framework for secure code generation. The key idea is to learn the order in which code tokens should be committed during diffusion decoding, prioritizing high-level program structures while delaying security-sensitive decisions until richer contextual information becomes available. The authors introduce two property-guided strategies based on AST syntactic depth and Bandit security-risk signals. These properties are learned through multitask training and subsequently used to adjust token commitment priorities during inference. Experiments on CWEval, SALLM, and QuixBugs demonstrate improvements in functional correctness, security, and inference efficiency. The results also suggest that the proposed approach can generalize across programming languages.
Critical strengths
- The paper addresses an important research problem with significant practical implications.
- The proposed approach demonstrates good novelty.
- The paper provides interesting analyses and encouraging experimental results.
Critical weaknesses
-
It remains unclear whether the reported security improvements primarily come from multitask training or the proposed property-guided decoding strategy. PACO introduces two mechanisms simultaneously: (1) multitask training with additional program-property supervision and (2) property-guided token commitment ordering during inference. However, the current experimental results suggest that a substantial portion of the improvement may come from the additional supervision rather than the proposed decoding strategy. Specifically, Table 4 shows that Sec@5 improves from 33.33% (Finetuned) to 50.00% (PACO with the AST head disabled). However, enabling the AST head does not further improve Sec@5, which remains at 50.00%. Similarly, Func-Sec@5 improves from 16.67% to 29.17% with the head disabled, and further to 33.33% when the head is enabled. These results suggest that multitask training contributes substantially to the overall improvement.
-
I recommend adding several controlled experiments to better isolate and validate the effectiveness of the proposed decoding strategy.
-
Random commitment ordering: Randomly determine the token commitment order to examine whether the improvements arise from property-aware scheduling rather than changes to the decoding order in general.
-
Reversed AST ordering: Prioritize deeper AST tokens instead of shallower ones to evaluate whether the proposed structure-first principle is essential.
-
Random risk-token delay: Delay randomly selected tokens rather than predicted security-sensitive tokens to verify the effectiveness of the learned security-risk signal.
Questions for authors
See "Critical Weaknesses"
Dear Area Chair,
Thank you for your time and effort in overseeing our submission. As the discussion period concludes, we would like to briefly summarize our new experiments, manuscript revisions, and the current status of the discussion.
All three reviewers appreciated the motivation of using program properties to guide diffusion decoding for secure code generation, and raised a shared set of constructive questions. Below is a high-level summary of how we addressed them in our responses and the revised PDF:
Qwen2.5-Coder-7Bfine-tuned on the same training data andSafeCodertrained on the same diffusion backbone, clarified the effect of base-model initialization, and evaluated 7B models on BaxBench to examine current model-scale limits on full-application generation.Thank you again to you and the reviewers for the constructive feedback, which has helped us make the paper much stronger and clearer.
Best regards, The Authors