- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents CoEvo-SQLi, a framework for improving SQL injection detection through iterative training and feedback-guided attack generation. It combines attack synthesis, LLM-based payload mutation, and adaptive sampling based on the detector's validation errors. The authors also introduce SQLI-BENCH, a schema-grounded benchmark covering 48 injection clusters. The proposed method achieves better results than static fine-tuning, and the experiments include ablation studies and evaluations on external datasets. The work is practically relevant and generally well presented, although some aspects of the method and evaluation could be strengthened.
Critical strengths
The paper addresses the challenge of detecting evolving SQL injection attacks. The idea of using detection errors to guide further training is reasonable and well aligned with the problem.
SQLI-BENCH provides a structured evaluation of different attack types. Separating databases, queries, and payload templates across data splits is a strength of the experimental design.
The ablation studies help explain the contributions of feedback and mutation, while the external evaluations and efficiency measurements provide additional evidence of its practical potential.
Critical weaknesses
The framework combines several established techniques, including adaptive sampling, multiplicative weight updates, LLM-based data augmentation, and iterative fine-tuning. While the combination is useful, it is not entirely clear what new learning capability it offers beyond existing approaches. Comparisons with stronger adaptive sampling and adversarial training baselines would help clarify its contribution.
Attack generation relies on predefined templates and constrained LLM mutation. These constraints help maintain consistency but may limit the diversity of generated attacks. The paper does not fully establish whether the generated SQL queries represent effective attacks in real database environments. A comparison with direct LLM-based attack generation would also help justify the proposed design.
The method repeatedly uses the same validation set to adjust sampling weights, which may introduce bias toward particular attack patterns. Although the authors evaluate external datasets, these experiments only cover the query-only variant, and some improvements over Static SFT are not statistically significant. Further analysis of training stability and performance under different validation splits would be helpful.
There are a few minor inconsistencies between the main tables and appendix. For example, the TPR of deepseek-v4-flash is 60.0% in Figure 4 but 60.1% in Table 6. Similar differences appear between Tables 1 and 7. Check these values and ensure consistent reporting throughout the paper.
Questions for authors
How much does continuously updating the attack-cluster weights improve detection performance compared with calculating the weights once after the first training round and keeping them fixed for subsequent rounds, while maintaining the same training budget and other settings?
What are the main advantages of the proposed template-based, schema-grounded attack generation approach over direct LLM-based attack generation, particularly in terms of attack validity, diversity, and downstream detection performance?
3 comments
Response to Q2 (Part 2 of 3)
Q2. What are the main advantages of the proposed template-based, schema-grounded attack generation approach over direct LLM-based attack generation, particularly in terms of attack validity, diversity, and downstream detection performance?
We agree that a complete end-to-end comparison with unconstrained LLM generation would be valuable. However, we did not add such a baseline in this revision because we tried it early and eventually abandoned it. We therefore do not claim that template-based generation has been shown to outperform unconstrained generation in downstream detection. The specific reasons are as follows: First, unconstrained generation often produces samples with syntax errors, missing placeholders, or references to nonexistent tables or columns. Such samples cannot pass the acceptance checks; retrying them is substantially more costly than mutation under template constraints, and they may also teach incorrect decision cues. Second, unconstrained generation has a diversity problem: its outputs may differ substantially at the surface level while repeating the same underlying patterns, and most candidates are discarded after applying the requirement that they differ from previously accepted mutations. Finally, in downstream detection, schema-bound generation produces TSR payloads. This is the point rather than a defect. The samples refer to target tables, columns, and aliases, and the text alone cannot determine whether these objects actually exist; the label must be determined using the schema. This mode itself provides evidence of schema awareness.
We therefore added two new data-augmentation baselines that share the budget with CoEvo. The first is WAF-A-MoLE augmentation, which uses the same Qwen2.5-Coder-0.5B-Instruct backbone, fine-tuning configuration, and training slots as the CoEvo-SQLi query-only variant, allowing a comparison of different data-generation strategies. The second is GAN+RL, which jointly optimizes a generator and detector during training and is a generative baseline closer to a jointly trained attacker. Because this method uses its native generator-detector architecture, we treat it as a broader generative-training comparison rather than a same-backbone comparison. The query-only comparison on SQLI-BENCH is as follows:
Setting: query-only; SQLI-BENCH
| Method | MCC | TPR (%) | FPR (%) | Worst-cluster recall (%) |
|---|---|---|---|---|
| Static SFT | 77.1 | 87.8 | 1.0 | 25.6 |
| WAF-A-MoLE augmentation | 71.9 | 83.9 | 1.2 | 16.8 |
| GAN+RL | 79.3 | 94.6 | 13.7 | 57.4 |
| CoEvo-SQLi | 81.5 | 91.0 | 2.0 | 46.9 |
WAF-A-MoLE augmentation uses the same Qwen2.5-Coder-0.5B-Instruct backbone, fine-tuning configuration, query-only input, and training-slot budget as the CoEvo-SQLi query-only variant. CoEvo-SQLi improves MCC by 9.6 points and worst-cluster recall by 30.1 points. GAN+RL has higher TPR and worst-cluster recall, but its FPR reaches 13.7%. In addition, CoEvo-SQLi achieves better performance on the other three public datasets; detailed results are provided in Appendix F.
Other revisions
We added stronger adaptive-sampling and adversarial-training baselines. For adaptive sampling, instantaneous error-proportional resampling replaces the original accumulated, centered multiplicative-weights update with sampling proportional to the current-round validation error, while keeping the backbone, mutation procedure, benign-ratio controller, number of training rounds, training budget, and available validation information unchanged. The schema-grounded results are:
Setting: schema-grounded; adaptive-sampling comparison
| Method | MCC | TPR (%) | FPR (%) |
|---|---|---|---|
| Instantaneous error-proportional resampling | 87.9 | 95.0 | 2.9 |
| Full CoEvo-SQLi | 92.5 | 97.5 | 3.2 |
(This response continues in Part 3 of 3, supplied as a separate file: the adversarial-training comparison, the training-time mutation audit, the validation-set analyses, the external-transfer results, and the numerical corrections.)
Response to other weakness; Revisions (Part 3 of 3)
For adversarial training, we reimplemented ModSec-AdvLearn in its native rule-derived feature space and classifier. The query-only results are:
Setting: query-only; adversarial-training comparison
| Method | MCC | TPR (%) | FPR (%) |
|---|---|---|---|
| L1-SVM + adversarial training | 43.0 | 71.5 | 20.2 |
| SecSVM + adversarial training | 41.8 | 82.4 | 36.9 |
| CoEvo-SQLi (query-only) | 81.5 | 91.0 | 2.0 |
The MCC of instantaneous error-proportional resampling is 87.9, whereas the MCC of the full system is 92.5, a decrease of 4.6 points. This shows that the contribution of CoEvo-SQLi is not limited to selecting the currently hardest attack clusters; as the detector state changes, accumulated validation errors across rounds provide additional information. The L1-SVM and SecSVM variants of ModSec-AdvLearn achieve MCC values of 43.0 and 41.8, respectively, under the query-only contract, substantially below the MCC of 81.5 for the CoEvo-SQLi query-only variant.
We also audited the 240 accepted mutations actually used during training. These samples cover three seeds and eight rounds. Two annotators with SQL-security backgrounds judged the samples independently, a third annotator resolved disagreements, and every query was executed on the target MySQL database.
Setting: training-time mutation audit
| Audit criterion | Result |
|---|---|
| Intended attack technique preserved | 227/240 (94.6%) |
| Full semantic validity | 226/240 (94.2%) |
| Schema-referencing examples consistent with the schema | 153/155 (98.7%) |
| Clear malicious/benign judgement with SQL and schema | 238/240 (99.2%) |
| Structural breakage or unparseable | 9/240 (3.8%) |
These results indicate that most of the generated SQL queries can constitute valid attacks in a real database environment, and they also support the validity and controllability of the constrained pipeline.
We also analyzed the risk that repeatedly using the same validation set could cause the model to overemphasize attack patterns that are particularly visible in that set. We reduce this risk in three ways:
- feedback-driven sampling is mixed with uniform exploration, so every attack cluster retains a nonzero sampling probability in every round;
- the training, validation, and test partitions are disjoint in databases, carrier queries, payload templates, and comment resources;
- mutation changes payload forms or embedded SQL structures rather than simply repeating validation queries.
We also tested sensitivity to validation-set size. The validation-set-size analysis is as follows:
Setting: schema-grounded; validation-set size
| Validation size | FPR (%) | FPR excess over Static SFT (percentage points) | TPR (%) | MCC |
|---|---|---|---|---|
| 40 examples per cluster + 40 benign | 3.2 | 1.9 | 97.5 | 92.5 |
| 80 examples per cluster + 80 benign | 3.0 | 1.7 | 97.0 | 91.5 |
| 160 examples per cluster + 160 benign | 2.6 | 1.3 | 97.2 | 92.2 |
After enlarging the validation set, the main detection gain remains, but we do not claim that this analysis establishes robustness to all possible validation splits.
Finally, the query-only detector performs comparatively well when transferred to three external datasets without additional adaptation. We therefore interpret these external results as evidence of positive transfer on independently collected datasets:
Setting: query-only; external transfer
| Dataset | Static SFT MCC | CoEvo-SQLi MCC |
|---|---|---|
| rbsqli | 92.5 | 93.1 |
| SQLiV3 | 65.8 | 77.1 |
| WAF-A-MoLE | 84.4 | 87.9 |
The external experiments use query-only inputs because the comparison methods and public benchmarks generally do not provide schema inputs. We quantify the contribution of schema context through paired schema-grounded and query-only experiments on SQLI-BENCH.
The numerical inconsistencies identified by the reviewer were our mistake. In the revision, all tables were regenerated from the same artifacts and checked cell by cell. The two differences identified by the reviewer, as well as other last-digit differences of the same type, have been corrected. We also added discussion of the non-significant results.
The revisions include the frozen-controller experiment in Appendix G; the mutation audit in Appendix H; the validation-size analysis in Appendix I; the low-FPR operating-point analysis in Appendix J; the complete comparator protocols in Appendices K and E.4; and the corresponding explanations in Sections 5.1, 5.3, 5.4, and 7.
Response to Q1 (Part 1 of 3)
Thank you for your careful reading and constructive comments. We first answer the two questions you raised and then summarize the other revisions.
We added a frozen-controller experiment in Appendix G. The backbone, fine-tuning configuration, eight-round training schedule, total training budget, validation set, and benign-ratio controller remain unchanged. The only change is that the cluster weights are computed from the first-round validation errors and then held fixed in rounds 2–8.
Setting: schema-grounded; frozen-controller comparison
Freezing the cluster weights after round 1 makes MCC 13.1 points lower than the full system and 2.6 points lower than uniform sampling with mutation. The first-round error distribution can only reflect a local state of detector difficulty. Once the initially difficult clusters become easier through training, other clusters may become the main source of later validation errors. Therefore, fixing the first-round weights allocates the subsequent training budget according to an outdated error profile.
To distinguish accumulated credit assignment from simple hard-example selection, we also replace the accumulated update with instantaneous error-proportional resampling.
Setting: schema-grounded; cluster-selection comparison
The instantaneous rule uses only the validation errors from the current round, whereas the accumulated controller retains information across rounds. The 4.6-point MCC difference indicates that, beyond selecting currently difficult clusters, the contribution of CoEvo-SQLi is also related to cross-round credit assignment.
(This response continues in Part 2 of 3, supplied as a separate file: Q2 on template-based versus direct LLM-based attack generation, the other revisions, and the adaptive-sampling comparison.)