acceptodds
Under review as a conference paper at ICLR 2027

ACE-Guard: Automated Co-Evolutionary Guardrails for LLM Security via Inference-Time Natural Language Gradients

Abstract

Prompt injection attacks pose a persistent threat to Large Language Model (LLM) deployments by embedding malicious instructions that override system controls. Existing defenses rely on static rules or require costly model fine-tuning, leaving them exposed when adversarial strategies shift over time. We present ACE-Guard, an automated co-evolutionary security framework that adapts defensive policies entirely without modifying model parameters (). ACE-Guard operates through a dual-phase architecture: an offline co-evolutionary calibration phase that models security adaptation as a repeated game among an Adversary LLM exploring nine structural injection strategies, a Defense LLM equipped with a deterministic Multi-Signal Prompt Inspector (MSPI), and an Evaluation LLM providing bidirectional natural language gradients, followed by an online pre-execution serving phase that enforces safety in a single forward pass. Across 20 co-evolutionary calibration rounds, ACE-Guard achieves an average Defense Success Rate of 83.4% and a Benign Acceptance Rate of 91.9%, yielding a Balanced Harmonic Score () of 87.4%. Extensive evaluations across out-of-distribution attack benchmarks confirm that natural language prompt steering maintains long-term stability, resists adversarial mode collapse, and achieves robust runtime protection without retraining overhead.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
ACE-Guard, prompt injection, large language models, adversarial co-evolution, natural language gradients, LLM safety

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VK9YOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ACE-Guard, a prompt injection defense framework that combines a rule-based Multi-Signal Prompt Inspector (MSPI) layer, an LLM-based forensic layer (MSPI Layer 2), and a Defense Arbiter. During offline calibration, adversarial generation and evaluation feedback are used to update attack strategies, defensive prompts, and decision thresholds while keeping the underlying LLM weights frozen. The resulting guard screens inputs before downstream execution. The paper evaluates this framework across nine attack streams and reports component ablations and a comparison with continual LoRA fine-tuning.

Critical strengths
  1. Relevant problem and practical deployment motivation. Separating offline adversarial exploration from online filtering is a reasonable design for deployments where model weights cannot be modified and runtime overhead matters.
  2. Explicit consideration of the security–utility trade-off. Reporting both defense success rate and benign acceptance rate recognizes that indiscriminate blocking is insufficient. The range of attack streams and the inclusion of round-by-round results provide useful material for examining robustness and adaptation.
Critical weaknesses
  1. The ablations do not isolate the contributions of prompt evolution or MSPI. The comparison against "without Eval" does not distinguish natural-language prompt evolution from threshold calibration, attack-distribution changes, or generic feedback-based tuning. The paper itself attributes the advantage of the full system partly to calibrating thresholds (Section 3.3) and notes that, without evaluation feedback, the adversarial agent fails to explore sophisticated mutations (Appendix E.3), so the two variants also face different attack distributions. Furthermore, the MSPI ablation changes the classifier's task instructions and output requirements, as shown in Appendix B.2, in addition to removing forensic augmentation. Consequently, the reported differences do not cleanly identify which components produce the gains. A controlled comparison on a common frozen test set, with consistent classification instructions and inference budgets, is needed to substantiate the contribution of the proposed optimization mechanism.
  2. The reported classification metrics contain substantial internal inconsistencies. Under the stated evaluation setting of 100 adversarial and 100 benign examples per round (Section 2.3), Table 10, round 4, reports DSR = 0.860 and FPR = 0.360 for the variant without MSPI. These imply TP = 86, FN = 14, FP = 36, and TN = 64, yielding Macro F1 ≈ 0.747, whereas the table reports 0.2692. Similarly, Table 9, round 6 (full ACE-Guard; DSR = 0.920, FPR = 0.200), implies Macro F1 ≈ 0.859 rather than the reported 0.667. Some reported rates are not multiples of 0.01, so the actual per-round counts may differ slightly from 100/100, but the gap persists for any plausible class ratio, and Algorithm 4 computes Macro F1 from the same TP, FP, TN, and FN as DSR and FPR. In addition, Table 4 is described as averaging 10 rounds, yet its values match the 20-round Mean rows of Tables 9 and 10, and several Mean rows do not equal the averages of their own columns. These discrepancies directly affect the central claim that removing MSPI causes a large F1 collapse. The metric definitions, underlying predictions, and aggregation procedure need to be reconciled before the ablation conclusions can be assessed reliably.
Questions for authors
  1. With the guard architecture and evaluation set held fixed, how much improvement comes from defensive prompt evolution beyond threshold-only calibration? Can this be established while keeping the classification instructions and output parser consistent?
  2. What does the prompt transformation operator T\mathcal{T} (Eq. 10) concretely do? Please provide a representative before-and-after defensive prompt, the associated feedback, and the final deployed prompt, distinguishing prompt edits from threshold and exemplar updates.
  3. Could you provide the per-round confusion matrices and metric computation script for Tables 9 and 10, explain the discrepancies above, and clarify how the averages in Table 4 relate to the 20-round trajectories? Are DSR, FPR, and Macro F1 computed using the same examples and labels?

I'll consider raising score if my questions are addressed.

2 comments

Author Response

We sincerely thank the reviewers for their thoughtful comments and constructive suggestions. We have carefully revised the manuscript to improve its clarity, methodological consistency, and reproducibility. We provide our point-by-point responses below and welcome further feedback during the ICLR 2027 discussion period.

Official Review of Submission 33053 by Reviewer VK9Y

Critical Weaknesses

1. The ablations do not isolate the contributions of prompt evolution or MSPI

Response: We clarified that the full framework, the variant without MSPI, and the variant without defense feedback were evaluated using three independent calibration streams, each with 20 rounds, rather than a shared fixed test set. Furthermore, the variant without MSPI used different input wrappers and output instructions, introducing additional variations beyond MSPI removal. Therefore, these results provide descriptive comparisons rather than establish the independent causal contributions of individual components. We revised the discussion, removed unsupported causal and statistical-significance claims based on paired comparisons, and acknowledged the need for controlled evaluation.

Changes: Section 3.3 (Ablation and Component Analyses), Table 5, and Tables 15 and 16 clarify the independent calibration streams and the need for evaluation on a controlled frozen test set.

2. Reported classification metrics contain substantial internal inconsistencies

Response: We clarified that DSR measures target-model resistance, whereas positive-class binary F1 and FPR evaluate defense-classifier predictions. We also added per-round confusion-matrix counts for verification.

Changes: Section 3.1 (Evaluation Metrics) and Appendix I (Tables 13 and 14).

Questions for Authors

1. With the guard architecture and evaluation set fixed, how much improvement comes from defensive prompt evolution beyond threshold-only calibration? Can this be established with consistent instructions and parsers?

Response: The current ablation study does not independently isolate the effects of defense-prompt evolution and threshold calibration. Their individual contributions therefore cannot be determined reliably. A rigorous comparison requires a shared fixed test set, consistent input instructions, output parsers, and evaluation protocols across configurations. We acknowledge this limitation and identify such controlled experiments as future work.

2. What does the prompt transformation operator (Equation 10) concretely do? Please provide before-and-after prompts, associated feedback, and the final deployed prompt.

Response: The operator combines FN/FP correction examples with recommendations from the Evaluation LLM, updates the defense system prompt, and retains historical snapshots. The updated prompt is then inserted into the runtime classifier. Representative before-and-after excerpts, including a Round 20 correction example, are provided in the Appendix.

Changes: Section 2.3 Stage 5 and Appendix E (Defense System-Prompt Evolution, Table 10).

3. Could you provide per-round confusion matrices and the metric computation procedure? Are DSR, FPR, and Macro F1 calculated using the same examples and labels?

Response: We clarified that these metrics evaluate different tasks and do not use the same labels. DSR is calculated from target-model responses to adversarial prompts as DSR = 1 − ASR, where ASR is the Attack Success Rate. F1-score and FPR evaluate the defense classifier using original_label as ground truth and defense_label as prediction. We also corrected the metric description: the implementation computes positive-class binary F1, not Macro F1. Per-round TP, FP, TN, and FN counts are provided to enable independent verification.

Changes: Section 3.1 (Evaluation Metrics) now distinguishes the two evaluation tasks. Appendix I (Tables 13 and 14) provides the updated 20-round calibration results and confusion matrices. Table 5 summarizes ablation results, with per-round trajectories in Tables 15 and 16.

Thank you for the detailed response and revisions. The added prompt examples clarify the update procedure, and the revised binary F1 and FPR values for the full system are consistent with the supplied confusion matrices. I appreciate these corrections and the explicit acknowledgment of the limitations of the ablation design.

However, my main concern remains unresolved. The independent calibration streams, together with the different input wrappers and output instructions in the no-MSPI configuration, prevent the reported comparisons from isolating the contributions of individual components. In particular, the current experiments do not establish whether defensive prompt evolution improves performance beyond threshold-only calibration. Narrowing the claims is appropriate, but a controlled comparison on a shared frozen test set, with consistent instructions, parsers, and inference budgets, is still needed to substantiate this central contribution.

I also have a remaining question about the revised DSR definition. Section 3.1 describes DSR as measuring post-defense target-model resistance. However, Algorithm 4 appears to obtain target-model outcomes before defense classification, and Algorithm 3 computes ASR from those outcomes without explicitly incorporating the guard’s admission decision. Could the authors clarify how blocked prompts enter this calculation and provide the exact guard-conditioned attack-success definition? As written, it is unclear whether this metric measures the guarded pipeline or the target model’s own resistance.

Finally, the revised confusion matrices establish consistency for the updated full-system results, but the relationship between the original and revised tables remains unclear. Please distinguish corrections to metric definitions from recomputation or new experimental runs, and provide the corresponding counts and metric computation code for the ablation configurations.

Overall, the revisions improve transparency and methodological clarity, but the evidence needed to establish the proposed optimization mechanism’s contribution is still missing. I therefore maintain my current recommendation of weak rejection.

Reviewer_FHJLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a co-evolutionary offline security framework for creating defensive policies based on interaction between an adversary, defense and evaluation LLM, with malicious and benign prompts from existing jailbreak datasets as input. After some rounds of offline evolution of the adversary and defense LLM, the defense LLM is deployed as a pre-filter in production, taking as input a mixture of text context and deterministic feature vectors to classify inputs as malicious. The method is evaluated against existing attack and defense approaches, showing good performance and efficiency. A full performance analysis and ablation of the various components of the system is also presented.

Critical strengths
  • The method presented seems to be more expressive than the presented baselines, with good performance on the settings evaluated and low latency. ACE-Guard appears to be more robust to token optimization and adaptive mutation attacks than baselines as well.
Critical weaknesses
  • My main concern with this paper is the excessive use of seemingly arbitrary hardcoded hyperparameters in the method, without any ablations or adequate explanation. The design of the offline phase in particular is quite poorly motivated. For example, how was the MSPI arrived at? How were the strategies in Stage 1 decided? Is the annotation and evaluation LLM calibrated against human preferences?
  • There is a lot of crucial detail missing from the paper. The most important stage, the defense prompt transformation operator, is not described at all as well. How are the adversarial and benign control prompts sampled? How do the authors ensure that the benign prompts are not random, but actually function as a contrastive control against the adversarial prompts?
  • The paper also suffers from presentation issues. The introduction is meandering and unfocused, and does not clearly motivate ACE-Guard. It is unclear to me what issues with existing approaches necessitates this approach. The paper also doesn't compare ACE-Guard to SelfDefend or DuoGuard, which it explicitly regards as motivating approaches. Table 2 and 3 are also very confusing due to the presence of "Ours" settings in both the rows and columns. The harmonic score ultimately used to judge performance isn't even presented in the table.
Questions for authors
  • Does the method to generalize to classes of attacks outside the 9 generated by the Attack LLM?
  • Do the authors have any guarantees that their evaluator LLMs are well-calibrated against human preferences?
  • Since MSPI explicitly discounts risk based on identified deterministic benign patterns, is it possible for attackers to game this system by stuffing their prompts with lots of positive words such as please, summarize, and translate?
2 comments

Author Response

We sincerely thank the reviewers for their thoughtful comments and constructive suggestions. We have carefully revised the manuscript to improve its clarity, methodological consistency, and reproducibility. We provide our point-by-point responses below and welcome further feedback during the ICLR 2027 discussion period.

Official Review of Submission 33053 by Reviewer FHJL

Critical Weaknesses

1. Excessive use of seemingly arbitrary hardcoded hyperparameters / offline phase poorly motivated / human calibration

Response: We clarified that the seven attack strategies, strategy-weight update factors, MSPI signals, and benign-pattern offset were selected using implementation heuristics, rather than systematic hyperparameter search or formal optimization. We also detailed the annotation process: generated annotations were manually inspected by one author, without independent second-annotator verification or formal human-preference calibration. We acknowledge these limitations and avoid overstating the empirical justification of the selected configurations or annotation reliability.

Changes: Section 2.3 Stage 1 (strategy-weight updates), Stage 2 (single-annotator limitation), Stage 3, and Appendix A.2 (MSPI signals and constants, Table 7).

2. Crucial detail missing: defense prompt transformation operator, sampling of adversarial and benign control prompts

Response: We clarified the defense-prompt update procedure. False negatives (FNs) receive a target score of 9.5, while false positives (FPs) receive 1.5. The update also incorporates general recommendations from the Evaluation LLM. The three most recent prompt snapshots are retained, and the updated system_prompt is inserted into subsequent runtime requests to guide defense-classifier predictions.

Benign control samples are drawn from dedicated templates covering coding, writing, question answering, translation, and summarization. We also acknowledge that some benign-looking false-negative samples may conflict with the Placid Presumption principle in the base prompt. This prompt-label inconsistency is a limitation that may affect evaluation reliability and is now explicitly discussed.

Changes: Section 2.3 Stage 5 (prompt updates and runtime insertion) and Appendix E (Defense System-Prompt Evolution, Table 10), including snapshot counts, base-prompt excerpts, and correction examples.

3. Presentation issues, meandering introduction, missing comparisons to SelfDefend/DuoGuard, confusing Tables 2 and 3

Response: We reorganized the introduction and clarified the evaluation matrix. ACE-Guard (Ours) refers to the defense method, whereas ACE-Guard co-evolved attacks refer to the attack source. We added SelfDefend DSR/BAR results and balanced harmonic scores. Numerical DuoGuard comparisons are omitted because corresponding evaluation data are unavailable.

Changes: Section 1 (Introduction) and Tables 2–4, including their captions and analysis.

Questions for Authors

1. Does the method generalize to attack classes beyond the nine generated by the Attack LLM?

Response: Evaluated external and co-evolved prompts are mapped to the seven predefined strategy categories (Table 8). This demonstrates coverage under the proposed taxonomy but does not establish generalization to unseen attack-strategy classes.

2. Are the evaluator LLMs calibrated against human preferences?

Response: Annotations were manually inspected by one author, without inter-annotator agreement measurements or formal human-preference calibration studies. We explicitly acknowledge this limitation.

3. Can attackers exploit the system by stuffing prompts with positive words (e.g., "please," "summarize," or "translate")?

Response: The rule-based layer reduces the risk score by 12 points for each matched benign pattern. However, we have not conducted a dedicated evaluation of adaptive benign-word-stuffing attacks. We therefore acknowledge this as a potential attack surface and future research direction.

I thank the authors for their transparency in the rebuttals, but they propose no new experiments addressing the methodological concerns raised by me. Therefore, I retain my recommendation for rejection, as I do not think the evaluation of the proposed methods is strong enough without them.

Reviewer_tdDJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes ACE-Guard, a dual-phase defense framework against prompt injection attacks on LLMs. The framework separates guardrail adaptation into an offline "co-evolutionary" calibration loop, modeled as a repeated game among an Adversary LLM, a Defense LLM, and an Evaluation LLM using frozen weights, and an online runtime gate. A core component is the Multi-Signal Prompt Inspector (MSPI), a 7-dimensional regex/heuristic extractor designed to provide structural features alongside semantic LLM judgments. The authors evaluate the pipeline across 9 attack benchmarks using Llama-3.1-8B-Instruct, reporting an average Defense Success Rate (DSR) of 83.4% and a Benign Acceptance Rate (BAR) of 91.9%.

Critical strengths
  1. Relevant and practically motivated problem: Developing adaptive prompt injection defenses without requiring repeated model fine-tuning is an important direction, particularly in deployment settings where model parameters are inaccessible or computational resources are limited.

  2. Potentially useful system design: Separating offline adversarial calibration from online inference is a reasonable architectural choice. Combining deterministic structural signals with semantic LLM-based classification could also be useful for reducing false positives caused by benign but syntactically unusual queries.

  3. Broad experimental scope: The manuscript attempts to evaluate multiple attack families, compares several existing defenses, and includes component ablations, trajectory analyses, and computational overhead measurements.

Critical weaknesses

1. Major mathematical inconsistencies in the reported classification metrics:

My most serious concern is that the reported Macro F1 scores in Tables 4, 9, and 10 are inconsistent with the reported DSR and FPR under standard binary classification definitions. Section 3.1 defines DSR as TP/(TP+FN), BAR as TN/(TN+FP), and FPR as 1−BAR. With adversarial inputs as the positive class, standard binary Macro F1 is

F1,macro=12(2TP2TP+FP+FN+2TN2TN+FP+FN)F_{1,\mathrm{macro}} = \frac{1}{2}\left(\frac{2TP}{2TP+FP+FN} + \frac{2TN}{2TN+FP+FN}\right)

The manuscript also states that each calibration round uses 100 benign and 100 adversarial queries (Section 2.3). Under this balanced evaluation, DSR and FPR determine the confusion matrix and therefore the Macro F1 score. For example:

  • Table 10, Without MSPI, Round 7: DSR = 0.895 and FPR = 0.410 imply Macro F1 ≈ 0.735 on a balanced dataset, whereas the reported value is 0.2264.
  • Table 10, Without MSPI, Round 4: DSR = 0.860 and FPR = 0.360 imply Macro F1 ≈ 0.750, whereas the reported value is 0.2692.
  • Table 9, Round 13: DSR = 0.750 and FPR = 0.390 cannot yield the reported Macro F1 = 0.736 even if class prevalence is allowed to vary. The theoretical maximum is approximately 0.681.

More seriously, at least 13 of the 20 Without MSPI rounds in Table 10 report Macro F1 scores below the mathematically possible minimum for their respective DSR and FPR values, regardless of class balance.

The discrepancies also propagate to Table 4. For example, using the average DSR = 0.857 and FPR = 0.364 for the Without MSPI configuration yields a balanced Macro F1 of approximately 0.743, substantially different from the reported 0.372. Although averaging nonlinear metrics across rounds can introduce differences, this cannot explain the individually impossible results in Table 10. There are also discrepancies in the reported aggregate statistics. For example, the arithmetic mean of the 20 DSR values in Table 9 is approximately 0.829, rather than the reported 0.834.

These inconsistencies directly affect the main ablation conclusions. In particular, the paper argues that removing MSPI causes Macro F1 to collapse from 0.719 to 0.372, which is used as central evidence of MSPI's effectiveness. However, the reported F1 measurements cannot be reconciled with the other classification metrics. Without resolving these discrepancies, I cannot assess the validity of the principal experimental findings.

2. Methodological inconsistencies and reproducibility concerns: There seems to be several inconsistencies between different parts of the method description.

First, Equation 9 defines the fast-path rejection condition using the structural MSPI risk score, whereas Algorithm 3 appears to apply the corresponding threshold to the LLM-derived danger score. These are different decision rules and can produce different classification outcomes.

Second, the hyperparameter specifications are not fully consistent. For example, Section 2.3 and Appendix A describe benign-signal mitigation using a coefficient of 0.8, while Table 8 lists a benign mitigation factor of −12.0 × N_benign. It is unclear whether these correspond to different scoring stages or incompatible implementations.

Third, the operational description emphasizes single-pass online inference, but Table 5 specifies two sequential LLM invocations: an MSPI forensic pass and a defense arbitration pass. Although two LLM calls can share one model instance, the paper should clearly distinguish single-model deployment from single-forward-pass inference.

Fourth, Table 4 is described as averaging 10 calibration rounds, while Appendix E uses 20-round trajectories and statistical tests to support the corresponding ablation conclusions. The relationship between these experiments is not explained.

These inconsistencies make it difficult to reconstruct the actual implemented framework or determine which version of the decision rule produced the reported results.

3. Inaccurate citations and characterization of related work: Several references contain incorrect bibliographic metadata. Most notably, the citation to ICON (arXiv:2602.20708), a major experimental baseline, lists an incorrect title and first author. Furthermore, the related-work discussion characterizes DuoGuard (arXiv:2502.05163) as involving online multi-agent deliberation, whereas the original method uses adversarial reinforcement learning for synthetic-data generation and multilingual guardrail training. Since these comparisons help motivate the proposed framework's novelty and efficiency, the inaccuracies raise concerns about the fidelity of the literature review.

Questions for authors
  1. Can the authors provide the TP, TN, FP, and FN values for each round in Tables 9 and 10 and the exact code used to compute Macro F1? How can the mathematically inconsistent entries be explained?

  2. Why does Table 4 describe averages over 10 rounds, while Appendix E describes 20-round ablation trajectories? Are these separate experiments?

  3. What improvement does adaptive adversarial generation provide over prompt optimization using fixed attack examples, given matched training and evaluation budgets?

1 comment

Author Response

We sincerely thank the reviewers for their thoughtful comments and constructive suggestions. We have carefully revised the manuscript to improve its clarity, methodological consistency, and reproducibility. We provide our point-by-point responses below and welcome further feedback during the ICLR 2027 discussion period.

Official Review of Submission 33053 by Reviewer tdDJ

Critical Weaknesses

1. Major mathematical inconsistencies in the reported classification metrics

Response: We clarify the distinction between the two evaluation tasks. Defense Success Rate (DSR) measures the defense's resistance to adversarial attacks and is defined as DSR=1−ASR\mathrm{DSR}=1-\mathrm{ASR}, where ASR is the Attack Success Rate. In contrast, F1-score and False Positive Rate (FPR) evaluate the defense classifier, using original_label as the ground truth and defense_label as the prediction. We also clarify that our implementation computes positive-class binary F1, not Macro F1. Per-round TP, FP, TN, and FN counts are provided to enable independent verification.

Changes: Section 3.1 (Evaluation Metrics) clarifies the metric definitions. Appendix I provides the updated 20-round calibration results and confusion matrices (Tables 13 and 14). Ablation results are summarized in Table 5, with per-round trajectories in Tables 15 and 16.

2. Methodological inconsistencies and reproducibility concerns

Response: We clarified the following methodological details:

  • The high-danger floor and danger threshold are applied to the combined forensic danger score, while the confidence condition is derived from the arbitration score.
  • Each matched benign pattern subtracts 12 points from the rule-based risk score before clipping it to [0,100][0,100]. This offset is separate from the MSPI subscore.
  • Online inference performs two sequential LLM calls: forensic analysis followed by defense arbitration. These are separate inference calls, even if they share the same model instance.
  • All calibration and ablation experiments contain 20 rounds. The outdated reference to 10 rounds has been corrected.

Changes: Equations 7 and 11, Section 2.2, Section 2.3 (Stage 3), Algorithm 5, and Appendix A.2 and I.

3. Inaccurate citations and characterization of related work

Response: We revised the descriptions of SelfDefend and DuoGuard to better reflect their methods, including DuoGuard's two-player adversarial reinforcement learning framework, and corrected the ICON citation metadata. Available SelfDefend results are included in the cross-benchmark comparison. We omit numerical comparisons with DuoGuard because corresponding evaluation data are unavailable. We also clarify the limitations of protocol comparability.

Changes: Section 1 (Introduction and Related Work) and Tables 2–4.

Questions for Authors

1. Can the authors provide per-round TP, TN, FP, and FN values and the exact code used to compute Macro F1? How can the mathematically inconsistent entries be explained?

Response: Our implementation computes positive-class binary F1, not Macro F1. Label 1 is positive and label 0 is negative. Given ground-truth labels y and predictions ŷ, the metrics are:

Confusion matrix:

TP = samples with y = 1 and ŷ = 1.

FP = samples with y = 0 and ŷ = 1.

TN = samples with y = 0 and ŷ = 0.

FN = samples with y = 1 and ŷ = 0.

Evaluation metrics:

Precision = TP / max(TP + FP, 1)

Recall = TP / max(TP + FN, 1)

F1 = 2 × Precision × Recall / max(Precision + Recall, 10⁻⁹)

FPR = FP / max(FP + TN, 1)

Thus, the reported F1 is positive-class binary F1, not macro-averaged F1. Per-round confusion-matrix counts are provided in Appendix I.2 (Table 14) for verification.

Changes: Appendix I.2, Table 14.

2. Why does an earlier table description refer to 10 rounds, while the appendix reports 20-round ablation trajectories? Are these separate experiments?

Response: The reference to 10 rounds was a typographical error in an earlier draft, not a separate experiment. All reported calibration and ablation experiments contain 20 rounds. Table 5 summarizes the ablation results, while Tables 13, 15, and 16 report the corresponding calibration and ablation trajectories.

Changes: Section 3.3, Table 5, and Appendix I (Tables 13, 15, and 16).

3. What improvement does adaptive adversarial generation provide over prompt optimization using fixed attack examples under matched training and evaluation budgets?

Response: Our current experiments do not include a matched-budget comparison with fixed-example prompt optimization. Therefore, we cannot isolate the independent contribution of adaptive adversarial generation. We have narrowed our claims accordingly and identified this comparison as future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.