- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AAPO, an inference-time defense that uses attacker–judge–defender interactions to generate and select defensive prompts without modifying model parameters. The method combines reactive guard generation with an Exp3-style selection policy and evaluates forgetting and utility on RWKU and WMDP.
Critical strengths
-
The framework is practical and compatible with existing unlearning methods. AAPO operates through system prompts without updating model parameters. Table 1 shows consistent reductions in forget-set scores across two model families and four unlearning methods.
-
The comparison with simpler prompt defenses is useful. Table 6 includes length-matched prompts, prompt concatenation, iterative refinement, and filtering. AAPO improves both forgetting and neighbor retention over iterative refinement, supporting the value of adaptive prompt orchestration.
Critical weaknesses
-
The prompt-selection update is insufficiently justified. Equation (14) divides the shared reward by (p_i), but the WDGS rule in Equation (12) selects multiple prompts without establishing that (p_i) represents their actual inclusion probabilities. This leaves the connection between prompt selection and the Exp3-style reward update unclear.
-
The related-work discussion overlooks adversarial prompt optimization. Robust Prompt Optimization (RPO; Zhou et al., 2024) already optimizes defensive suffixes against adaptive attacks without updating model weights, but is not discussed in this paper. A direct comparison of objectives, model-access assumptions, and adaptation mechanisms is needed to clarify AAPO’s additional contribution.
-
The analysis understates the loss of model utility. In Table 1, adding AAPO reduces LLaMA-3 factuality from 54.6 to 38.5 for DPO and the neighbor score from 66.7 to 53.7 for RT. These substantial drops weaken the utility-preservation claim and require a clearer analysis of the forgetting–utility trade-off.
Questions for authors
-
Clarify the prompt-selection update. How are prompt subsets sampled under WDGS, and why is (p_i) the appropriate denominator in Equation (14)? How does the update account for interactions among prompts that receive the same joint reward?
-
Clarify the distinction from RPO. How does AAPO differ from RPO in its optimization objective, model-access requirements, and adaptation mechanism? What evidence establishes the benefit of these differences for robust unlearning?
-
Explain the utility losses. What causes the substantial factuality and neighbor-score drops in Table 1? Does AAPO still preserve utility better than simpler prompt defenses when compared at matched forgetting levels?
3 comments
Response to Reviewer jrfp [part 2]
Comment 2: The related-work discussion overlooks adversarial prompt optimization. Robust Prompt Optimization (RPO; Zhou et al., 2024) already optimizes defensive suffixes against adaptive attacks without updating model weights, but is not discussed in this paper. A direct comparison of objectives, model-access assumptions, and adaptation mechanisms is needed to clarify AAPO’s additional contribution. Clarify the distinction from RPO. How does AAPO differ from RPO in its optimization objective, model-access requirements, and adaptation mechanism? What evidence establishes the benefit of these differences for robust unlearning?
Response 2: Thank you very much for your valuable comment. RPO [1] is an excellent work. Below we compare the two methods from the three aspects you mentioned.
| RPO | AAPO | |
|---|---|---|
| Optimization objective | Targets jailbreak defense, making the model refuse harmful instructions under worst-case attacks | Targets robust unlearning, preventing leakage of unlearned information while keeping normal answers on retained knowledge |
| Model access | Requires gradients from a white-box model to optimize the suffix | Requires no gradients or weights |
| Adaptation mechanism | Includes worst-case attacks during optimization, and the suffix is fixed after deployment | Keeps generating new guards from leakage cases, and dynamically selects and combines them |
These differences are also reflected in the experiments. For the optimization objective, unlearning requires not only blocking leakage but also avoiding harm to related knowledge. In Appendix Table 6, refusal-based prompt defenses reduce the forget-set score, but their neighbor scores also drop to 40.1–49.0. In contrast, AAPO achieves 76.8, almost the same as the original model.
For the adaptation mechanism, we further tested two adaptive attacks against the final defense. The schema-aware attack can access AAPO's full defense prompt pool, and the PAIR attack iteratively searches for attack prompts based on model responses. On the original model + AAPO, we used these attacks as feedback for 10 rounds of adaptation. The changes in ASR are shown below:
| Defense state | Schema-aware | PAIR |
|---|---|---|
| 0.58 | 0.67 | |
| 0.41 | 0.29 | |
| ASR | -0.17 | -0.38 |
Without retraining the target model, AAPO reduces the ASR of the two attacks by 17 and 38 percentage points, respectively. In contrast, the defensive suffix of RPO stays fixed after deployment, and it needs to be re-optimized to handle new attacks. We will add a discussion of and comparison with RPO to the related work in the revised version. Thanks again!
[1] Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks
Response to Reviewer jrfp [part 3]
Comment 3: The analysis understates the loss of model utility. In Table 1, adding AAPO reduces LLaMA-3 factuality from 54.6 to 38.5 for DPO and the neighbor score from 66.7 to 53.7 for RT. These substantial drops weaken the utility-preservation claim and require a clearer analysis of the forgetting–utility trade-off. Explain the utility losses. What causes the substantial factuality and neighbor-score drops in Table 1? Does AAPO still preserve utility better than simpler prompt defenses when compared at matched forgetting levels?
Response 3: Thank you for your comment. We address your questions one by one below.
• Cause of the factuality drop. This drop mainly comes from AAPO making the target model answer more conservatively. The defense prompts of AAPO mainly use refusal as the form of defense. Since DPO has already changed the model's behavior, this refusal tendency can spread to some factual questions. This phenomenon mainly occurs on LLaMA-3-8B-Instruct. On Qwen2.5-14B-Instruct, the factuality of DPO(+AAPO) instead increases from 32.5 to 35.8.
• Neighbor drop of RT(+AAPO). This drop is closely related to the nature of RT itself. RT uses predefined refusal responses as its optimization target, so it already tends to refuse conservatively. In Table 1, RT alone reduces the neighbor score from 77.4 to 66.7, which is the largest drop among the four unlearning methods. On top of this, AAPO forms multi-layer protection through the dynamic selection, combination, and iterative update of defense prompts, which makes RT(+AAPO) more conservative than the other combinations.
• Comparison with prompt defenses at matched forgetting levels. We compare the AAPO results under different λ in Appendix Table 3 with the prompt-defense baselines in Table 6. Compared with Iterative (47.0 / 67.0), AAPO (λ=0.5) is better on both the forget-set score and the neighbor score (43.5 / 76.8). When λ=0.8, AAPO achieves a forget-set score of 40.9, which is close to that of Guardrail, Multi-Guardrail, and Length-Matched (33.0–34.7), while its neighbor score of 70.1 is 21.1–30.0 points higher than these baselines. Therefore, at similar forgetting levels, AAPO better preserves related knowledge.
We will add the above analysis to the revised version and correct the statements about utility preservation in the main text. Thank you again!
In summary, we are truly grateful for your insightful comments, and we sincerely hope you will consider the overall value of our paper. Thank you sincerely!
Response to Reviewer jrfp
Thank you for your time and constructive feedback! We are glad that you recognized the practicality of AAPO and its compatibility with existing unlearning methods. This is a great encouragement to us.
AAPO models robust unlearning as a closed-loop adversarial process among the Attacker, the Judge, and the Defender. Unlike methods that rely on parameter updates, AAPO works at the system-prompt level and requires neither access to model weights nor additional training. Therefore, it can serve as a plug-and-play module on top of different unlearning methods.
We respond to your comments on the prompt-selection update, the difference from RPO, and the utility loss point by point below, and we will revise the paper accordingly.
Comment 1: The prompt-selection update is insufficiently justified. Equation (14) divides the shared reward by (p_i), but the WDGS rule in Equation (12) selects multiple prompts without establishing that (p_i) represents their actual inclusion probabilities. This leaves the connection between prompt selection and the Exp3-style reward update unclear. Clarify the prompt-selection update. How are prompt subsets sampled under WDGS, and why is (p_i) the appropriate denominator in Equation (14)? How does the update account for interactions among prompts that receive the same joint reward?
Response 1: Thank you for this insightful question. We answer it one by one below.
• Sampling in WDGS. In each round, the Defender first determines the combination length kt according to Eq. 11. It then samples the first prompt according to the probability p(t), and selects the remaining kt−1 prompts one by one according to Eq. 12. In the revised version, we will provide the full selection procedure as pseudocode.
• Denominator in Eq. 14. When kt=1, the inclusion probability of prompt i is pi, and Eq. 14 is consistent with the standard Exp3 importance-weighted estimator. When kt>1, prompt i can be sampled as the first prompt (with probability pi), or selected later by Eq. 12. Therefore, its actual inclusion probability satisfies qi≥pi. In this case, Eq. 14 should be viewed as an Exp3-inspired heuristic update. The ablation study in Table 2 compares AAPO with two simpler settings. In w/o Exp3, the prompt weights are not updated, and all prompts keep the same weight. In Fixed-k1, only one prompt is selected in each round.
Compared with w/o Exp3, the weight update in Eq. 14 reduces the forget-set score by 17.4 points. Compared with the single-prompt setting (Fixed-k1), which matches the standard Exp3 estimator, extending to multi-prompt combinations further reduces the forget-set score by 15.8 points. Therefore, although Eq. 14 is a heuristic update when kt>1, it is effective in practice.
• Credit assignment among multiple prompts. AAPO does not explicitly model the interactions among prompts. Instead, it handles them implicitly through two mechanisms. First, since the first prompt is randomly sampled according to p(t), the same prompt appears together with different prompts in different rounds. After many rounds of updates, its weight reflects the average performance of the various combinations it takes part in, which can be seen as an approximation of its marginal effect. Second, the diversity term in WDGS lowers the chance that semantically similar prompts are selected together, which reduces confusion among redundant prompts. Appendix Table 4 shows that the forget-set score is the lowest when β is 0.5. Leaning too much toward historical weights (β=0.8) or toward diversity (β=0.2) increases the forget-set score to 47.3 and 52.4, respectively. This shows that balancing effectiveness and diversity is necessary for handling the interactions among prompts.
We will add the above analysis to the revised version. Thank you very much!