- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces PICLAlign to investigate safety degradation in LLMs when context-to-adapter frameworks (D2L, SHINE) convert text documents into reusable parametric LoRA adapters. Evaluating models like Gemma2-2B-IT and Qwen3 (4B/8B), the authors demonstrate that parametric internalization of benign documents (e.g., SQuAD) significantly increases attack success rates compared to standard in-context learning, which shows minimal or negative correlation with attack success.
Critical strengths
-
A specific safety failure is demonstrated with a relevant comparison. Table 1 holds document-query pairs fixed and contrasts ICL with PICL across three systems. For benign SQuAD contexts, PICL adds 12.5%, 8.5%, and 13.0% of ASR, while ICL changes ASR by -2.5, +0.5, and -8.5 points. This makes automatic document internalization a concrete safety concern even without malicious context construction, which matters to deployment of context-to-adapter systems.
-
The intervention has substantial controlled empirical support. The Standard Safety LoRA control has the same objective and datasets but trains without generated adapters (Appendix A.5.2). Even without Doc-DRO, simulation reduces average ASR from 9.9 to 3.2 on D2L-Gemma2 and from 48.3 to 3.5 on SHINE, with higher or only modestly lower reported utility (Tables 3-4). With Doc-DRO, PICLAlign-L obtains average ASRs of 0.0, 0.1, and 0.4 and utility scores of 52.8, 61.0, and 51.9, versus original scores of 53.1, 62.8, and 52.5 (Tables 3-5). This supports a practically consequential improvement across distinct hypernetworks and backbones, independently of whether the strongest theorem interpretation is retained.
Critical weaknesses
The evidence does not sufficiently test the adversarial document pathway central to the robustness claim. The stated attacker can control documents (Section 2.2), but the reported ICA/source-variation tests and the limited suffix experiment do not demonstrate effective optimization against the aligned document-to-adapter pathway (Appendices A.5.1 and A.2).
Questions for authors
Does a control matched to Doc-DRO's perturbation target and comparable search budget retain the reported advantage?
3 comments
Response to Reviewer AvaC (2/3)
W1: Additional experiments on adaptive attacks that test the adversarial document pathway. (Part 2/2)
Universal adversarial document: We further optimize a universal adversarial document and evaluate its generalizability to these challenging harmful queries, which is even more realistic. Since traditional jailbreak algorithms are designed for query-specific settings (i.e., they optimize a tailored prefix/suffix for a given harmful query), we extend them for universal attacks and optimize the adversarial document on a separate training set of N’ = 10 harmful queries drawn from the same distribution, disjoint from the tested queries. Specifically, we select PAIR [2], which achieves the best query-specific attack performance in terms of efficiency and accuracy, as our base algorithm. We then adapt it to optimize a universal adversarial document by using the average PAIR judge score and aggregated feedback across multiple harmful queries as signals to guide the optimization at each iteration. For a fair comparison, we fix the restart to 1 and report the ASR under different attack settings.
From the tables below, we can observe that the universal documents can promote the ASR without any repeated query-specific attack optimization (e.g., 20% to 65% for unaligned SHINE-Qwen3-8B), although their ASR remains lower than that of the query-specific variant, especially for D2L-Gemma2-2B-IT. Moreover, our PICLAlign can still effectively defend against them, further demonstrating its strong adversarial robustness.
Universal adversarial documents against unaligned/aligned D2L-Gemma2-2B-IT
| Setting | #restarts | #pre-attack optimization iteration T | Unaligned #pre-attack queries | Unaligned Compliance ASR (%) | Aligned #pre-attack queries | Aligned Compliance ASR (%) |
|---|---|---|---|---|---|---|
| SQuAD | 1 | - | - | 0 | - | 0 |
| Query-specific PAIR | 1 | - | - | 25 | - | 0 |
| Universal PAIR | 1 | T=25 | 750 | 5 | 750 | 0 |
| Universal PAIR | 1 | T=50 | 1500 | 5 | 1500 | 0 |
Universal adversarial documents against unaligned/aligned SHINE-Qwen3-8B
| Setting | #restarts | #pre-attack optimization iteration T | Unaligned #pre-attack queries | Unaligned Compliance ASR (%) | Aligned #pre-attack queries | Aligned Compliance ASR (%) |
|---|---|---|---|---|---|---|
| SQuAD | 1 | - | - | 20 | - | 0 |
| Query-specific PAIR | 1 | - | - | 90 | - | 0 |
| Universal PAIR | 1 | T=25 | 750 | 65 | 750 | 0 |
| Universal PAIR | 1 | T=50 | 1470 | 65 | 1470 | 0 |
References:
[1] Ma et al., "PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling," ICML 2025.
[2] Chao et al., "Jailbreaking Black Box Large Language Models in Twenty Queries," arXiv 2023.
[3] Mehrotra et al., "Tree of Attacks: Jailbreaking Black-Box LLMs Automatically," NeurIPS 2024.
Response to Reviewer AvaC (3/3)
Q1: Adversarial robustness analysis at matched perturbation target and budget.
Our black-box attack experiments show that PICLAlign is strongly robust against realistic attacks. In practice, the attacker manipulates only the text and cannot control the perturbation induced in the PICL-generated LoRA. To explicitly investigate the adversarial robustness gained from Doc-DRO, we consider a stronger threat in which the attacker directly perturbs the generated LoRA (this threat is impractical and primarily serves as a robustness analysis). Since documents affect the model only through , this upper-bounds any document-level attack under the same budget (Lemma A.4.1).
Our setup. We sample N’’ = 50 harmful queries from the HarmBench dataset, each paired with the same benign SQuAD document as before. For each (document, query) pair, the attacker perturbs with an FGSM-style attack under the ℓ2 norm (the same norm as Doc-DRO), matching Doc-DRO’s search budget during training, with a radius of 0.5×, 1× and 2× the Doc-DRO perturbation size. The objective is to elicit the target response "Sure, I can help with that". The perturbation is computed separately against each system: the unaligned system, the aligned system without Doc-DRO, and the aligned system with Doc-DRO. All other evaluation settings follow the black-box experiments above. We conduct this analysis on the D2L-Gemma2-2B-IT systems, whose backbone exhibits strong inherent safety alignment (e.g., low ASR on these queries for the original PICL systems), so that increases in ASR can be clearly attributed to the LoRA perturbation.
As shown in the table below, Doc-DRO substantially improves robustness against these weight-level perturbations. At the matched budget (1× PS), it reduces ASR from 60% to 0% for PICLAlign-L and from 30% to 8% for PICLAlign-H, and its advantage persists at 2× PS (72% to 10% and 66% to 16%, respectively). These results show that Doc-DRO retains its reported advantage under perturbations matched to its target and search budget. In our paper, we do not further strengthen the alignment methods for such attacks, since such weight-level perturbations are impractical and serve only for analysis.
Adversarial robustness under weight-level perturbations. PS denotes the Doc-DRO perturbation size. Each column assumes the attacker can add a perturbation of the given size. Note that this threat is impractical and primarily serves as a robustness analysis.
| Variant | ASR under no perturbation (%) | ASR (%) (PS*0.5) ↓ | ASR (%) (PS*1) ↓ | ASR (%) (PS*2) ↓ |
|---|---|---|---|---|
| Original PICL | 10 | 34 | 46 | 66 |
| PICLAlign-L (w/o Doc-DRO) | 0 | 18 | 60 | 72 |
| PICLAlign-L | 0 | 0 | 0 | 10 |
| PICLAlign-H (w/o-DRO) | 0 | 0 | 30 | 66 |
| PICLAlign-H | 0 | 0 | 8 | 16 |
Beyond demonstrating robustness against strong weight-level perturbations, we further show that Doc-DRO outperforms local smoothing and adversarial regularization methods, including SAM-based approaches [1], under matched perturbation budgets, as detailed in our response to Reviewer BfJC (W2). For example, Doc-DRO consistently reduces the average ASR across various datasets by more than 2.1 percentage points compared with existing methods for both D2L-Gemma2-2B-IT and SHINE-Qwen3-8B. We attribute its superior empirical performance to its principled problem formulation and theoretically grounded designs.
References:
[1] Foret et al., "Sharpness-Aware Minimization for Efficiently Improving Generalization", ICLR 2021.
Response to Reviewer AvaC (1/3)
We sincerely thank the reviewer for recognizing the novelty of the identified vulnerability, the effectiveness of our proposed mitigation, and the comprehensiveness of our core experiments. We appreciate the insightful and constructive feedback on improving this paper! Since the concern is related to the adversarial robustness of PICLAlign, we extensively evaluate robustness against realistic black-box attacks in W1 and robustness against strong weight-level perturbations in Q1.
In this part, we experiment with black-box adaptive attacks that optimize adversarial documents to induce unsafe responses, including query-specific adversarial attacks common in jailbreak literature and universal adversarial attacks. Our results show that query-specific attacks achieve desirable results before alignment, while PICLAlign presents strong robustness to diverse attacks.
Query-specific adversarial document: Consistent with the jailbreak literature and our results in Appendix A.2, it is difficult to optimize a single adversarial document that induces unsafe responses for many harmful queries. During the rebuttal period, we therefore first focus on evaluating targeted adaptive attacks, which is in favor of the attacker: for each target harmful query, the attacker optimizes an adversarial document that induces the PICL-integrated system to produce an unsafe response to that query.
For practicality, we consider black-box attacks adapted from the jailbreak literature. Given the substantial cost of query-based optimization (e.g., at least 78 seconds per harmful query using PAIR [2] under D2L-Gemma2 in our experiments), we randomly sample N=20 challenging harmful queries, which are difficult to elicit unsafe responses under benign documents, from the HarmBench dataset, and optimize a separate adversarial document for each of them.
Specifically, we evaluate PANDAS [1], PAIR [2], and TAP [3], and include experiments with benign SQuAD documents as baselines. Based on reviewer suggestions, we report attack efficiency and ASR (compliance score measured by WildGuard) under different settings across unaligned and aligned variants of D2L-Gemma2-2B-IT and SHINE-Qwen3-8B. As shown in the table below, these black-box optimization methods consistently increase ASR through query-based document optimization, raising ASR from 0% to 25% (via PAIR) for unaligned D2L-Gemma2-2B-IT and from 20% to 90% (via PAIR) for unaligned SHINE-Qwen3-8B even with a single restart. Moreover, the average number of queries is small (e.g., less than 10 for PAIR). Under a comparable query budget, benign documents raise ASR far less, even with multiple restarts (e.g., 5% vs. 25% on D2L-Gemma2 and 45% vs. 90% on SHINE-Qwen3-8B; see "PAIR with 1 restart" vs. "SQuAD with multiple restarts"), where we count an attack as successful if any restart succeeds.
Despite the promising results of query-specific adversarial documents, our alignment still effectively reduces the ASR to 0 across all the attack and PICL settings, even under large query budgets. We attribute this to PICLAlign’s ability to robustly learn the safety erosion patterns introduced by PICL and generalize to unseen documents. Moreover, we highlight that our main contribution is not about proposing an adversarially robust alignment technique. We will clarify this and discuss the results in the next version.
Query-specific adversarial documents against unaligned/aligned D2L-Gemma2-2B-IT
Query-specific adversarial documents against unaligned/aligned SHINE-Qwen3-8B
We further investigate universal adversarial documents, which do not require query-specific optimization, in Part 2.