acceptodds
Under review as a conference paper at ICLR 2027

Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates

Abstract

Adversarial training is one of the most effective defenses against adversarial attacks, yet the computational cost remains prohibitive at modern scales, especially for large language models (LLMs). While existing mitigation strategies, e.g., latent adversarial training (LAT), have been developed, they still incur a high computational cost. In this work, focusing on LLM-based classifiers, we investigate computation-efficient strategies to speed up LAT from two complementary perspectives: (1) Defense-side optimization: We explore representation fine-tuning (ReFT) within LAT, and reveal a potential issue if there is a mismatch on which tokens to apply ReFT and the attack. (2) Attack-side optimization: When computing adversarial attacks in each LAT iteration, we extract only the relevant circuits from the LLM to construct a lightweight surrogate model, avoiding the computation in the forward-backward passes through the full model during the attack generation. For both perspectives, we provide theoretical justifications and numerical evidence to illustrate the effectiveness of the proposed strategies. Compared to standard LAT with full fine-tuning, our method on average reduces per-step adversarial-training FLOPs by 29.6% while requiring only 0.0118% trainable parameters, at a moderate cost in robustness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
adversarial training, representation fine-tuning, circuits

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EWgVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to reduce the computational cost of latent adversarial training (LAT) for LLM-based classifiers. On the defense side, it proposes LAT-ReFT, which freezes the backbone and applies low-rank ReFT interventions over a suffix window rather than only the final token. On the attack side, it constructs a lightweight surrogate for inner-loop adversarial optimization by pruning MLP neurons according to an activation-gradient (ActGrad) importance score. The paper also provides theoretical analyses motivating suffix-window defense and the surrogate construction. Experiments on IMDB, EnronSpam, and PasswordMatch with Pythia-1.4B, Qwen2.5-3B, and Llama-3.1-8B evaluate robustness, trainable parameters, FLOPs, runtime, and memory. The proposed approach reduces adversarial-training FLOPs by 29.6% on average and uses a very small fraction of trainable parameters, while incurring some loss in robustness compared with full-parameter LAT.

Critical strengths
  1. The paper presents a coherent approach to reducing the cost of LAT from both the defense and attack sides. The suffix-window ReFT defense is motivated by the mismatch between multi-token attacks and position-local interventions, while the pruned surrogate targets the expensive inner-loop attack generation. The theoretical analysis and ablations generally support these design choices, making the method more than a simple combination of existing components.
  2. The efficiency gains are supported by both algorithmic and implementation-level evidence. In addition to FLOPs and trainable-parameter counts, the paper reports wall-clock time and peak memory reductions. The ActGrad surrogate also substantially outperforms random pruning and ActDiff at the same pruning ratio, suggesting that the attack-aware neuron selection is consequential to the method.
Critical weaknesses
  1. The empirical scope is relatively narrow compared with the general framing of LLM adversarial training. The main evaluation focuses on LLM-based classifiers and primarily token-level suffix attacks. It remains unclear whether the proposed robustness benefits extend to open-ended generation or to stronger adaptive attacks that explicitly exploit knowledge of the defended token window or intervention layer.
  2. The robustness-efficiency tradeoff is not uniformly favorable across settings, and the cross-dataset evaluation is somewhat limited. Compared with full-parameter LAT, the proposed method shows noticeable increases in attack success rate in several settings. In addition, the most complete multi-model evaluation is conducted on IMDB, while the additional datasets are mainly evaluated with Pythia-1.4B. This makes it difficult to determine how consistently the reported tradeoff generalizes across both models and tasks.
Questions for authors
  1. Clarification / additional evidence: How robust is LAT-ReFT to adaptive attacks that explicitly account for the defended suffix window and the intervention layer? The current GCG evaluation differentiates through the ReFT intervention, but it is not clear whether the attack is specifically adapted to exploit the known defense placement. Results or further clarification on this point would materially affect my assessment of the robustness claim.
  2. Clarification / additional evidence: The full three-model comparison is mainly reported on IMDB, whereas the cross-dataset results on EnronSpam and PasswordMatch are primarily shown for Pythia-1.4B. Do the authors have corresponding results for Qwen2.5-3B or Llama-3.1-8B on these datasets? This would help establish whether the reported robustness-efficiency tradeoff generalizes across both model families and tasks.
1 comment

For Weakness 1: We thank the reviewer for raising this concern. Our work focuses on improving the computational efficiency of LAT for LLM-based classifiers under the established suffix-attack setting of Howe et al. [1]. As discussed in Appendix A, this setting provides a well-defined measure of attack success and is practically relevant to applications such as content moderation and filtering [2, 3]. We therefore consider LLM-based classification a meaningful research setting in its own right.

Importantly, the proposed representation-level approach is not restricted to classification. Both the inner-loop adversarial optimization and the outer-loop ReFT interventions operate on hidden representations. This provides a basis for extending the framework to open-ended generation with appropriate task objectives, which we view as a direction for future work.

Regarding adaptive attacks, our evaluation follows the established suffix-based threat model of Howe et al. [1]. GCG is performed in a white-box setting against the full defended model, with gradients computed through the ReFT interventions. Thus, adversarial suffixes are already optimized against the actual defense. In addition, Appendix C.3 evaluates alternative attack positions, including prefix and infix attacks, providing further evidence beyond the suffix setting.

For Weakness 2:

To further examine generalization across models and tasks, we extend our evaluation on EnronSpam to Qwen2.5-3B and Llama-3.1-8B.

DatasetModelLATLAT-ReFT
EnronSpamPythia-1.4B0.290.08
EnronSpamQwen2.5-3B0.200.26
EnronSpamLlama-3.1-8B0.000.04

The additional results show that the robustness trade-off varies across models. On Pythia-1.4B, LAT-ReFT achieves lower GCG ASR than full LAT (0.08 vs. 0.29). On Qwen2.5-3B and Llama-3.1-8B, LAT-ReFT achieves ASRs of 0.26 and 0.04, compared with 0.20 and 0.00 for full LAT, respectively. These results extend our evaluation beyond Pythia-1.4B and further characterize the model-dependent robustness trade-off of our framework.

For Question 1 and Question 2: These questions are addressed in our responses to W1 and W2 above. W1 discusses GCG evaluation and alternative attack positions (Appendix C.3), while W2 addresses robustness–efficiency trade-offs and cross-dataset generalization.

References

[1] Howe, Nikolaus, et al. "Scaling trends in language model robustness." arXiv preprint arXiv:2407.18213 (2024).

[2] Inan, Hakan, et al. "Llama guard: Llm-based input-output safeguard for human-ai conversations." arXiv preprint arXiv:2312.06674 (2023).

[3] Achara, Akshit, and Anshuman Chhabra. "Watching the AI watchdogs: A fairness and robustness analysis of AI safety moderation classifiers." Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). 2025.

Reviewer_oUnjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates computationally efficient latent adversarial training (LAT) for LLM-based classifiers. On the defense side, the authors combine LAT with representation fine-tuning (ReFT), freezing the backbone and learning low-rank interventions over a suffix window of hidden states. They study how the intervention layer and defended token span influence robustness. On the attack side, they construct a lightweight surrogate for inner-loop adversarial optimization by pruning MLP neurons using activation-times-gradient importance scores (ActGrad), and transfer the resulting perturbations to the full model for training. The approach is supported by theoretical analyses of attention-level perturbation propagation and surrogate approximation. Experiments evaluate three model families on IMDB, with additional cross-dataset results on EnronSpam and PasswordMatch using Pythia-1.4B. The authors report substantial parameter and memory savings, alongside moderate computational improvements and varying levels of adversarial robustness.

Critical strengths

S1: Substantial parameter and memory efficiency. The proposed method updates only a small fraction of the model parameters (0.0066%-0.0203% in Table 1), while Appendix F.2 reports reductions of 56.6%-73.8% in peak GPU memory compared with full-parameter LAT. These savings could be practically valuable when computational resources are limited, particularly for maintaining multiple task-specific defenses. The inclusion of measured wall-clock performance alongside analytical FLOP estimates is also appreciated, although the actual speedups are more modest and model-dependent.

S2: Targeted ablations of defense placement and surrogate construction. Figures 2 and 3 examine the defended token span, intervention layer, and neuron selection strategy, primarily on Pythia-1.4B/IMDB. In particular, ActGrad substantially outperforms random and activation-difference pruning at the same pruning ratio in the reported experiment. Although these findings do not establish a broader mechanistic explanation, they provide empirical support for some of the proposed design choices.

Critical weaknesses

W1: Limited methodological novelty and insufficient evidence of a competitive robustness-efficiency trade-off. The framework primarily combines two established approaches: low-rank representation interventions from ReFT [1] and activation-gradient importance scores closely related to first-order Taylor pruning (e.g., [2]). Their application to LAT is reasonable, but the methodological contribution appears relatively incremental. The reported experiments do not show a practical advantage sufficiently compelling to offset this limited novelty. On IMDB (Table 1), GCG ASR increases from 2% to 15% on Llama-3.1-8B, 10% to 26% on Qwen-2.5-3B, and 14% to 33% on Pythia-1.4B relative to full LAT. In particular, the Llama result entails a 13-point increase in ASR for only a 16.3% reduction in estimated per-step FLOPs (167.0167.0 to 139.8×1012139.8 \times 10^{12}). Across models, the mean estimated FLOP reduction is 29.6%, while measured step-time reductions are 12.7%-31.8% on Pythia and Qwen (Appendix F.2). On Qwen, RandomToken ASR also increases from 6% to 38%. The more favorable EnronSpam and PasswordMatch results (Table 6) deserve recognition, but suggest substantial task dependence. Crucially, the comparisons conflate parameter-efficient adaptation with surrogate pruning, use differing attack layers, radii, and PGD update rules, and omit a matched PEFT-LAT baseline. Although the CAT baseline already employs LoRA-based parameter-efficient fine-tuning (Appendix B.4), it operates in embedding space rather than latent space, so it cannot isolate the benefits of the proposed ReFT-based defense and surrogate pruning. Robustness is also evaluated with only 10 GCG iterations on 100 examples (Appendix B.5). Given the incremental nature of the method, establishing its significance would require a controlled comparison of robustness as a function of training costs against fairly tuned alternatives such as LAT, CAT, or PEFT-LAT. The current results do not convincingly establish such an advantage.

W2: The theoretical justification does not adequately characterize the implemented surrogate. I recognize that simplifying assumptions are necessary for tractable theoretical analysis. However, some of the assumptions underlying Theorem 3 appear to abstract away precisely the challenges introduced by the proposed method. The proof in Appendix E.4 considers an MLP expansion at a single layer lal_a, whereas the implemented surrogate prunes neurons across multiple Transformer layers, without adequately accounting for downstream pruning or residual gradient pathways. More importantly, the analysis approximates the surrogate gradient by a projected full-model gradient, rts≈PSrtr_t^s \approx P_S r_t, and explicitly sets the associated approximation error to zero. This assumption is especially consequential because preserving adversarial optimization directions despite pruning is the central property the theorem aims to justify. Furthermore, Assumption 2 requires the activations of all pruned neurons to be bounded away from zero, with the resulting constant scaling as 1/m01/m_0; it is unclear whether this yields a meaningful bound for low-ActGrad neurons. These issues do not invalidate the empirical pruning results, but substantially limit the theoretical justification provided for the implemented method.

Questions for authors

Q1: What evidence supports a favorable robustness-efficiency trade-off over matched parameter-efficient LAT alternatives? (W1) The main comparison against full-parameter LAT does not isolate ReFT from surrogate pruning. In particular, the 0%-pruning ReFT result on Pythia/IMDB has GCG ASR 0.26, versus 0.33 with 25% pruning (Table 7), while full LAT achieves 0.14 (Table 1). If these runs are otherwise comparable, this appears to attribute a larger share of the robustness difference to the ReFT-based defense than to surrogate pruning. Can the authors confirm the comparability of these configurations and clarify the individual efficiency and robustness contributions? How do the results compare with matched LoRA-LAT or other PEFT-LAT alternatives? Table 4 and Appendix B.4 also differ in attack-layer choices, radii, and PGD updates: what projection norm is used for LAT-ReFT, and how were these settings tuned? Suggested evaluation: evaluate robustness as a function of training costs for the proposed method and at least one appropriately tuned alternative (at least one of LAT [3] or CAT [4]). I appreciate that such an experiment may be too costly for rebuttal; clarification of the available evidence would still be valuable.

Q2: How does Theorem 3 extend to the actual multi-layer surrogate? (W2) Appendix E.4 analyzes a single MLP expansion and assumes that the surrogate gradient approximates a projection of the full-model gradient. How does this account for pruning in subsequent layers and residual gradient pathways, and under what conditions is rts≈PSrtr_t^s \approx P_S r_t justified? Can Assumption 2 be shown to give a non-vacuous constant for the selected neurons? It would also help to explain how clean-calibration ActGrad scores in Eq. (3) relate to the omitted gradient mass evaluated along adversarial PGD iterates in Theorem 3. Suggested analysis: measure full-versus-surrogate gradient alignment along adversarial trajectories at different pruning ratios.

Q3: How reliable are the reported robustness results under stronger adversarial optimization? (W1) GCG is evaluated using only 10 iterations on 100 attack examples (Appendix B.5). Prior work [5] has shown that attack success rates can increase substantially with additional attack compute, even against adversarially trained models. Consequently, it is unclear whether the reported robustness differences between LAT and LAT-ReFT would persist under stronger attacks. Were representative checkpoints evaluated with larger GCG budgets or multiple attack initializations? Suggested evaluation: report ASR as a function of attack budget to establish whether the relative performance of the methods remains stable.

Q4: What explains the substantial differences in relative robustness across datasets? (W1) LAT-ReFT is less robust than LAT on IMDB but performs better under GCG on EnronSpam and PasswordMatch (Table 6). For Pythia/IMDB the attack and intervention layers differ (la=4l_a=4, lr=12l_r=12), whereas they coincide for the two additional datasets; Qwen/IMDB also uses coincident layers without showing the same advantage. Can the authors clarify whether the differences are driven by task characteristics, attack/defense placement, perturbation budgets, or baseline tuning? Understanding when and why the method improves over LAT would help establish its generality.

Q5: How is label preservation handled under adversarial suffix attacks? The evaluation retains the original ground-truth labels when adversarial suffixes are appended to IMDB or EnronSpam inputs. However, depending on their content, some suffixes might arguably change the interpretation of the original example, even for a human annotator. Could the authors clarify the intended classification objective and whether successful attacks were inspected to distinguish genuine adversarial failures from cases where the suffix changes the apparent classification target?

I would welcome the authors' perspective on these points during rebuttal and would be glad to discuss them further.

References

[1] Wu et al., ReFT: Representation Finetuning for Language Models, NeurIPS 2024.

[2] Molchanov et al., Importance Estimation for Neural Network Pruning, CVPR 2019.

[3] Casper et al., Defending Against Unforeseen Failure Modes with Latent Adversarial Training, TMLR 2025.

[4] Xhonneux et al., Efficient Adversarial Training in LLMs with Continuous Attacks, NeurIPS 2024.

[5] Howe et al., Scaling Trends in Language Model Robustness, ICML 2025.

[6] Zou et al., Universal and Transferable Adversarial Attacks on Aligned Language Models, arXiv:2307.15043, 2023.

3 comments

We thank the reviewer for the detailed comments.

For (W1).

  1. Methodological novelty: Our work addresses two challenges specific to LAT. On the defense side (G1), ReFT allows explicit control over intervention positions and layers, unlike other weight-space adaptation methods that modify weight matrices across all tokens uniformly. We analyze how perturbations from uncovered suffix positions can propagate through attention (Theorem 1), motivating suffix-window interventions and the investigation of intervention depth (Theorem 2). On Pythia-1.4B/IMDB, extending the defended window from one to 20 tokens reduces GCG ASR from 61% to 26%. On the attack side (G2), our surrogate must generate adversarial perturbations that remain effective on the full model. This motivates ActGrad-guided neuron selection. Our additional 8-step PGD experiments show that ActGrad retains 59.2% of the full-model adversarial loss increase, compared with 47.5% for layer-matched random pruning and 48.1% for ActDiff.

  2. Isolating the contributions of ReFT and surrogate pruning: The Pythia-1.4B/IMDB results in Tables 1 and 8 use the same evaluation protocol. Full LAT achieves 0.14 GCG ASR, compared with 0.26 for LAT-ReFT without surrogate pruning and 0.33 with 25% ActGrad pruning. The first comparison reflects the change from full-parameter updates to low-rank representation interventions, while the second isolates the additional effect of surrogate pruning under otherwise matched LAT-ReFT configurations. On EnronSpam, 25% pruning instead improves ASR from 0.26 to 0.08. These results characterize the task-dependent robustness--efficiency trade-off of the two components.

  3. PGD settings and hyperparameter tuning: Both methods use an ℓ2\ell_2-bounded latent perturbation restricted to 20 suffix token positions, with K=8K=8 PGD steps. LAT-ReFT uses signed-gradient updates followed by ℓ2\ell_2 projection, whereas the full LAT baseline with ℓ2\ell_2-normalized gradient updates and the same projection constraint. For both methods, the attack layer and perturbation radius were selected from the same candidate grids, as detailed in Table 4 and Appendix B.4.

For (W2).

We appreciate the reviewer's rigorous engagement with Theorem 3 and Appendix E.4. We have revised them (highlighted in blue) to make the exact scope explicit:

  1. The goal of Theorem 3 is to isolate the local mechanism motivating ActGrad-based neuron selection, following standard one-layer practice in Transformer theory [1, 2]. In our revised Theorem 3 and Appendix E.4, we explicitly retain the residual mismatch term εt:=∥(gt−gts)−Jt⊤(I−PS)rt∥\varepsilon_t := \|(g_t - g_t^s) - J_t^\top (I - P_S) r_t\|: $ \sum_{t=1}^{K} G_t \leq C_{\mathrm{surr}}, \eta \sum_{t=1}^{K} M_t(S) + c,\eta \sum_{t=1}^{K} \varepsilon_t + o(K\eta). $ For the multi-layer setting, extending the analysis would require accounting for the propagation of pruning errors through downstream Jacobians and residual pathways [3]. Such an extension would involve additional cross-layer interactions and stronger regularity assumptions. We therefore use the single-layer formulation to isolate the omitted-neuron contribution, while empirically evaluating the implemented multi-layer surrogate.

  2. Assumption 2 is introduced as a sufficient condition to relate the omitted gradient norm to the omitted ActGrad mass, yielding a simpler conditional bound. The assumption can also be relaxed by introducing a threshold (τ>0)(\tau>0) and partitioning the pruned neurons into Sactivec=j∉S:∣at,j∣≥τS_{\mathrm{active}}^c={j\notin S:|a_{t,j}|\geq\tau} and Ssmallc=j∉S:∣at,j∣<τS_{\mathrm{small}}^c={j\notin S:|a_{t,j}|<\tau}. We then obtain $ |(I-P_S)r_t| \leq \frac{M_t(S)}{\tau} +|r_{t,S_{\mathrm{small}}^c}|. $ This decomposition does not require a uniform positive lower bound on the pruned activations. It explicitly retains the gradient contribution from small-activation neurons as an additional residual term.

  3. Theorem 3 provides a conditional bound for any fixed pruning mask SS. If the omitted ActGrad mass along the PGD trajectory is small and the additional approximation errors are controlled, the local optimization gap is also bounded. In practice, we select the mask once using clean calibration data. Its effectiveness is evaluated directly through multi-step PGD transfer experiments, where the ActGrad surrogate retains 59.2% of the full-model adversarial loss increase, outperforming layer-matched random pruning (47.5%) and ActDiff (48.1%).

References

[1] Zhang, Ruiqi, Spencer Frei, and Peter L. Bartlett. "Trained transformers learn linear models in-context."

[2] Edelman, Benjamin L., et al. "Inductive biases and variable creation in self-attention mechanisms."

[3] Castin, Valérie, Pierre Ablin, and Gabriel Peyré. "How smooth is attention?"

[4] Howe, Nikolaus, et al. "Scaling trends in language model robustness."

For Q1 and Q2: Please find the related responses for W1 and W2 above.

For Q3: Following the reviewer's suggestion, we evaluated the checkpoints under stronger white-box GCG attacks, increasing the optimization budget from 10 to 50 and 100 iterations.

Method (Qwen2.5-3B / IMDB)GCG (T=10)GCG (T=50)GCG (T=100)
LAT10%31%33%
LAT-ReFT26%46%50%
Absolute ASR Gap (Δ)+16%+15%+17%

Stronger optimization increases ASR for both methods, while the absolute ASR gap remains nearly unchanged (16, 15, and 17 percentage points). The additional ASR increase also becomes smaller between 50 and 100 iterations. Thus, although the original 10-step evaluation underestimates absolute attack success, the relative robustness comparison remains stable across the evaluated budgets.

For Q4: We believe that task characteristics contribute to the observed robustness differences. Different classification tasks may rely on different representations and exhibit different adversarial vulnerabilities. For example, sentiment classification often requires context-dependent linguistic features, whereas spam detection may rely more heavily on a subset of discriminative textual patterns. Consequently, pruning may affect attack-relevant representations differently across tasks. One possible explanation is that moderate pruning removes less informative optimization directions in some tasks while discarding useful adversarial information in others.

For Q5: We clarify that our evaluation follows the standard white-box adversarial suffix benchmark of [4]. During evaluation, the original review or email text is kept and GCG appends a short N=10N=10 token suffix at the end of the input. Because the original sentence is unmodified, a 10-token suffix cannot legitimately reverse the true semantic category of a full movie review or spam email for a human reader. Furthermore, we manually inspected the successful GCG suffixes and added representative examples in Appendix B.5. As shown in Table 6, successful GCG suffixes consist of syntactic punctuation and incoherent token fragments (e.g., !!!!!!!Turk)). Jeffrey flipping a $25.5M Nigerian-prince scam email from SPAM to HAM, and !!!ERVFort236 useless junk Ass Marxist flipping a glowing review of the film from POSITIVE to NEGATIVE). None of the inspected suffixes alter the human ground-truth label; they are genuine adversarial failures.

Thank you for the detailed response and the additional experiments. I appreciate the clarifications regarding the experimental setup and Theorem 3, as well as the stronger GCG evaluation. These address several of my concerns and make the results more convincing.

However, the new experiments mostly confirm the robustness differences already observed, and my main concern remains the limited novelty and significance of the contribution. While the computational and memory savings are useful, I am still not convinced that the resulting robustness–efficiency trade-off is sufficiently competitive, particularly without comparisons against appropriately matched parameter-efficient alternatives. I understand that such comparisons would be expensive to conduct during the rebuttal, and I appreciate the authors' efforts to address the questions that could be answered. Nevertheless, the rebuttal has not substantially changed my overall assessment, so I will maintain my original recommendation.

Reviewer_g8MjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to reduce the cost of adversarial training for LLM-based classifiers while preserving robustness. It targets two sources of cost: updating the defense and generating training attacks. On the defense side, it trains only a small, low-rank ReFT module. On the attack-generation side, it uses an ActGrad to select and prune MLP neurons, creating a smaller surrogate model for generating attacks. The approach has it's practical value.

Critical strengths
  1. The paper addresses two major costs of adversarial training: updating the model and generating attacks for training. Reducing both costs is a sensible approach with practical benefits.
  2. The authors evaluate their method on Llama, Qwen, and Pythia, providing some evidence that it can work across different model families.
Critical weaknesses
  1. The novelty needs to be made clearer. Both training small ReFT modules and pruning neurons using activation and gradient information build on existing work. What new designs does this paper introduce when combining these ideas? The authors should explain what specific challenges these designs address and what benefits they provide beyond a straightforward combination of existing methods.

  2. Do attacks generated on the smaller model remain effective on the full model? Pruning neurons can change a model’s behavior. An attack that fools the smaller model may not exploit the same weaknesses in the full model. The authors should directly evaluate how effective these transferred attacks are on the full model. They should also examine whether training could focus too much on the surrogate’s weaknesses while missing vulnerabilities in the full model.

  3. Are the computational savings worth the loss in robustness? The method reduces training costs, but robustness also drops noticeably in some experiments. The authors should explain under what resource constraints and application settings this drop would be acceptable. Could the original method achieve better robustness under the same compute budget by training for less time or using fewer attack-search steps? A comparison under equal training time or compute budgets would help establish the practical value of this trade-off.

Questions for authors

N/A

1 comment

For Weakness 1.

We thank the reviewer for the comments. Instead of simply combining existing tools, our work identifies why naive applications of ReFT and model pruning fail under the min-max dynamics of LAT, and introduces targeted designs to resolve these failures:

  1. Defense side (G1): Addressing spatial and depth mismatches in latent defense: Position-local ReFT interventions may be insufficient against multi-token adversarial perturbations. We analyze how perturbations from uncovered positions propagate through attention (Theorem 1) and investigate how intervention coverage and layer placement affect robustness (Figure 1). Our suffix-window design (Theorem 2) reduces GCG ASR from 61% with a single-token intervention to 26% with a 20-token intervention on Pythia-1.4B/IMDB. Late-layer interventions yield substantially higher ASR (91%–93%), further highlighting the importance of intervention placement.
  2. Attack side (G2): Preserving effective adversarial optimization: In LAT, the pruned surrogate is not used for inference; it must generate multi-step inner-loop PGD perturbations that remain effective when transferred to the full model. Naive compression methods that rely on random selection or clean activation differences (ActDiff, widely used in representation engineering) destroy the backward gradient pathways needed for PGD, resulting in a completely ineffective defense (GCG ASR rises to 95% for random pruning and 97% for ActDiff at 25% pruning; see Figure 3b). Our ActGrad scoring explicitly targets neurons with the largest contribution to the inner adversarial objective (Theorem 3), reducing GCG ASR to 33% at the same 25% pruning ratio.

We have revised the Abstract and Introduction (highlighted in blue) to clarify these specific challenges and designs.

For Weakness 2.

  1. In our framework, the pruned surrogate is used only for inner-loop PGD during training (Algorithm 1, Lines 7–10). Once the inner PGD perturbation is generated on the surrogate, it is injected directly into the full defended model to compute the outer adversarial training loss Ladv\mathcal{L}_{\mathrm{adv}} (Algorithm 1, Line 12). At evaluation time, white-box GCG and RandomToken attacks directly target the full defended model, not the surrogate. Thus, the reported robustness reflects the actual full-model defense under these attacks.
  2. Following the reviewer's suggestion, we independently generate eight-step PGD perturbations on the full model and pruned surrogates, then evaluate all perturbations on the same full defended model. At 25% pruning, ActGrad retains 59.2% of the full-model PGD's mean adversarial loss increase, outperforming layer-matched random pruning (47.5%) and ActDiff (48.1%). This confirms useful transferability while quantifying the expected approximation trade-off.
  3. Our pruning ablations further examine whether surrogate approximation affects the resulting defense. Aggressive pruning (50%–75%) leads to substantially higher full-model GCG ASR (89%–95%), whereas moderate ActGrad pruning (25%) maintains ASR close to the unpruned surrogate baseline on IMDB (0.33 vs. 0.26) and improves it on EnronSpam (0.08 vs. 0.26). These results demonstrate the importance of surrogate quality for the final defense.

For Weakness 3. As reported in Appendix F.2, LAT-ReFT reduces peak GPU memory from 25.39 GB to 11.01 GB on Pythia-1.4B and from 56.44 GB to 14.77 GB on Qwen2.5-3B, while also reducing per-step training time. Our framework further achieves an average estimated per-step FLOPs reduction of 29.6% while updating only 0.0118% of model parameters. These savings are particularly relevant when full-parameter LAT exceeds the available GPU memory.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.