- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies Inoculation Adapters (IAs) for selective generalization, aiming to retain desired traits while suppressing undesired ones during fine-tuning. It introduces two gated variants, GIA and CGIA, and evaluates their effectiveness across nine setups and five model families. The results suggest improved suppression–retention trade-offs and fewer surprising backdoors compared with several existing methods.
Critical strengths
Strengths
- Clear presentation. The paper is well-written, easy to follow, and clearly motivated.
- Comprehensive evaluation. The experiments cover diverse traits, model families, and selective-generalization baselines.
- Systematic Pareto analysis. The paper provides a useful characterization of the trade-off between desired-trait retention and undesired-trait suppression.
- Interesting empirical findings. The results on hard-to-elicit traits and surprising backdoors offer valuable insights into adapter-based selective generalization.
Critical weaknesses
Major Weaknesses
1. Missing comparison with Security Vectors.
Vanilla IA closely resembles Security Vectors (Zhou et al., 2024). Although the authors acknowledge this connection, a direct empirical comparison is missing. Evaluating the original Security Vectors method, including its KL regularization and bi-level optimization, on representative setups would better establish the paper's incremental contribution.
2. Unclear contribution of dynamic gating.
It remains unclear whether GIA/CGIA improve the Pareto frontier through input-dependent gating or simply by adjusting IA strength. Two ablations would help clarify this: (1) sweeping fixed IA scaling coefficients and (2) replacing dynamic gates with trainable input-independent coefficients.
Minor Weaknesses
- Task complexity. Several desired traits involve relatively simple stylistic or formatting behaviors. Generalization to more complex capabilities remains unclear.
- Backdoor trade-offs. GIA and especially CGIA introduce more surprising backdoors than vanilla IA. The implications of this trade-off could be discussed more explicitly.
Robustness Concern
Each configuration uses only one training seed. Additional seeds on representative setups would strengthen the conclusions, particularly given the wide confidence intervals in the aggregate Pareto analysis.
Questions for authors
Questions for the Authors
-
Comparison with Security Vectors: Could the authors compare IA/GIA/CGIA with the full Security Vectors method on representative tasks? This would better establish the empirical contribution beyond the closest prior work.
-
Contribution of dynamic gating: Do GIA/CGIA outperform simpler alternatives, such as fixed IA scaling or trainable input-independent gating? This would clarify the independent methodological contribution of the proposed gating mechanisms.
-
Robustness: Are the reported Pareto improvements consistent across multiple training seeds? This would strengthen the reliability of the empirical conclusions.
-
Backdoor mechanisms: Why do GIA/CGIA introduce more surprising backdoors than vanilla IA? Is this due to weaker suppression or additional conditionalization induced by gating? Clarifying this would improve the mechanistic understanding of the method and its safety trade-offs.
2 comments
I appreciate the additional results on comparison with security vectors and contribution of dynamic gating. The concern of robustness has also been solved by running multiple seeds. I therefore raise my score.
Thank you for a precise review, and for calling our results on hard-to-elicit traits and backdoors "interesting empirical findings". Your two ablations of the gated variants changed what the paper says about them. The changes are in blue or red in the revised PDF.
Q1 and weakness 1: comparison with security vectors
Done on seven setups; point 1 of the general reply gives the results.
Q2 and weakness 2: input-dependent gating or IA strength?
Good suggestion! We ran both ablations (Appendix E.5): the frozen IA scaled by a fixed factor from 0.2 to 0.8, on B1, B2, B3, U2, and E1, and GIA with trainable input-independent gates, on the same five setups.
Per setup, strength explains most of GIA's gain. GIA falls on the path traced by the fixed scales on four of five setups, and the input-independent gates on all five. E1 is the exception for GIA: desired trait 0.56 against 0.50 for the closest fixed scale and 0.46 for the input-independent gates, with overlapping confidence intervals. On B1, the input-independent gates learn a weaker strength and retain less than GIA, while a fixed scale matches GIA. GIA's gates also spread from fully closed to fully open, while the input-independent gates stay near their mean, and traces of specific gates show that they vary with the input, mostly from token to token (29 to 55% of the gate variance lies within gates; Appendix E.5). GIA thus learns an input-dependent policy, although this does not show by itself that the input dependence improves retention.
Averaged over the five setups, GIA still retains more of the desired trait than the fixed scale with similar suppression or the input-independent gates (normalized, coherence-aware: 0.87 against 0.71 and 0.75; over all answers: 0.81 against 0.71 to 0.75 and 0.73), as does CGIA against its fixed scale (0.86 against 0.68, coherence-aware), with overlapping confidence intervals. A fixed scale also has to be selected per setup, which the input-dependent or input-independent gates avoid.
The contributions now state both results. To the sentence on GIA and CGIA, we added: "Per setup, GIA mostly performs like the inoculation adapter scaled by a fixed factor, whose strength must be selected, while a variant of GIA whose gates ignore the input learns that strength itself." A new Discussion paragraph ("Does gating add more than a weaker adapter?", Section 5) and Appendix E.5 ("Fixed inoculation strength and input-independent gates") give the details.
Q3: robustness across seeds
Mostly yes, on the three setups we repeated (general reply, point 2). GIA retains more of the desired trait than IA at every seed on B2 and E1, and at four of five seeds on U2.
Q4 and minor weakness 2: why do GIA and CGIA introduce more backdoors?
The paper states the trade-off (Section 3.3): gating gives up part of IA's advantage on backdoors. Our analysis suggests the rise is mostly CGIA's. Its backdoor strength, averaged over the surprising prompt families, is +0.053 on B1 and +0.041 on B2, against +0.006 and +0.001 for IA, while GIA stays within 0.016 of IA.
On B1, weaker suppression does not explain the surprising backdoors. Without a system prompt, CGIA and IA express the trait equally (0.019 and 0.017), yet CGIA's backdoor is about eight times stronger, which points to a dependence on the context.
On B2 and B3, CGIA also expresses more of the trait without a system prompt (EM 0.083 against 0.016 for IA on B2), so there the two explanations are confounded.
This rests on one seed per setup and does not test either explanation directly (e.g., by evaluating IA at a fixed strength matched to CGIA's mean gate), so we left it out of the paper.
Minor weakness 1: task complexity
We agree. We chose stylistic and formatting traits because they can be measured reliably, and the Discussion now leaves more complex capabilities for future work.
====
If the security-vector comparison and the gating ablations answer your questions, we would be grateful if you would consider raising your score; otherwise, please tell us what remains unconvincing.