- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the Condition-Aware Network (CAN), which augments a shared backbone with lightweight condition-dependent affine adaptors. CAN is first jointly trained across several conditions and then adapted to unseen conditions by freezing the backbone and fine-tuning only the adaptors. Experiments cover adversarial image classification, image restoration, and adversarial text classification.
Critical strengths
- CAN provides a useful engineering alternative to maintaining many full specialist models. Tab. 9 reports clear savings in cumulative training time while keeping inference overhead small.
- The paper evaluates the conditioning mechanism on adversarial image classification and text classification. Tab. 6 shows that adaptor-only fine-tuning can recover performance at unseen perturbation budgets with only 5 additional epochs.
Critical weaknesses
- CAN closely resembles Once-for-All Adversarial Training [1]: both condition one model on robustness levels using feature modulation. Fourier encoding and FiLM appear incremental, while the Lipschitz bound provides no robustness guarantee.
- Condition-Matched CAN requires the exact attack budget or degradation type at inference, which may be unavailable in practice. Condition-Blind CAN fixes , largely eliminating dynamic switching. Moreover, adapting to new conditions still requires labeled data, adversarial-example generation, and five epochs of fine-tuning. Missing uncertainty estimates and inconsistent ResNet-18/VGG-style architecture descriptions further weaken the reliability of the reported comparisons.
[1] Once-for-All Adversarial Training, NeurIPS 2020.
Questions for authors
-
How does CAN differ technically and empirically from Once-for-All Adversarial Training? Please clarify why OAT is not discussed or used as a baseline. Which architecture produced Tables 1, 4–10: a standard ResNet-18 with residual connections, or the plain VGG-style 18-layer CNN described in Appendix A.2?
-
How is the condition value obtained at inference? What happens when the true perturbation budget or degradation category is unknown, incorrectly specified, or varies across samples? Are results averaged across multiple random seeds? If so, please provide standard deviations or confidence intervals for the main tables.
-
For Stage 2, how much labeled data and adversarial-example generation is used during the five adaptation epochs? Is this data volume identical to that used for training specialists?
9 comments
Weaknesses
W1
CAN closely resembles Once-for-All Adversarial Training [1]: both condition one model on robustness levels using feature modulation. Fourier encoding and FiLM appear incremental, while the Lipschitz bound provides no robustness guarantee.
Answer:
We thank the reviewer and agree that OAT [1] is the closest prior work; we will make the comparison more explicit in the paper. We address the three points in turn.
(1) Relation to OAT. Both methods use feature modulation to condition one network, but they condition on different quantities and solve different problems.
-
What is conditioned on. OAT conditions on the loss weight , a training hyperparameter, and every condition is trained against the same attack (). CAN conditions on the threat budget , a property of the attack faced at test time, and is trained against a distribution of budgets. CAN therefore models how the optimal features change with the threat model, which OAT does not.
-
Extension to new threat models. OAT’s discrete codes (random orthogonal vectors) carry no notion of order, so it cannot be queried at an unseen condition. CAN’s continuous budget encoding can be queried at any , and it adapts to a new budget by tuning only the 8.3k-parameter embedding MLP.
-
Empirical comparison. We reimplemented OAT on ResNet-18/CIFAR-10 with the same training recipe. Condition-Blind CAN, which receives no budget information at test time, reaches a summed accuracy of 365.71 over . This is higher than OAT even when OAT’s is chosen per budget with an oracle (358.00), and higher than OAT’s best fixed (346.99), while CAN uses 34% fewer parameters (11.69M vs. 17.79M).
(2) Are the design choices incremental? We agree that Fourier encoding and FiLM are not new as individual components, and we do not claim them as our contribution. Our contribution is the formulation: treating the threat budget as an input, so that one network covers a continuum of threat models and can be extended to new budgets cheaply. The design choices are nonetheless not interchangeable with OAT’s. To test this, we built a controlled variant that keeps the OAT architecture (discrete encoding, dual BN, FiLM) but, like CAN, is conditioned on and trained with CAN’s objective and budget distribution. The only remaining difference is the conditioning design. CAN outperforms this variant by (Condition-Matched: 373.32 vs. 360.16) and (Condition-Blind: 365.71 vs. 349.54) in summed accuracy. In addition, a condition-blind PGD-AT model trained with exactly CAN’s budget distribution reaches only 358.00. Conditioning, rather than the sampling distribution, therefore accounts for most of the gain.
Table R9. Summed PGD-20 accuracy (%) over on CIFAR-10 with ResNet-18. $^\dagger$$\lambda$ chosen per budget using test accuracy (oracle).
| Method | Condition | Sum | # Param. |
|---|---|---|---|
| PGD-AT, condition-blind, same distribution as CAN | – | 358.00 | 11.17M |
| OAT (best fixed ) | 346.99 | 17.79M | |
| OAT (oracle ) | 358.00 | 17.79M | |
| OAT-arch. (-cond.), Condition-Blind | 349.54 | 17.79M | |
| OAT-arch. (-cond.), Condition-Matched | 360.16 | 17.79M | |
| CAN (Condition-Blind) | 365.71 | 11.69M | |
| CAN (Condition-Matched) | 373.32 | 11.69M |
(3) The Lipschitz bound. The reviewer is correct that the bound is not a certified robustness guarantee, and it was not intended as one; as stated in Section 2.3, a Lipschitz bound alone does not imply robustness, since robustness also depends on the classification margin. Instead, the analysis explains the mechanism through which the condition controls the model’s behavior. Theorem 1 shows that, for a fixed condition , the Lipschitz constant of the conditioned model with respect to the input is bounded by , where the block constants belong to the shared backbone and do not depend on . Therefore, the condition modulates the input sensitivity of a single shared backbone only through the channel-wise scales . Empirically, CAN uses this knob in the expected direction: the learned scaling decreases with in every layer reported in Table 8, and the product decreases monotonically from at to at . That is, CAN reduces its sensitivity to input perturbations as the threat level increases, which is consistent with its improved robustness at larger budgets. In the revision, we have clarified that this result characterizes the conditioning mechanism rather than certifying robustness.
References
[1] Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuei Hu, Ji Liu, and Zhangyang Wang. Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
No author response was provided during the rebuttal period. Therefore, the main concerns raised in my original review remain unresolved, and I maintain my score of 2.