acceptodds
Under review as a conference paper at ICLR 2027

All-in-One Robust Model for Dynamic Environments

Abstract

Real-world machine learning systems are increasingly deployed in diverse and dynamic environments, where a model must handle a range of operating conditions and adapt to new ones as they arise. The standard solution, training and maintaining a separate specialist model for each condition, becomes increasingly costly as the number of conditions grows and new conditions continue to appear. We propose the Condition-Aware Network (CAN), which consists of a condition-invariant shared backbone and a lightweight conditional adaptor, allowing a single model to switch flexibly between operating conditions. CAN is trained with a two-stage recipe: joint training over a spectrum of conditions to build a shared representation, followed by fast adaptation that fine-tunes only the lightweight adaptor to a new condition without retraining the backbone. We instantiate CAN primarily in the context of adversarial image classification, and further validate the same conditioning mechanism on image restoration and adversarial text classification, showing that it generalizes across domains with structurally different notions of operating condition. Extensive experiments show that CAN substantially reduces the resources needed to cover multiple conditions while maintaining robustness comparable to a collection of independently trained specialist models. In the CIFAR-10 experiment with a ResNet-18 backbone, CAN reduces training GPU hours by 87.5% and stored parameters by 88.0%, while nearly matching the specialists' robust accuracy. Our code is available at https://anonymous.4open.science/r/CAN-CF44/.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Robustness, Representation Learning, Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LwwLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Condition-Aware Network (CAN), which augments a shared backbone with lightweight condition-dependent affine adaptors. CAN is first jointly trained across several conditions and then adapted to unseen conditions by freezing the backbone and fine-tuning only the adaptors. Experiments cover adversarial image classification, image restoration, and adversarial text classification.

Critical strengths
  1. CAN provides a useful engineering alternative to maintaining many full specialist models. Tab. 9 reports clear savings in cumulative training time while keeping inference overhead small.
  2. The paper evaluates the conditioning mechanism on adversarial image classification and text classification. Tab. 6 shows that adaptor-only fine-tuning can recover performance at unseen perturbation budgets with only 5 additional epochs.
Critical weaknesses
  1. CAN closely resembles Once-for-All Adversarial Training [1]: both condition one model on robustness levels using feature modulation. Fourier encoding and FiLM appear incremental, while the Lipschitz bound provides no robustness guarantee.
  2. Condition-Matched CAN requires the exact attack budget or degradation type at inference, which may be unavailable in practice. Condition-Blind CAN fixes ϵ=8/255\epsilon=8/255, largely eliminating dynamic switching. Moreover, adapting to new conditions still requires labeled data, adversarial-example generation, and five epochs of fine-tuning. Missing uncertainty estimates and inconsistent ResNet-18/VGG-style architecture descriptions further weaken the reliability of the reported comparisons.

[1] Once-for-All Adversarial Training, NeurIPS 2020.

Questions for authors
  1. How does CAN differ technically and empirically from Once-for-All Adversarial Training? Please clarify why OAT is not discussed or used as a baseline. Which architecture produced Tables 1, 4–10: a standard ResNet-18 with residual connections, or the plain VGG-style 18-layer CNN described in Appendix A.2?

  2. How is the condition value obtained at inference? What happens when the true perturbation budget or degradation category is unknown, incorrectly specified, or varies across samples? Are results averaged across multiple random seeds? If so, please provide standard deviations or confidence intervals for the main tables.

  3. For Stage 2, how much labeled data and adversarial-example generation is used during the five adaptation epochs? Is this data volume identical to that used for training specialists?

9 comments

No author response was provided during the rebuttal period. Therefore, the main concerns raised in my original review remain unresolved, and I maintain my score of 2.

Weaknesses

W1

CAN closely resembles Once-for-All Adversarial Training [1]: both condition one model on robustness levels using feature modulation. Fourier encoding and FiLM appear incremental, while the Lipschitz bound provides no robustness guarantee.

Answer:

We thank the reviewer and agree that OAT [1] is the closest prior work; we will make the comparison more explicit in the paper. We address the three points in turn.

(1) Relation to OAT. Both methods use feature modulation to condition one network, but they condition on different quantities and solve different problems.

  • What is conditioned on. OAT conditions on the loss weight λ\lambda, a training hyperparameter, and every condition is trained against the same attack (ϵ=8/255\epsilon=8/255). CAN conditions on the threat budget ϵ\epsilon, a property of the attack faced at test time, and is trained against a distribution of budgets. CAN therefore models how the optimal features change with the threat model, which OAT does not.

  • Extension to new threat models. OAT’s discrete λ\lambda codes (random orthogonal vectors) carry no notion of order, so it cannot be queried at an unseen condition. CAN’s continuous budget encoding can be queried at any ϵ\epsilon, and it adapts to a new budget by tuning only the 8.3k-parameter embedding MLP.

  • Empirical comparison. We reimplemented OAT on ResNet-18/CIFAR-10 with the same training recipe. Condition-Blind CAN, which receives no budget information at test time, reaches a summed accuracy of 365.71 over {0,1,2,4,8}/255\{0,1,2,4,8\}/255. This is higher than OAT even when OAT’s λ\lambda is chosen per budget with an oracle (358.00), and higher than OAT’s best fixed λ\lambda (346.99), while CAN uses 34% fewer parameters (11.69M vs. 17.79M).

(2) Are the design choices incremental? We agree that Fourier encoding and FiLM are not new as individual components, and we do not claim them as our contribution. Our contribution is the formulation: treating the threat budget as an input, so that one network covers a continuum of threat models and can be extended to new budgets cheaply. The design choices are nonetheless not interchangeable with OAT’s. To test this, we built a controlled variant that keeps the OAT architecture (discrete encoding, dual BN, FiLM) but, like CAN, is conditioned on ϵ\epsilon and trained with CAN’s objective and budget distribution. The only remaining difference is the conditioning design. CAN outperforms this variant by +13.2+13.2 (Condition-Matched: 373.32 vs. 360.16) and +16.2+16.2 (Condition-Blind: 365.71 vs. 349.54) in summed accuracy. In addition, a condition-blind PGD-AT model trained with exactly CAN’s budget distribution reaches only 358.00. Conditioning, rather than the sampling distribution, therefore accounts for most of the gain.

Table R9. Summed PGD-20 accuracy (%) over {0,1,2,4,8}/255\{0,1,2,4,8\}/255 on CIFAR-10 with ResNet-18. $^\dagger$$\lambda$ chosen per budget using test accuracy (oracle).

MethodConditionSum# Param.
PGD-AT, condition-blind, same distribution as CAN–358.0011.17M
OAT (best fixed λ\lambda)λ\lambda346.9917.79M
OAT (oracle λ\lambda)†^\daggerλ\lambda358.0017.79M
OAT-arch. (ϵ\epsilon-cond.), Condition-Blindϵ\epsilon349.5417.79M
OAT-arch. (ϵ\epsilon-cond.), Condition-Matchedϵ\epsilon360.1617.79M
CAN (Condition-Blind)ϵ\epsilon365.7111.69M
CAN (Condition-Matched)ϵ\epsilon373.3211.69M

(3) The Lipschitz bound. The reviewer is correct that the bound is not a certified robustness guarantee, and it was not intended as one; as stated in Section 2.3, a Lipschitz bound alone does not imply robustness, since robustness also depends on the classification margin. Instead, the analysis explains the mechanism through which the condition controls the model’s behavior. Theorem 1 shows that, for a fixed condition ϵ\epsilon, the Lipschitz constant of the conditioned model with respect to the input is bounded by ∏ℓsℓ(ϵ)Λℓ\prod_\ell s_\ell(\epsilon)\Lambda_\ell, where the block constants Λℓ\Lambda_\ell belong to the shared backbone and do not depend on ϵ\epsilon. Therefore, the condition ϵ\epsilon modulates the input sensitivity of a single shared backbone only through the channel-wise scales sℓ(ϵ)s_\ell(\epsilon). Empirically, CAN uses this knob in the expected direction: the learned scaling Γℓ(ϵ)\Gamma_\ell(\epsilon) decreases with ϵ\epsilon in every layer reported in Table 8, and the product ∏ℓ(1+Γℓ(ϵ))\prod_\ell(1+\Gamma_\ell(\epsilon)) decreases monotonically from 1.761.76 at ϵ=0\epsilon=0 to 1.311.31 at ϵ=8/255\epsilon=8/255. That is, CAN reduces its sensitivity to input perturbations as the threat level increases, which is consistent with its improved robustness at larger budgets. In the revision, we have clarified that this result characterizes the conditioning mechanism rather than certifying robustness.

References
[1] Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuei Hu, Ji Liu, and Zhangyang Wang. Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free. In Advances in Neural Information Processing Systems (NeurIPS), 2020.

Reviewer_8g4AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a Condition-Aware Network (CAN) that shares a backbone across operating conditions and uses Fourier-encoded condition inputs to generate lightweight, channel-wise feature modulations. Its main contributions are a two-stage training procedure—joint training across conditions followed by adaptor-only fine-tuning for new conditions—and a Lipschitz analysis of the modulation mechanism. Validation covers adversarial image classification, image restoration, and adversarial text classification, reporting broadly competitive performance against condition-specific specialists.

Critical strengths

1.Strong practical efficiency with competitive performance. In the CIFAR-10 experiment covering 11 perturbation budgets, CAN reports 87.5% fewer training GPU hours and 88.0% fewer stored parameters than independently trained specialists, while largely matching their robust accuracy. This is the strongest argument for acceptance: it addresses a concrete deployment cost with measured savings and limited performance sacrifice.

2.Effective adaptation through a small trainable component. Freezing the backbone and fine-tuning only the adaptor for five epochs brings CAN close to matched specialists across six additional budgets, including two beyond the initial training range. This supports the usefulness of the shared representation and makes the framework valuable when operating requirements expand after initial training.

Critical weaknesses

1.Efficiency comparisons omit stronger adaptation baselines. The headline savings compare CAN against training every specialist from scratch. Without comparisons to warm-started fine-tuning or other parameter-efficient adaptation methods, it remains unclear how much efficiency is specific to CAN.

2.The effect of conditioning is not cleanly isolated. CAN uses a clean-biased budget distribution, whereas the uniform-budget PGD-AT baseline uses a different sampling scheme. A condition-blind model trained with exactly the same distribution is needed to distinguish architectural benefits from sampling effects.

3.Evidence for unseen-condition generalization is limited. Four of the six new budgets lie inside the original training range; at the two outside budgets, accuracy before adaptation falls to 0.59% and 1.57%. These results support inexpensive supervised adaptation, but provide weaker support for the broader claim of robustness in dynamic environments.

Questions for authors

1.Which architectures produced the reported results? The main text specifies ResNet-18 and WideResNet, while Appendix A.2 describes a plain network without residual connections. Were CAN and its baselines evaluated with identical backbones?

2.What distinguishes CAN from OAT and FLOAT beyond the conditioning variable and adaptation stage? Are comparable results available that establish the benefit of these differences?

3.Does CAN retain its advantage when the condition-blind baseline uses the same clean-biased budget sampling distribution? This would clarify whether the gains arise from conditioning or the training distribution.

7 comments

Weaknesses

W1

Efficiency comparisons omit stronger adaptation baselines. The headline savings compare CAN against training every specialist from scratch. Without comparisons to warm-started fine-tuning or other parameter-efficient adaptation methods, it remains unclear how much efficiency is specific to CAN.

Answer: We thank the reviewer for this point, which we agree with, and we have added the requested baselines. Every adaptation method starts from the same ResNet-18 model trained with PGD-AT at ϵ=8/255\epsilon=8/255, and adapts to six budgets unseen during training, ϵ∈{3,5,6,7,9,10}/255\epsilon\in\{3,5,6,7,9,10\}/255, using the same 5 epochs of training with PGD-7 adversarial examples generated at the new budget:

  • Warm-start full fine-tuning: all 11.17M parameters are fine-tuned with SGD. We swept lr∈{0.001,0.01,0.05}\text{lr}\in\{0.001, 0.01, 0.05\} and report the best lr for each budget, chosen on the test set, which favors this baseline.

  • BN-only: the BN affine parameters are re-trained and the running statistics are re-estimated, with all other weights frozen.

  • LoRA: rank r∈{1,4}r\in\{1,4\} adapters are added to every convolution and the classifier, with the backbone frozen.

  • CAN Stage 2: only the 8.3k-parameter condition-embedding MLP (out of the 0.52M conditioning module) is tuned, with the backbone and the FiLM heads frozen.

The results are reported in Table R5, from which we draw three conclusions.

(1) The saving in training time comes mainly from warm-starting. The reviewer is right that this saving is not specific to CAN. Every 5-epoch adaptation method costs about 1.0–1.4 GPU hours for all six budgets, a reduction of about 97% compared with the 45 GPU hours needed to train six specialists from scratch. We will revise the paper so that the time saving is attributed to warm-starting in general, rather than claimed as a unique advantage of CAN.

(2) CAN achieves the best robust accuracy among the adaptation methods. CAN reaches an average robust accuracy of 51.10%, matching the specialists trained from scratch (51.13%) and slightly exceeding warm-start full fine-tuning with a per-budget oracle learning rate (51.03%), LoRA (50.54% and 50.98%), and BN-only (49.47%). The advantage is most visible at 3/2553/255, where CAN reaches 72.78%, close to the specialist (72.87%), while all other adaptation methods remain at or below 70.94%.

(3) CAN requires the least storage and keeps a single model. We first clarify the two parameter counts reported for CAN, which refer to different parts of the model (Table R4). The 0.52M parameters are the full conditioning module that CAN adds to ResNet-18 (11.17M →\rightarrow 11.69M), consisting of the Fourier frequencies, a shared embedding MLP, and a linear FiLM head at each of the 17 modulation sites. All of them are trained jointly with the backbone in Stage 1 and shared by all budgets. In Stage 2, only the 8.3k-parameter embedding MLP is fine-tuned, while the backbone, the frequencies, and the FiLM heads remain frozen and shared. Therefore, each new budget requires storing only these 8.3k parameters.

Table R4. Parameter breakdown of CAN with ResNet-18.

Module# Param.Stage 1Stage 2
ResNet-18 backbone11,173,962trainedfrozen
Fourier frequencies32trainedfrozen
Embedding MLP (64→64→6464\rightarrow64\rightarrow64)8,320trainedfine-tuned
FiLM heads (17 sites)507,520trainedfrozen
Conditioning module (total)515,872
CAN (total)11,689,834

Accordingly, CAN stores only 8.3k parameters per new budget (−99.93%-99.93\% relative to a specialist), which is 2.3×2.3\times fewer than BN-only, 4.4×4.4\times fewer than LoRA (r=1r=1), 17×17\times fewer than LoRA (r=4r=4), and 1,340×1{,}340\times fewer than full fine-tuning. Moreover, since the budget enters CAN as an input, all budgets are served by one network that switches between them by changing the condition, rather than by loading a separately adapted model or adapter for each budget. Covering all 11 budgets thus requires 11.74M parameters in total (one 11.69M CAN model plus six 8.3k embedding MLPs), compared with 122.91M for 11 specialists, a 90.4% reduction. We have corrected the storage reported in Table 9 and Figure 1 accordingly, which previously counted a full copy of the 0.52M conditioning module for each new budget.

Table R5. Adapting to six budgets unseen during training on CIFAR-10 with ResNet-18. RA is PGD-20 robust accuracy at each new budget. GPU hours are summed over the six budgets. Storage counts the parameters kept per new budget on top of the shared base model. Percentages are relative to the specialists trained from scratch. ∗^\astLearning rate chosen per budget on the test set. ‡^\ddaggerExtrapolated from one measured epoch.

Method3/2553/2555/2555/2556/2556/2557/2557/2559/2559/25510/25510/255Avg RAGPU hStorage / budget
Specialist from scratch (200 ep.)72.8759.0453.1947.6739.3334.7051.1345.09$^\ddagger$11.17M
PGD-AT 8/2558/255, no adaptation70.9760.1154.3848.7138.0832.9650.870 (−100%-100\%)0 (−100%-100\%)
Warm-start full FT$^\ast$70.9459.2953.6248.2038.7335.3951.031.14 (−97.5%-97.5\%)11.17M (±0%\pm0\%)
BN-only69.4257.7052.1846.8637.5533.0849.471.03 (−97.7%-97.7\%)19.2k (−99.83%-99.83\%)
LoRA (r=1r=1)70.3458.8253.5548.2738.1734.0950.541.41 (−96.9%-96.9\%)36.3k (−99.68%-99.68\%)
LoRA (r=4r=4)70.3159.5654.0248.8338.8434.3150.981.42 (−96.9%-96.9\%)145.3k (−98.70%-98.70\%)
CAN (Ours)72.7859.1953.3448.0539.0834.1651.101.01 (−97.8%-97.8\%)8.3k (−99.93%-99.93\%)

W2

The effect of conditioning is not cleanly isolated. CAN uses a clean-biased budget distribution, whereas the uniform-budget PGD-AT baseline uses a different sampling scheme. A condition-blind model trained with exactly the same distribution is needed to distinguish architectural benefits from sampling effects.

Answer:

We thank the reviewer for this suggestion and have run exactly this control. We trained a condition-blind ResNet-18 with PGD-AT using the same clean-biased budget distribution as CAN, {0,0,0,1,2,4,8}/255\{0,0,0,1,2,4,8\}/255 (i.e., M=3M=3). Budgets are assigned to samples in the same way, and the optimizer, schedule, weight decay, PGD-7 training attack, and checkpoint selection (epoch 150) all match CAN. The only difference from CAN is that this model receives no budget condition and has no FiLM layers. Results are in Table R6.

Table R6. Isolating the effect of conditioning on CIFAR-10 with ResNet-18 (PGD-20 accuracy, %). The clean-biased rows use the same budget distribution, {0,0,0,1,2,4,8}/255\{0,0,0,1,2,4,8\}/255. The condition-blind PGD-AT row with clean-biased sampling is averaged over two seeds.

MethodTraining distribution0/255\mathbf{0/255}1/255\mathbf{1/255}2/255\mathbf{2/255}4/255\mathbf{4/255}8/255\mathbf{8/255}Sum# Param.
PGD-AT (condition-blind)Uniform {0,1,2,4,8}\{0,1,2,4,8\}87.5983.7878.5867.0139.21356.1711.17M
PGD-AT (condition-blind)Clean-biased (same as CAN)88.7584.5779.3466.9538.38358.0011.17M
CAN (Condition-Blind)Clean-biased87.4483.7379.5869.8045.16365.7111.69M
CAN (Condition-Matched)Clean-biased93.8587.4680.4966.3645.16373.3211.69M

The results separate the two effects clearly.

  • The sampling distribution alone contributes little. Switching condition-blind PGD-AT from uniform to clean-biased sampling raises the summed accuracy by only +1.8+1.8 (356.17 →\rightarrow 358.00). The extra clean samples give small gains at 00–2/2552/255, but these are offset by a loss at 8/2558/255 (39.21 →\rightarrow 38.38).

  • Conditioning accounts for most of the improvement. Under the identical distribution, CAN improves over the condition-blind model by +7.7+7.7 when given no budget information (Condition-Blind, fixed ϵ=8/255\epsilon=8/255) and by +15.3+15.3 when given the evaluation budget (Condition-Matched).

  • A single condition-blind network cannot reconcile the budgets. With the same data, the condition-blind model must trade clean accuracy against robustness at 8/2558/255 (38.38). With the evaluation budget as input, CAN is better at four of the five budgets and is within 0.6 points at 4/2554/255, with gains of +5.1+5.1 at 0/2550/255 and +6.8+6.8 at 8/2558/255. Using the conditioning value, a single set of weights covers several clean–robust trade-offs, which a different sampling distribution alone does not provide.

We will add this control to Table 4 and correct the text that describes the baseline’s sampling.

Reviewer_VjqROpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses the training and storage costs of maintaining separate models for different operating conditions, particularly adversarial perturbation budgets. It proposes the Condition-Aware Network (CAN), which combines a shared backbone with lightweight conditional adaptors. CAN encodes the condition using Fourier features and uses MLPs to generate channel-wise scaling and bias parameters. Training comprises joint optimisation across conditions, followed by adaptation to a new condition through five epochs of adaptor-only fine-tuning. Experiments cover adversarial image classification, image restoration, and adversarial text classification. Compared with training 11 specialists from scratch, CAN reportedly reduces GPU training time by 87.5% and stored parameters by 88.0%, while approaching specialist performance. These findings primarily support efficient parameter sharing and supervised adaptation under controlled conditions.

Critical strengths
  1. The paper addresses a practical problem: reducing the cost of maintaining multiple condition-specific models.
  2. Table 9 reports training time, storage requirements, and inference cost, providing concrete evidence of efficiency.
  3. Cross-budget evaluations and adaptation analyses distinguish performance under matched, mismatched, and newly introduced conditions.
Critical weaknesses
  1. The baseline coverage is insufficient. Tables 2 and 4 omit closely related conditional and unified methods, only include several old methods before 2024.
  2. The principal datasets have limited representativeness. The adversarial image-classification experiments rely exclusively on CIFAR-10/100, which cannot adequately establish performance on complex, high-resolution deployment data. The restoration and text extensions provide useful supplementary evidence but do not resolve this limitation.
  3. The backbone specification is inconsistent. Section 3.1 reports ResNet-18, whereas Appendix A.2 describes a network without residual connections. This potentially correctable inconsistency must be resolved before the fairness of the comparisons can be assessed.
Questions for authors
  1. What distinguishes CAN from existing conditional robustness methods such as OAT, and are any comparisons with relevant unified models or warm-start baselines already available?
  2. What existing evidence supports generalisation beyond CIFAR-scale adversarial classification? Would the authors narrow their claims where broader validation is unavailable?
  3. Which backbone was actually used in the reported experiments, and were its architecture and training settings matched across CAN and the baselines?
10 comments

No author response was provided during the rebuttal period. Therefore, the main concerns raised in my original review remain unresolved, and I maintain my score of 1.

W1

The baseline coverage is insufficient. Tables 2 and 4 omit closely related conditional and unified methods, only include several old methods before 2024.

Answer:

We thank the reviewer for this suggestion. Following it, we add closely related conditional and unified baselines for both adversarial image classification (Table 4) and image restoration (Table 2).

(1) Adversarial image classification. We add two representative methods that obtain multiple accuracy–robustness operating points from a single model:

  • OAT [1], the most closely related conditional method, conditions a single model on the trade-off weight λ\lambda of the adversarial training loss through FiLM layers and dual batch normalization. Following the original paper, it is trained at a fixed budget ϵ=8/255\epsilon=8/255, so that λ\lambda controls the balance between clean accuracy and robustness.

  • MixedNUTS [2], a recent training-free unified method, mixes the outputs of a standard and a robust ResNet-18, with the mixing strength controlled by β\beta. It requires both base models at inference, i.e., 2×2\times parameters and FLOPs.

For both methods, we report two settings: (i) the best fixed knob, where a single λ\lambda or β\beta is used for all budgets, and (ii) an oracle knob, where λ\lambda or β\beta is selected separately for each budget using test accuracy. The oracle setting is an upper bound that is not available at deployment, and thus favors these baselines. All methods use the ResNet-18 backbone, CIFAR-10 training data, and PGD-20 evaluation as in Table 4.

Table R1. Accuracy (%) under PGD-20 on CIFAR-10 with ResNet-18. Sum adds accuracy over the five budgets. †^\daggerKnob selected per budget using test accuracy (upper bound).

Method0/255\mathbf{0/255}1/255\mathbf{1/255}2/255\mathbf{2/255}4/255\mathbf{4/255}8/255\mathbf{8/255}Sum# Param.
Natural Training95.0957.868.940.140.00162.0311.17M
PGD-AT (ϵ=8/255\epsilon=8/255)84.0780.2775.8265.7143.10348.9711.17M
PGD-AT (Uniform ϵ\epsilon)87.5983.7878.5867.0139.21356.1711.17M
TRADES (λ=2.0\lambda=2.0)81.9278.0073.5162.8941.50337.8211.17M
TRADES (λ=0.2\lambda=0.2)91.4962.7427.5810.273.31195.3911.17M
MART (λ=5.0\lambda=5.0)80.9277.3673.4664.5744.85341.1611.17M
MART (λ=0.5\lambda=0.5)82.2978.1873.6663.5642.25339.9411.17M
NuAT80.2377.6175.1069.1346.18348.2511.17M
MoRE84.5277.1072.8063.1041.22338.7455.87M
Ensemble94.4687.2773.9549.8920.13325.7055.87M
OAT [1] (best fixed λ=0.4\lambda=0.4)85.9081.4476.3564.5738.73346.9917.79M
OAT [1] (oracle λ\lambda)†^\dagger91.5682.6076.8664.5742.41358.0017.79M
MixedNUTS [2] (best fixed β=99.5%\beta=99.5\%)84.2280.2775.8865.6442.75348.7622.35M
MixedNUTS [2] (oracle β\beta)†^\dagger95.0980.2775.8865.6442.75359.6322.35M
CAN (Condition-Blind)87.4483.7379.5869.8045.16365.7111.69M
CAN (Condition-Matched)93.8587.4680.4966.3645.16373.3211.69M

We make the following observations:

  • CAN outperforms the conditional and unified baselines, even when they use an oracle knob. Condition-Matched CAN achieves the highest summed accuracy (373.32373.32), exceeding OAT with oracle λ\lambda (358.00358.00) and MixedNUTS with oracle β\beta (359.63359.63) by 15.3215.32 and 13.6913.69, respectively. Even Condition-Blind CAN, which uses a single fixed condition for all budgets, achieves 365.71365.71 and outperforms both oracle baselines, which select their knob per budget using test accuracy.

  • Conditioning on the threat level is more effective than conditioning on the accuracy–robustness trade-off. OAT and MixedNUTS only trade off clean accuracy against robustness at a fixed training budget. As a result, their robustness at larger budgets is bounded by that of the underlying robust model, even with the oracle knob (e.g., 64.57%64.57\% and 42.41%42.41\% for OAT at 4/2554/255 and 8/2558/255). By conditioning directly on the threat level, CAN specializes its representation to each budget and achieves higher accuracy at every budget except 0/2550/255, where it remains close to the best result (93.85%93.85\%).

Comments on the paper

1 comment

We sincerely thank all reviewers for their time and constructive feedback. We are encouraged that the reviewers recognize several strengths of our work. Reviewers find that our work addresses a practical problem, reducing the cost of maintaining multiple condition-specific models, and provides a useful engineering alternative to maintaining many full specialist models (VjqR, LwwL). They acknowledge the strong practical efficiency of CAN, supported by concrete measurements of training time, storage, and inference cost, with limited performance sacrifice relative to independently trained specialists (VjqR, 8g4A, LwwL). They also find the lightweight adaptation effective, as adaptor-only fine-tuning for only five epochs brings CAN close to the matched specialists at newly introduced budgets (8g4A, LwwL). Finally, the reviewers appreciate our cross-budget evaluations and adaptation analyses, which distinguish performance under matched, mismatched, and newly introduced conditions (VjqR), as well as the evaluation of the same conditioning mechanism across adversarial image classification, image restoration, and adversarial text classification (8g4A, LwwL).

We have carefully addressed all comments in our point-by-point responses to each reviewer, including additional experiments and analyses that further support our claims. We have also revised the manuscript accordingly, with all changes highlighted in blue in the revised PDF. We hope our responses address the reviewers’ concerns, and we would be happy to answer any further questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.