acceptodds
Under review as a conference paper at ICLR 2027

Randomized Feature Squeezing against Unseen Attacks without Adversarial Training

Abstract

Deep learning has made tremendous progress in the last decades; however, it is not robust to adversarial attacks. The most effective approach is perhaps adversarial training, although it is impractical because it requires prior knowledge about the attackers and incurs high computational costs. In this paper, we propose a novel approach that can train a robust network only through standard training with clean images without awareness of the attacker's strategy. We add a specially designed network input layer, which accomplishes a randomized feature squeezing to reduce the malicious perturbation. It achieves excellent robustness against unseen and attacks at one time in terms of the computational cost of the attacker versus the defender through just 100 and 50 epochs of standard training with clean images in CIFAR-10 and ImageNet respectively. The thorough experimental results validate the high performance. Moreover, it can also defend against unlearnable examples generated by One-Pixel Shortcut which breaks down the adversarial training approach.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Adversarial Training, Randomized Feature Squeezing, Unseen $ {l_p} $ Attacks

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_h7JPOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an randomized adversarial defense with no knowledge of attacks. The main idea is to design a randomized feature squeezing input processing layer which is learnt during training. The input layer utilizes Gaussian noise addition, 2d convolution, and reciprocal. The authors modify the loss function to work with the proposed special input layer. After training, the learned 2d convolution and reciprocal are replaced with the sign function. The simplified input processing that serves as extreme feature squeezing is used for inference. The authors evaluate the robustness agains different lp attacks and show better performance across attacks. The proposed defense uses only clean images, so there is no overhead during training. Experiments include adaptive attacks such as BPDA and EOT to show the robustness. In addition, the proposed defense can also bypass unlearnable examples (one-pixel shortcut).

Critical strengths

The special processing layer is interesting and the settings, training only with clean images without requiring any attack-specific knowledge reflects practical settings. The layer can be plugged into different networks. Its simple cheap mechanism does not add additional training cost. The special layer is learnt during training before squeezing.

Cross attack results are interesting. The experiments include ImageNet-scale evaluation and consider adaptive attacks.

Critical weaknesses

The novelty may be a bit over-claimed. The specific reciprocal multiplication may be new, but learned input squeezing inside the network and randomized layer as a defense are not new. For example, spartan networks, QuSecNets, thermometer encoding. https://www.sciencedirect.com/science/article/abs/pii/S0167404819300069 https://arxiv.org/pdf/1811.01437 https://openreview.net/forum?id=S18Su--CW This, "the only one that does not require any prior knowledge about the attacks with standard training with clean images" may be over claimed. Randomized smoothing and feature denoising (cited in the paper) or diffpure do not require prior knowledge. The experiments do not show accuracy vs noise budget.

Questions for authors

How do you sample ImageNet 5k for clean accuracy and 1k for robust accuracy?

4 comments
AuthorsOpenReview ↗

Response on novelty and related work.

We thank the reviewer for the positive assessment and we agree that parts of our novelty claim were over-stated. We will revise them as follows.

(1) On the components. We agree that learned input squeezing inside the network (Spartan Networks, QuSecNets) and randomized layers as a defense are not new, and we will cite these works explicitly in the revision, together with thermometer encoding. Our claim will be narrowed to the specific combination we introduce: a reciprocal-multiplication input layer trained end-to-end with a sparse loss, a detour training scheme whose auxiliary conv/reciprocal/Sigmoid path is discarded at inference, and a test-time predictor that reduces to Sign(x+ϵ)\mathrm{Sign}(x+\epsilon).

(2) On "the only one that does not require any prior knowledge." We agree this is an over-claim and we will remove it. Randomized smoothing, feature denoising, and DiffPure indeed do not require knowledge of the attacker, so "no attack prior" alone does not distinguish our method. We will restate the claim precisely as: among methods trained only with standard training on clean images and no attack-specific prior, ours is, to our knowledge, the only one that reports robustness against l0,l1,l2,l∞l_0,l_1,l_2,l_\infty and OPS at once without data augmentation or a pre-trained generative model.

(3) On accuracy vs. noise budget. We agree that a curve against the perturbation budget ϵ\epsilon (attack strength) is missing, and we will add it for both CIFAR-10 and ImageNet. Results at different training noise levels σ\sigma are already reported in Tables 9 and 10.

AuthorsOpenReview ↗

Response on how ImageNet images are sampled for clean and robust evaluation.

For the 5K evaluation, we follow the RobustBench protocol and use the 5,000 images it specifies for ImageNet.

For the 1K evaluation, we use the first 1,000 images of this same 5,000-image set.

Thank you for clarifying the novelty and evaluation dataset. Since the paper bring new components that are different from traditional defenses, I increase my score.

We thank the reviewer for the re-evaluation and for the positive update. We are glad the clarifications were helpful, and we appreciate the recognition of the method's distinction from traditional defenses.

Reviewer_WvHFOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper trains image classifiers to resist different adversarial perturbations without generating adversarial examples during training. The input layer normalizes the image and adds Gaussian noise. A learned convolutional branch then controls how a sigmoid converts these values toward zero or one. At test time, this branch is removed and a fixed threshold produces binary inputs. Experiments on CIFAR-10 and ImageNet cover several perturbation norms and attacks that account for the randomization. Results show some resistance to these attacks and One-Pixel Shortcut poisoning.

Critical strengths
  1. I am impressed to learn that the method avoids the cost of generating adversarial examples during training, and using a learnable module just works.

  2. The design requires little inference-time overhead, while still offering strong security against attacks which were never seen in the training.

  3. The authors are clever in also testing unlearnable samples (a different type of attacks that poison the training and produce useless models on benign inputs). OPS is a specific type of noise that can be well mitigated by the design: 56.25% accuracy, versus 15.56% for standard training and 11.08% for adversarial training.

Critical weaknesses
  1. “Does the attacker know and optimize against the exact noise sample that will be used for the final prediction?” That is not reported for the main stochastic defense. The fixed-noise experiment addresses a related setting, but changes the prediction rule to an average over several fixed draws. This is actually the fundamental worst-case attack one can perform - no matter what noise is added, optimize against this specific sampled noise.

  2. The presentation is poor with repeated bolded sentences highlighting the same novelty claim. The introduction contains a self-evaluation “which is very novel” - this offers no information to the readers. Whether the work is novel needs evaluations from the community, and repeating the statements will not strengthen that. The paper definitely lacks a rigorous editorial pass: there is a weird “than then st” at the end of Table 3’s caption, and I suspect there are much more mistakes like this. It is a bit hard to understand the method’s superiority in security when looking at Table 2 / 3 (why are equations here) / 4 (why is N.T. and A.T. abbreviated here even you have space)

  3. The paper explains how to train the noisy binary classifier, but not why it should be robust to adaptive attacks. Binarization and noisy gradients do not themselves establish robustness, and Sec. 6 does not justify this claim for the actual nonlinear, stochastic model.

Questions for authors

See weaknesses

4 comments
AuthorsOpenReview ↗

Response to the question on the noise-aware attacker.

Response to the question on the noise-aware attacker.

We thank the reviewer, and we would like to clarify a conceptual point that we believe underlies this question.

If the attacker knows the exact noise realization ϵ∗\epsilon^{\ast} used for the final prediction, the defended model becomes the deterministic function f(x)=Fnet(Sign(x+ϵ∗))f(x)=F_{\text{net}}(\text{Sign}(x+\epsilon^{\ast})). The randomized defense then reduces to a deterministic classifier, and the attacker is no longer attacking a randomized defense—it is attacking a fixed network. This is not a "worst-case attack on the randomized defense"; it is a different threat model—one in which the attacker is given the final noise realization, which is a strictly stronger information assumption than the standard randomized-defense setting. The same holds for any randomized defense (e.g., randomized smoothing): once the noise is known, the defense is deterministic by definition.

We agree this is a legitimate and strong setting, and it is exactly what we study in Section 6. There we fix the noise, obtain a deterministic predictor, and run APGD-l1l_1/l2l_2/l∞l_\infty (Tables 11–12). The reviewer's setting corresponds to N=1N=1 in that framework (a single fixed draw); our Section 6 reports N=5/20/30N=5/20/30 averaged over fixed seeds, which is the same deterministic-predictor threat model with a smoother decision boundary. We will report the N=1N=1 single-fixed-draw case in the revision.

To be explicit: the fresh-noise results in Tables 2/3/5/6/8 are under the standard randomized-defense threat model (the attacker does not know the final ϵ\epsilon, but may query arbitrarily and use EOT). Section 6 is an additional deterministic probe, where we deliberately fix the noise to check whether the defense relies entirely on randomness; it does not. We do not claim the latter is weaker; we claim it is a different setting, and we report both.

AuthorsOpenReview ↗

Response on presentation and editorial issues.

We thank the reviewer and fully agree. Below is a point-by-point list of the changes we will make in the revision.

Reviewer commentOur responseWhere revised
Self-evaluation "which is very novel" offers no informationRemoved. All self-evaluative wording ("very novel", "the only one", etc.) deleted from abstract, intro, and contributions. We state what is new relative to prior work once, factually, without self-evaluative wording, and leave the judgment to the community.Abstract; Sec. 1 (Contributions)
Repeated bolded sentences highlighting the same novelty claimDeleted. The introduction states what is new relative to prior work once, without bold emphasis; contributions rewritten as three neutral, verifiable statements.Sec. 1
"than then st" fragment at end of Table 3 caption; likely more such mistakesFixed. A full proofreading pass covers all captions, table headers, abbreviations, and cross-references.Table 3 caption; whole manuscript
Table 2: mathematical notation appears inside the table without explanationColumn headers and the caption made self-explanatory: attack names and norms are written out, the Square/SignHunter distinction is explained in the caption, and the table is restructured into four clearly separated blocks (attacks / defenses / robust accuracy / iterations).Table 2; Sec. 4
Table 3: mathematical expressions in the header row are unexplainedColumn headers rewritten to be self-explanatory, e.g., "APGD-l1l_1 (BPDA: Softsign)", "PGD-l2l_2 (BPDA: Softsign)", "DDN-l2l_2 (BPDA: Sigmoid(9x))", "PGD-l∞l_\infty (BPDA: Sigmoid(3x))", with the BPDA choice per attack stated in the caption.Table 3 caption and headers
Table 4: "N.T." / "A.T." abbreviated despite available spaceSpelled out as "Normal training" / "Detour training" in caption; terminology unified throughout.Table 4 caption; Sec. 5.4
Hard to see the method's superiority in security from Tables 2–4Added a self-contained one-sentence takeaway under each table, so each table states what is measured, the threat model, the number of images, and the BPDA choice.Tables 2–4 captions
AuthorsOpenReview ↗

Response on why the method is robust to adaptive attacks.

We thank the reviewer for this substantive point. We agree that binarization and noisy gradients are not, by themselves, a robustness argument, and we will state the scope of the Section 6 analysis explicitly. The Rademacher analysis there is carried out for a linear sign model, which is a deliberate simplification; such simplified analyses are standard in this literature (including the work we cite), and they are meant to provide intuition rather than a guarantee for the full nonlinear, stochastic network. We will present it as such, and we will not rely on it to claim provable robustness. The mechanism we do rely on is the one below, and it is supported by the experiments in the paper.

Mechanism: the set of reachable discrete inputs is small, not the gradient masked. At test time the model computes Fnet(Sign(x+ϵ))F_{\text{net}}(\mathrm{Sign}(x+\epsilon)), so the effective input space is {0,1}d\lbrace 0,1 \rbrace^d, not Rd\mathbb{R}^d. An adaptive ℓp\ell_p attack of radius ϵ\epsilon can change the prediction only if it flips the sign of at least one coordinate of x+ϵx+\epsilon. For coordinates with ∣xi∣>ϵ|x_i|>\epsilon—the vast majority under the ϵ\epsilon's used in our experiments—the sign is invariant to the perturbation, so the attack must concentrate its budget on the few coordinates near zero. This is a restriction on the set of reachable discrete inputs, not a claim that gradients are uninformative. The randomized feature squeezing learned through the sparse loss pushes most coordinates into the saturated region, which is what makes this set small.

This assumption is empirically verified on the full test sets. For every image in the test set and every pixel and channel, we record whether ∣xi(k)∣>ϵ|x_i^{(k)}| > \epsilon. For CIFAR-10, 98.3% of these components satisfy ∣xi(k)∣>8/255|x_i^{(k)}| > 8/255; for ImageNet, 99.1% satisfy ∣xi(k)∣>4/255|x_i^{(k)}| > 4/255, where xi(k)x_i^{(k)} denotes the kk-th component of the input xix_i. Equivalently, only 1.7% (CIFAR-10) and 0.9% (ImageNet) of the components satisfy ∣xi(k)∣≤ϵ|x_i^{(k)}| \le \epsilon, confirming that the condition ∣xi(k)∣>ϵ|x_i^{(k)}|>\epsilon assumed in the sign-invariance argument holds for the overwhelming majority of input dimensions. Note that a coordinate with ∣xi(k)∣>ϵ|x_i^{(k)}|>\epsilon cannot change sign under any ℓp\ell_p perturbation of radius ϵ\epsilon, so this statistic directly bounds the fraction of coordinates an adaptive attacker can flip. We will include these statistics in the final version.

Why this matters for adaptive and black-box attacks. An attacker that accounts for the defense is exactly what we evaluate: EOT-20/50 in Table 3 and the BPDA variants in Table 12, together with strong black-box attacks (Square/SignHunter in Table 2) and the deterministic fixed-noise model in Section 6. The fact that none of these drive the robust accuracy down to the level of an undefended model is consistent with the mechanism above. The attacker is optimizing over a discrete, sign-quantized input space whose reachable set is small, and the noise lets the network generalize across that set rather than commit to a single discrete input.

We do not attribute robustness to noisy gradients. The noise serves two different roles. During training, it perturbs each input so that the network learns to classify the noisy, sign-quantized versions of it, acting as a data-augmentation and attack-simulation device. Both the training and test inputs are sign-quantized, and the clean accuracy on the test set already shows that the network generalizes from the noisy training neighborhood to the test neighborhood. At test time, the noise samples different points from the reachable discrete set around each test input, and this same generalization covers many of those points. The attacker can still find individual discrete points that flip the label, but these points are not stable: a point that fools one noise draw need not fool another, and the attacker cannot target a single discrete input across queries. Neither of these is gradient obfuscation; the robustness comes from the fact that the reachable discrete input set is small and that the noise lets the network generalize across it.

We will remove any wording that suggests the method is provably robust.

Reviewer_WvHFOpenReview ↗

Score raised to 3

Thanks for addressing the presentation issues, distinguishing the threat models, and limiting the Section 6 analysis to the simplified model. I am happy to increase the score.

Reviewer_MiuDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies input-space constraints for adversarial robustness and argues that the gap between clean and robust accuracy may stem from the limited overlap between the network’s generalization region and the human perceptual space in the vast input space. The authors propose a randomized feature-squeezing input layer: during training, inputs are randomly perturbed and features are learned to be squeezed toward near-binary values, while a simplified form is used at test time. The method is pluggable into different network architectures, uses standard training on clean images only without adversarial training or attack priors, and can simultaneously handle adversarial attacks and One-Pixel Shortcut. Experiments on multiple benchmarks under various black-box/white-box attacks, OPS, and ablations show competitive robust accuracy.

Critical strengths

None

Critical weaknesses

None

Questions for authors
  1. Section 1 suggests that compressing inputs toward near-binary values can reduce the gap between the network’s generalization region and human perceptual space. Please clarify why near-binary compression would “align with perceptual space,” rather than merely reducing input dimensionality, discarding information, or introducing discretization effects. I suggest adding quantitative indicators of feature/semantic preservation before and after compression, or explicitly stating that this motivation should be understood as an intuitive explanation rather than a provable mechanism.
  2. Training uses Reciprocal, Sigmoid, and BPDA-like custom gradients, while testing removes the convolution branch and uses a non-differentiable Sign. Please clarify: how does the deviation between the custom gradients and the true gradients affect the robustness evaluation? Does testing still use normalized inputs and noise of the same scale? How is the approximation error between the training formula and the test formula controlled? Why does removing the convolution branch not cause a significant distribution shift?
  3. In Table 3, accuracy under EOT-20 to EOT-50 continues to decrease in some settings, and DDN-l2 on CIFAR-10 is actually higher under EOT-20 than under the standard attack. Please provide EOT curves (e.g., EOT-1/5/10/20/50/100) with means and variances over multiple random seeds to show whether EOT has sufficiently converged, and explain why EOT is stronger on DDN-l2.
5 comments
AuthorsOpenReview ↗

Response on the Section 1 motivation.

We thank the reviewer and we agree with the second option. We will not claim that near-binary compression aligns with human perceptual space, and we will revise Section 1 accordingly.

Specifically, in the revision we will:

  1. Explicitly state that the discussion of SgS_g and SaS_a in Section 1, together with the illustrative picture in Figure 1, is an intuitive motivation for why constraining the input space may help, not a provable mechanism.
  2. Remove the phrase "align with human perceptual space" (and the related wording in the abstract and introduction), and replace it with the factual description: constraining the input to a lower-dimensional, sign-quantized set.
  3. Make clear that the formal robustness argument of the paper is the reachable-set compression under Sign(x+ϵ)\mathrm{Sign}(x+\epsilon) together with the empirical evaluation, and that the perceptual-space discussion plays no role in the method or in the claims.

We agree that near-binary compression could also act through dimensionality reduction, information discarding, or discretization effects, and we will not attribute the observed robustness to perceptual alignment. We believe framing the motivation as an intuition, and separating it from the robustness argument, addresses the reviewer's concern directly.

AuthorsOpenReview ↗

Response on the training/testing framework.

The conv/reciprocal/Sigmoid branch belongs to detour training: it provides a continuous forward path toward the sign-quantized regime and is discarded once training is done. After training, the sparse loss drives x^→0\hat{x}\to 0, so xˉ\bar{x} is large and the Sigmoid saturates; the training output is already binary and coincides with Sign(x~+ϵ)\mathrm{Sign}(\tilde{x}+\epsilon). Since testing uses the same normalization and the same noise scale σ\sigma as training (CIFAR-10: 0.650.65; ImageNet: 1.41.4), both frameworks feed the classifier binary inputs from the same distribution, with a fresh noise sample per forward pass.

Both paths require BPDA for back-propagation, because their true gradients are unusable: in training, the reciprocal is ill-conditioned and the saturated Sigmoid has vanishing gradient, so we set the reciprocal gradient to −1-1 and the Sigmoid gradient to 11; at test time, Sign is non-differentiable and is replaced by the surrogates in Table 12, with the worst case reported. All reported robustness numbers use true forward passes.

Moreover, we compare the transfer attack in Table 8, which generates examples on the training framework and transfers them to the test framework, against an EOT attack on the test framework: on CIFAR-10, 55.70/51.90/34.30 vs. 38.50/33.90/34.00; on ImageNet, 52.00/47.50/18.70 vs. 46.90/37.90/18.80. Any difference between these numbers is not a distributional effect. The input distribution is the same by construction (same normalization, same noise scale, binary input after saturation); the two paths differ only in how the backward pass interpolates the piecewise-constant discrete function, i.e., in the choice of BPDA surrogate, which is the mechanism discussed in Section K. This is also expected: the EOT attack on the test framework interpolates the actual deployed discrete function with a surrogate for Sign, whereas the transfer attack has to interpolate through the additional conv/reciprocal/Sigmoid path, whose backward behaviour is farther from the true discrete function. The EOT attack on the test framework should therefore be at least as strong as the transfer attack, which is what we observe (with PGD\mathrm{PGD}-l∞l_\infty tied within run-to-run variance).

AuthorsOpenReview ↗

Response on EOT convergence and the DDN- anomaly.

We thank the reviewer. One clarification about Table 3 first: the standard-attack rows are computed on the full 5,000-image RobustBench set for ImageNet (10K test set for CIFAR-10), whereas the EOT rows use the first 1,000 images of that same 5,000-image set (300 images for CIFAR-10, due to time constraints). The two rows are therefore computed on different subsets, and the DDN-l2l_2 comparison between EOT-20 and the standard attack in Table 3 mixes these subsets; a direct comparison at the level of a single number is not meaningful.

We have added EOT curves with EOT-1/5/10/20/50/100 (CIFAR-10: 300 images, three seeds; ImageNet: 1,000 images, single run). The mean ℓ2\ell_2 is computed over samples correctly classified on the clean input, after applying DDN.

Table 1: CIFAR-10 DDN-l2l_2 robust accuracy under EOT (300 images, 3 seeds).

EOT stepsMean ± STDMean ℓ2\ell_2
148.89 ± 3.10%0.9376
551.67 ± 1.86%0.8189
1052.22 ± 1.35%0.7820
2050.56 ± 2.14%0.7590
5051.22 ± 2.59%0.7396
10052.33 ± 4.04%0.7262

Table 2: ImageNet DDN-l2l_2 robust accuracy under EOT (1,000 images, single run).

EOT stepsRobust accMean ℓ2\ell_2
150.20%3.0178
547.80%2.5922
1050.40%2.4737
2049.50%2.3837
5049.60%2.3302
10046.90%2.3183

On both datasets, as EOT increases, robust accuracy does not decrease systematically and the mean ℓ2\ell_2 shrinks by about 23% (CIFAR-10: 0.9376 → 0.7262; ImageNet: 3.0178 → 2.3183). More EOT steps therefore do not produce a stronger DDN attack.

Why EOT does not suit DDN here. DDN records and returns the smallest norm at which the sample has ever been misclassified. In the implementation we use, EOT averages the gradient over several noise draws, while this success test is performed on a single forward pass. On our sign-quantized, stochastic model, Sign(x+ϵ)\mathrm{Sign}(x+\epsilon) hops among neighboring discrete points, so a single draw can land on the wrong side of the boundary at a small norm by chance; DDN records exactly this value. As EOT increases, the averaged gradient gives a more stable descent direction, so DDN reaches such a chance misclassification earlier, at a smaller norm—but these are single-draw events that need not remain misclassified under a fresh sample. The recorded norm thus shrinks while the attack does not become stronger. This is a mismatch between DDN's norm-growth assumption and our piecewise-constant predictor, not a failure of EOT to converge.

This also explains why APGD-l2l_2 is unaffected: it does not record the smallest norm of a lucky single draw, but optimizes the EOT-averaged loss within a fixed budget, which a chance misclassification of one draw does not lower. APGD-l2l_2 remains the stronger ℓ2\ell_2 evaluation and is the one we report in Table 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.