acceptodds
Withdrawn from ICLR 2027

Reject Classes and Anchored Energy: Bypassing the Clean-Robust Trade-off in Adversarial OOD Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural networks are often overconfident on out-of-distribution (OOD) inputs, a vulnerability exacerbated by adversarial perturbations designed to evade detection. We reveal that In-Distribution (ID) adversarial training is indispensable for genuine OOD robustness—without it, even the strongest OOD-centric defenses collapse under principled attacks. However, it induces a severe clean-robust trade-off: ID adversarial training suppresses the maximum softmax probability (MSP) of clean samples, crippling the separation between ID and OOD data. We establish two structural routes to mitigate this trade-off: (1) replacing MSP with the LogSumExp (LSE) energy score and supplying its missing anchor via energy-margin regularization (RCE+EMR); and (2) GRoOD, which unlocks an auxiliary reject class whose probability stays insulated from ID confidence degradation. We then uncover a fundamental bottleneck when attempting to transfer EMR across the two routes: the variance-free vulnerability—EMR fails on GRoOD because its OOD objective supervises only the reject class, leaving the internal logit variance unconstrained and exposing the LSE operator's extreme-value sensitivity to single-dimension adversarial breakthroughs. Introducing OOD label smoothing to architecturally lock this missing variance unleashes EMR's full potential on GRoOD, yielding GRoOD+OOD-LS+EMR that attains the best robustness in reject-dominated regimes while providing both detection signals in a single model. Together, the two structural repairs cover both signal regimes, with RCE+EMR leading where LSE detection dominates.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Robustness, Out-of-Distribution Detection, Energy-Based Detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_Ba5pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose several ``fixes'' to existing adversarial training methods (RCE and GRoOD), aiming to achieve both good robustness against adversarial attacks and good (non-malicious) out-of-distribution detection. They identify that RCE is limited by there being no constraint on the absolute energy scale of the logits -- to correct this problem the authors introduce an energy-margin regularization (EMR) term. For GRoOD+EMR, they identify a variance-free vulnerability that is corrected by the use of label smoothing. The proposed fixes are supported both analytically and experimentally. There are some paper clarity concerns. There are also concerns about the overall approach (with a single model needing to simultaneously achieve good in-distribution classification accuracy, good out-of-distribution detection, and good adversarial robustness) -- the authors do not investigate alternatives to this approach (which include pairing the classifier with an auxiliary detector).

Critical strengths

The paper probes mechanistic sources of suboptimal adversarial robustness/outlier detection for two existing methods -- RCE and GroOD, and proposes sound corrections to these methods.

Experiments support that these fixes improve adversarial robustness.

Critical weaknesses

The authors claim(sec. 2.1) that adversarial training is "the leading defense against adversarial attacks" -- and to support this they cite very old papers (the latest being Papernot et al., 2017). The authors acknowledge that adversarial training "degrades standard accuracy", but despite that they do not, in their work, consider use of auxiliary detectors. Such detectors do not impair the "standard accuracy of the classifier". It would be instructive to see how standard classifier+auxiliary detector fares against their adversarial training methods, with respect to clean accuracy, clean OOD detection performance, and detection of adversarial attacks. There is also no discussion of clean accuracies (the ACC column) in the paper.

There are several hyperparameters of the authors' methods, which appear to be optimistically set (m, alpha) -- the appendix discusses hyperparameters, but this discussion still suggests optimistic tuning.

Questions for authors

Why do the authors focus only on detectors that are based on the model's outputs -- perhaps more informative detection signals can be extracted from internal model signals (with anomaly detection, based on a null (clean ID data) model for these signals) ?

Section 3: "both terms with the uniform target U = [1/K,...,1/K]" -- the first term uses the clean label y_i^in -- so why do the authors say both terms use U ?"

What conclusions would the authors draw from the Acc column in their tables ?

Why are AA results shown in table 2 and table 3? These are not discussed.

Section 5.2: "Empirically, this appears as a small unstable gap under RCE " -- gap between what and what ? This is not explained.

Section 8, Corollary 1: "P_IDp_j^ID" -- unclear what this is...

The gains of GRoOD+ over GRoOD in Table 3 appear to be marginal.

Reviewer_FjV3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies adversarial out of distribution (OOD) detection under the well known clean robust trade off. The authors first demonstrate that in distribution (ID) adversarial training is mandatory to defend against strong adversarial OOD evasion attacks. However, ID adversarial training suppresses maximum softmax probability (MSP) for clean ID samples and degrades ID OOD separation, creating the so called clean robust trade off. Two structural mitigation routes are proposed:

  1. RCE+EMR: Replace MSP with LogSumExp(LSE) energy score and introduce Energy Margin Regularization (EMR) to fix the unanchored energy scale suffered by uniform target RCE.
  2. GRoOD: Augment an auxiliary reject class pK+1, whose aggregate probability mass property insulates it from MSP degradation.
Critical strengths

1.Important problem formulation: The paper highlights the under discussed clean robust trade off for adversarial OOD detection, pointing out that ID adversarial training is non negotiable for real robustness, while it corrupts the widely used MSP detection signal. This reframes existing end to end robust OOD detection research. 2.Novel architectural duality insight (Proposition 1): The theoretical analysis reveals an interesting optimization manifold duality between uniform target training (RCE) and reject class training (GRoOD). The “variance free vulnerability” is a new, well motivated observation for the community. 3.Two structurally motivated fixes: RCE+EMR and GRoOD+OOD LS+EMR are not heuristic tweaks; each is directly derived from the diagnosed optimization deficiencies. The OOD LS mechanism elegantly re introduces variance constraints for reject class models. 4.Decent empirical evidence on standard benchmarks: The authors conduct controlled comparisons against representative baselines (RCE, ACET, ATOM, ALOE, RODEO U, Virtual BG). Ablations validate the benefit of switching to LSE score and reject class signals. Figure 2 gives visual evidence of single dimension logit breakthrough under TAPGD lgt. 5.Good reproducibility intent: The manuscript provides full hyper parameter settings, attack definitions in appendix and promises anonymous code in supplementary materials.

Critical weaknesses
  1. Incomplete robustness evaluation, missing adaptive & black box attacks All main experiments are restricted to white box threat model with TAPGD lgt/PGD ce. Adaptive attacks tailored specifically for EMR loss and OOD LS objective are absent, which is a core requirement for adversarial defense papers at ICLR. Without adaptive attack evaluation, we cannot rule out attack over fitting to the used TAPGD lgt. Moreover, evaluations over alternative norms (L2), black box transfer attacks, varying perturbation radii are not provided. Real world OOD adversaries may not have full white box access to loss components.
  2. Heavy hyper parameter dependence & insufficient justification RCE+EMR uses m=1 while GRoOD+OOD‑LS+EMR uses m=5, without clear ablation explaining why different margins are mandatory. α is manually hand‑tuned per‑dataset via a heuristic “signal‑balance rule”. There is no result demonstrating model performance under a unified fixed hyper‑parameter set across all datasets. Performance degradation under suboptimal hyper‑parameters is not characterized. If exhaustive per‑dataset grid‑search is required, practical deployment burden increases substantially.
  3. Theory assumes global convergence, mismatch with practical deep network training Proposition 1 and Corollary 1 build upon the assumption that training converges exactly to global loss minima. However adversarial training of deep networks only converges towards local minima. The paper provides no empirical validation to verify whether real trained networks lie on the theoretical manifolds (equal LSE surface / flat logit manifold) described in Proposition 1. Therefore the variance energy duality is rigorously true only in ideal loss space but its applicability for practical neural networks remains partially conjectural.
  4. Incomplete component ablation for GRoOD+OOD LS+EMR The final method GRoOD+OOD LS+EMR combines three ingredients: GRoOD architecture, OOD LS label smoothing, EMR regularization. The manuscript reports GRoOD and GRoOD+EMR, but the critical ablation GRoOD + OOD LS (without EMR) is missing in main tables. Without this result, we cannot cleanly separate gains originating purely from OOD LS versus synergistic gain after adding EMR. This ablation is essential to validate the central claim that “OOD LS unlocks EMR’s effectiveness on GRoOD”. Alternative margin loss ablations (e.g., replace hinge EMR with MSE loss) are also missing.
  5. Baseline comparisons are incomplete; missing certified / post hoc robust OOD baselines Several important families of robust OOD detection are omitted from comparison: Certified / provable robust OOD detection approaches; Post‑hoc robust OOD detectors that do not need joint ID/OOD adversarial training; Classic reject‑class baseline SelectiveNet under adversarial training setting; Denoised‑smoothing‑based robust OOD pipelines. Moreover, tables prioritize AUC‑ROC metric while FPR‑95 (crucial for safety‑critical scenarios) is not consistently analyzed and discussed for all competitors.
  6. Limited experimental scale; ambiguous definition of “congested / spacious feature space” Main experiments run only on small‑scale datasets CIFAR‑10, CIFAR‑100, Tiny‑ImageNet‑200. Only cursory ImageNet‑200 results exist in appendix; full ImageNet‑1K evaluations are missing. Many empirical tricks observed on small benchmarks fail to transfer to large‑scale real‑world data. Additionally, the paper introduces qualitative terms “congested feature space” / “spacious feature space” to explain dataset‑dependent performance preference between two methods, without providing any quantifiable metric for feature‑space congestion. This distinction is a post‑hoc interpretation of experimental outcomes. Practitioners cannot predict which route to pick for unseen new datasets.
  7. Missing computation overhead analysis and auxiliary OOD data sensitivity The proposed pipeline requires joint adversarial training over ID plus auxiliary OOD samples, plus extra EMR regularization loss. GRoOD further modifies network output layer with extra reject logit. Training wall‑clock time, GPU memory consumption and inference latency are never reported. Real‑world deployment cost remains unknown. The approach heavily depends on auxiliary OOD training data. The authors do not study performance degradation under limited‑size or low‑quality auxiliary OOD data; ablations varying auxiliary‑OOD data quantity are missing.

Minor Weaknesses • 1.Many critical derivations, attack formalisms and ablation details are pushed into appendix. Key propositions and supporting empirical evidence should be partially condensed to main text for readability. • 2.Only one LSE oriented baseline (Virtual BG) is compared; more energy based robust OOD competitors would strengthen empirical validation. • 3.The work assumes access to auxiliary OOD samples for training, which is not always available in real world application scenarios; discussion on this limitation should be added in main text.

Questions for authors
  1. Could you provide results under adaptive attacks specifically tailored against EMR and OOD LS losses? Can you also add black box transfer attack evaluations?
  2. Please provide the missing ablation GRoOD + OOD LS (without EMR) and report corresponding LSE / reject AUC numbers, to isolate the standalone effect of OOD LS.
  3. Why must you set different margin hyper parameter m for RCE+EMR (m=1) versus GRoOD+OOD LS+EMR (m=5)? Please provide ablation study over margin choices.
  4. Can you propose a computable quantitative metric for “congested / spacious feature space”, so practitioners can pre select between RCE+EMR and GRoOD+OOD LS+EMR for unseen datasets?
  5. Can you provide empirical sanity check to what degree real trained GRoOD / RCE models satisfy the global minima manifold assumptions in Proposition 1?
  6. Please report standard deviations over multiple random seeds for your key tables to demonstrate statistical significance of performance gains.
  7. How does your method behave when auxiliary OOD training data is scarce or noisy? Could you add ablations varying auxiliary OOD data size?
Reviewer_xYv5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how ID adversarial training weakens MSP-based OOD detection. It proposes energy-margin regularization (RCE+EMR) and a reject-class model with OOD label smoothing (GRoOD+OOD-LS+EMR). Results are promising, but the central mechanistic explanation and the final method’s incremental contribution need stronger support.

Critical strengths
  • The controlled comparisons expose a useful interaction between adversarial training and detection scores.
  • Table 2 separates improvements from switching to LSE and adding EMR: on CIFAR-100, robust AUROC increases from 52.71 to 67.13 to 73.57.
  • The uniform-target loss decomposition and reject-probability energy-gap identity provide useful insights.
Critical weaknesses
  1. The central gradient argument is incorrect as stated. Reject-class cross-entropy supervises each ID logit with gradient (p_j), which vanishes only as rejection becomes confident. Active EMR has OOD-logit gradient proportional to (\operatorname{softmax}(z_{1})_j): larger logits receive stronger downward updates. This contradicts Corollary 1’s claim that EMR leaves relative logits unchanged. The observed failure may be real, but Proposition 1 and Figure 2 do not establish its proposed cause or connect logit variance to input-space robustness.
  2. The key ablation is missing from the quantitative results. Table 3 should include GRoOD+OOD-LS without EMR. Without the full four-way comparison, the paper cannot establish that smoothing specifically enables EMR rather than providing the improvement independently.
  3. The final model’s reject-score gains are small. Improvements over GRoOD are only 0.37, 0.05, and 0.20 AUROC points. Multiple-seed uncertainty and FPR-95 are needed to assess their significance. Improving the additional LSE signal is useful, but should be distinguished from improving the strongest detector.
  4. Additional issue: EMR’s novelty needs clarification. Liu et al. (2020), Section 3.2 and footnote 2, already discuss energy-margin training, including a single-margin ID–OOD energy hinge. The contribution therefore needs to be distinguished through its batch aggregation, adversarial setting, or interaction with target distributions, with a matched comparison to prior energy-margin objectives.
Questions for authors
  1. How did you reconcile Corollary 1 with EMR’s nonuniform logit gradients?
  2. Does the full smoothing-by-EMR ablation support the claimed interaction?
  3. How does EMR compare with prior energy-margin objectives under identical adversarial training?
Reviewer_erZ9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies adversarial out-of-distribution detection and argues that ID adversarial training can weaken clean OOD detection by reducing maximum softmax confidence. It explores two ways to address this: adding energy-margin regularization to a uniform-target baseline, and training an additional reject class through GRoOD. The paper then proposes OOD label smoothing to improve the combination of GRoOD and energy regularization.

The main experiments cover CIFAR-10, CIFAR-100, and Tiny-ImageNet-200. They show substantial improvements over the MSP-based RCE baseline, with different detection scores working best on different datasets. The paper also offers a mechanistic explanation based on the distinction between controlling relative logits and controlling energy separation.

Critical strengths
  1. The paper separates the effect of the detection score from the effect of additional training. Table 2 includes RCE with LSE detection as well as RCE with energy-margin regularization. This is an important control. On CIFAR-100, switching the score increases robust AUC from 52.71 to 67.13, and adding EMR increases it further to 73.57. This makes it possible to see which part of the gain comes from the score and which part comes from the proposed regularizer.

  2. The proposed changes are simple and give meaningful gains in the reported settings. GRoOD improves both clean and attacked OOD detection over the MSP-based RCE baseline across the three main datasets, while maintaining similar ID classification performance. I also appreciate that the paper acknowledges that its two approaches are complementary rather than claiming that one method wins everywhere.

Critical weaknesses
  1. The central gradient-based explanation is not correct as written. The introduction says that the reject-class objective gives the ID logits no gradient signal. With standard softmax cross-entropy, however, every ID logit receives a gradient on an OOD example. These gradients become small when rejection is already confident, but that is different from receiving no supervision during training. The more defensible observation is that the objective does not require the ID logits to become equal.

    More importantly, Corollary 1 says that EMR leaves the relative logit configuration untouched. When the energy-margin hinge is active, its gradient on the OOD logits is larger for larger ID logits. It can therefore change their relative configuration and does not generally act as a uniform translation. The small gradient of the reject-class cross-entropy does not establish that the separate EMR term also supplies negligible resistance.

    Proposition 1 provides a useful characterization of the isolated OOD objectives, but it does not establish the claimed failure of EMR under adversarial training. The observed asymmetry may be real; the proposed explanation needs a more careful derivation and direct empirical support.

  2. The evidence is not yet strong enough for the broad robustness and “bypassing” claims. The main evaluation uses a multi-targeted logit attack, while the paper relies on Appendix B.6 to establish robustness against direct energy-score optimization. That appendix, along with the attack details, hyperparameter-selection procedure, and several ablations, is absent from the supplied PDF. I therefore cannot verify these checks from the material available.

    The final combination also offers only small improvements over plain GRoOD on its reject score: 0.37 AUC points on CIFAR-10, 0.05 on CIFAR-100, and 0.20 on Tiny-ImageNet-200. Table 3 provides no uncertainty estimates, and clean reject-score AUC decreases on CIFAR-100. These results support a useful refinement, but not yet a clear claim that the trade-off has been bypassed. Likewise, the failure of the tested methods without ID adversarial training supports its importance in this setting, rather than establishing that it is universally indispensable.

Questions for authors
  1. Could you revise the gradient argument in Section 7 and Corollary 1? In particular, how does the explanation account for the nonzero ID-logit gradients from reject-class cross-entropy and the unequal gradients from active EMR? Direct measurements of the two loss terms’ gradients would help test the proposed mechanism.

  2. Could you provide the referenced appendices and clarify the strongest attack evaluation for each detection score? For energy detection, please include direct LSE optimization; for reject detection, please include direct optimization of the reject probability or its equivalent energy gap. Does combining the strongest examples from these attacks change the rankings?

  3. How is the dataset-specific reject weight selected without using the final OOD test results? Please specify the validation data and whether choosing between the energy and reject scores is also part of that selection procedure.

  4. Could you report repeated-run variability and a complete ablation of plain GRoOD, GRoOD with EMR, GRoOD with OOD label smoothing, and the full combination? This is particularly important for determining whether the small reject-score gains in Table 3 are consistent.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.