acceptodds
Under review as a conference paper at ICLR 2027

Same Classifier, Different OOD Decisions: The Hidden-Basis Dependence of Activation Clipping

Abstract

Post-hoc OOD detectors that shape the penultimate representation coordinate by coordinate—REACT, ASH, DICE—are popular because they are nearly free. We show that their verdicts are not a function of the classiffer function alone: they depend on the network’s hidden basis. We ask which reparameterization symmetries such a detector can and should respect, and answer at three tiers. Under the architecture’s own symmetry (positive channel rescaling), no global threshold reproduces REACT’s clipping, however re-calibrated; demanding basis-agnosticism rules out every continuous coordinate-wise map except uniform rescaling and, among continuous pointwise shapers, forces the radial form T(h) = α(∥h∥) h; within radial norm-based constructions, afffne covariance leads to the Mahalanobis geometry of Maha-Radial. Empirically, across random bases of the same classiffer function, REACT’s AUROC swings by up to 15.8 points and an adversarially optimized basis inverts it entirely (0.97 → 0.003), while the exactly invariant, training-free Radial-ReAct and Maha-Radial remove the dependence exactly—Radial-ReAct essentially matching raw energy at ImageNet scale, Maha-Radial exposing a calibration-rank trade-off that covariance shrinkage mitigates. At scale the artifact dominates the benefft: on an untouched ImageNet ResNet-50 over six standard OOD suites, REACT beats raw energy at the trained basis on all six sets, yet on every one its worst of 20 random bases falls below the energy score—reparameterization alone erases the entire gain—and the swing explodes as the penultimate dimension shrinks (23 points at d ≤ 16), exactly where compact deployed heads sit. Code reproducing every result: https: //anonymous.4open.science/r/react_invariance-A33F/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Post-hoc out-of-distribution, ReAct

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_LeGAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how post-hoc OOD detectors depend on the coordinate representation of the penultimate feature. It considers three symmetry levels: architecture-native positive channel rescaling/permutation, arbitrary orthogonal changes of basis before the classifier head, and a stronger affine-covariant tier. The paper analyzes ReAct, ASH, DICE and geometric baselines, proposes PC-ReAct, Radial-ReAct and Maha-Radial, and evaluates the resulting sensitivity on several small-scale tasks, pretrained ResNets, ImageNet, and a ViT backbone. The main empirical message is that coordinate-wise shaping can change substantially across function-preserving reparameterizations, whereas geometric constructions can be exactly invariant by design. I find the architecture-native rescaling analysis and the careful quantitative study useful, but I have a central concern about the motivation and novelty of the orthogonal-basis results.

Critical strengths
  1. The paper is careful and systematic. It distinguishes architecture-native symmetries from stronger basis-agnostic design requirements, checks logit preservation numerically, and evaluates the effect over many models, tasks, and reparameterizations. The positive channel-rescaling result is particularly relevant because it concerns a genuine symmetry of common ReLU-style parameterizations rather than an externally inserted rotation.
  2. The paper does more than diagnose sensitivity: it gives a clean hierarchy of equivariant alternatives (PC-ReAct, Radial-ReAct, Maha-Radial) and quantifies the robustness/accuracy trade-off at ImageNet scale. This makes the study practically informative even where the qualitative transformation behavior is mathematically expected.
Critical weaknesses
  1. My main concern is that much of the qualitative invariant/non-invariant pattern under arbitrary orthogonal changes of basis is already implied by the detector definitions. Coordinate-wise operators such as ReAct and ASH are generically non-equivariant to coordinate mixing; DICE ranks coordinate-wise products before masking; in contrast, Euclidean distance, a re-fitted PCA residual norm, and a properly transformed Mahalanobis quadratic form are orthogonally invariant by construction. Therefore, the qualitative distinction in Table 5 is largely predictable a priori. The nontrivial empirical contribution is the magnitude of the effect, its worst-case severity, and whether it occurs under practically relevant reparameterizations. The paper would be stronger if it more sharply separated definition-level consequences from the genuinely new theoretical characterizations and empirical findings, rather than presenting the qualitative rotation sensitivity itself as a central discovery.
MethodCore dependenceOrthogonal QQPositive diagonal DDDefinition-level reason
ReActGlobal coordinate clippingGenerally changesGenerally changes for nonuniform DDR(Qh)≠QR(h)R(Qh) \neq Q R(h) in general; one global threshold cannot match heterogeneous djd_j.
PC-ReActPer-channel clippingGenerally changesExact after per-channel recalibrationcj′=djcjc'_j = d_j c_j, so clipping commutes with DD; QQ mixes channels.
ASH-BCoordinate ranking / pruningGenerally changesGenerally changes for nonuniform DDRanking, survivors, and reassignment statistics change.
DICECoordinate contribution ranking + head maskGenerally changesExact for positive diagonal DD(wcj/dj)(dj E[hj])=wcjE[hj](w_{cj}/d_j)(d_j\,\mathbb{E}[h_j]) = w_{cj}\mathbb{E}[h_j]; QQ mixes contributions before ranking.
kNNEuclidean distanceInvariantGenerally changes for nonuniform DDOrthogonal maps preserve Euclidean distances.
ViMLogits + re-fit PCA residualInvariantNot generally invariantUnder QQ the residual subspace co-rotates; general DD changes Euclidean PCA geometry.
MahalanobisMean/covariance quadratic formInvariantExact if covariance is full-rank and re-estimated covariantly; otherwise not guaranteedFull-rank Mahalanobis geometry is invariant to invertible linear coordinates; rank deficiency/truncation is the caveat.
Radial-ReActEuclidean normInvariantGenerally changes for nonuniform DD∥Qh∥2=∥h∥2\lVert Qh\rVert_2 = \lVert h\rVert_2; heterogeneous DD changes the norm.
Maha-RadialMahalanobis radiusInvariantAffine-covariant only in the full-rank unshrunk caseFull-rank μ,Σ\mu,\Sigma transform covariantly; rank deficiency/shrinkage weakens nonorthogonal invariance.
  1. A second concern is the normative step from classifier-function equivalence to treating arbitrary basis changes as nuisance variation for a representation-dependent detector. The paper explicitly acknowledges that arbitrary basis invariance is a design desideratum, not an architecture fact (Fig. 5), and later calls the symmetry choice a value judgment. End-to-end training jointly shapes the representation and the linear head, so the trained coordinates can contain axis-dependent statistics (e.g., channel-wise sparsity, marginal tails, or coordinate-specific contribution patterns) that ReAct/ASH/DICE intentionally exploit. Applying an arbitrary Q mixes these statistics even though W is compensated so that the raw classifier function is unchanged. The paper's own ImageNet result—that the trained basis is the best of the sampled Haar bases on all six OOD sets, while Radial-ReAct largely falls back to raw-energy performance—is at least consistent with this alternative interpretation. Thus, a changed feature-based OOD score is not by itself evidence of an undesirable parameterization artifact. A stronger justification is needed for treating the trained coordinate system as nuisance variation rather than potentially useful learned structure.
Questions for authors
  1. The paper correctly states that arbitrary basis invariance is a design desideratum rather than an architecture fact. What is the strongest justification for imposing this desideratum on representation-dependent OOD detectors? Can the authors provide evidence that the trained coordinate statistics exploited by ReAct/ASH/DICE are nuisance variation rather than useful learned structure? A direct analysis linking trained-basis statistics to OOD gains would materially affect my assessment.
  2. Can the authors separate more explicitly three components of the contribution: (i) invariance/non-invariance that follows directly from the detector definitions, (ii) the nontrivial characterization results (e.g., restrictions imposed by orthogonal equivariance), and (iii) empirical findings about effect size and deployment relevance? In particular, which main conclusions remain if the evaluation is restricted to architecture-native or otherwise deployment-plausible reparameterizations rather than arbitrary orthogonal changes of basis?

If the authors’ response can adequately address the two concerns above, I would be willing to reconsider my current assessment and potentially raise the score accordingly.

4 comments
Reviewer_LeGAOpenReview ↗

Response to Rebuttal: Remaining Concerns

Thank you for the detailed rebuttal and additional experiments. We appreciate the authors' efforts to address our concerns.

While the rebuttal provides useful clarifications and additional evidence, we remain unconvinced about the novelty of the theoretical contributions beyond the definition-level observations. Moreover, although the architecture-native scaling experiments validate the reported sensitivity, the observed performance changes under the tested mild rescalings are modest, leaving their broader practical significance unclear.

Therefore, we maintain our original score.

AuthorsOpenReview ↗

Official Comment 1 — Beyond definition-level invariance

Thank you for the follow-up. We appreciate your distinction between predictable invariance properties and nontrivial contributions, and agree that the qualitative taxonomy alone does not establish theoretical novelty.

The distinction we would like to clarify is between verifying the transformation behavior of a particular detector and characterizing the class of shaping rules that can satisfy a prescribed symmetry.

  • Impossibility of recalibration (Propositions 1 and 4). Definition-level non-equivariance does not establish whether recalibration can recover the original shaping. Proposition 1 rules out every replacement global threshold as an exact operator-level remedy under nonuniform positive rescaling. Proposition 4 further establishes that, for a fixed input satisfying its active-clipping assumptions, almost every orthogonal transformation admits no scalar threshold recovering the original clipped representation, even if that threshold is optimized rather than percentile-calibrated.

  • Characterization of admissible shapers (Proposition 3 and Theorem 2). Orthogonal equivariance restricts fixed continuous coordinate-wise maps to uniform linear scaling, and fixed continuous per-sample maps, without auxiliary data-dependent structures, to radial forms. Consequently, within these classes, nontrivial coordinate-wise clipping or sparsification cannot coexist with full orthogonal equivariance.

We do not claim novelty for the underlying group-equivariance principles themselves. Our contribution is their formulation as explicit restrictions on OOD activation shaping, together with the impossibility of scalar recalibration, the corresponding constructive alternatives, and their measured accuracy–invariance trade-offs.

Importantly, even without arbitrary orthogonal transformations, Propositions 1 and 2 establish an operator-level obstruction and an exactly equivariant alternative under the architecture's native symmetry, while retaining the learned coordinate axes.

We will restructure the abstract, contribution list, Table 5, and symmetry figure to explicitly distinguish (i) definition-level consequences, (ii) impossibility and characterization results, and (iii) empirical findings about effect size and practical relevance. We agree that the first component should not be presented as an independent discovery.

AuthorsOpenReview ↗

Official Comment 2 — Native rescaling: aggregate robustness versus decision consistency

Thank you for highlighting the practical significance of native rescaling. We agree that the modest aggregate effects under mild transformations should not be overstated. We extended the ViT experiment to ResNet-50 and investigated whether aggregate stability translates into consistency of individual decisions.

1. Architecture-native parameter rewrite on ResNet-50.

We now implement positive channel rescaling directly in the existing ResNet-50 parameters, without inserting any layer. Scaling is propagated through the final residual stage via the BatchNorm affine parameters, compensated bottleneck input weights, and inverse classifier-head scaling. Positive homogeneity of ReLU preserves the classifier function in exact arithmetic. A 512-image forward audit confirms numerical agreement with the cached-feature computation and zero top-1 disagreements.

Using the validated cached-feature protocol, we evaluated 20 random scalings and one ID-calibrated scaling at kappa=1.25, with detector fitting and operating-threshold calibration on disjoint ID splits.

Table L5. ReAct AUROC changes (pp relative to identity).

OOD suiteRandom meanRandom minRandom maxID-calibrated
iNaturalist-0.11-0.45+0.24-0.23
SSB-hard-0.09-0.64+0.32-1.15
NINCO-0.01-0.29+0.32-0.03
OpenImages-O-0.02-0.24+0.18+0.35
Textures0.00-0.26+0.28+1.68
Places365-0.02-0.27+0.55-0.52
Six-suite mean-0.04-0.20+0.09+0.02

PC-ReAct and raw energy remain numerically unchanged at the reported AUROC precision. These results confirm that mild native rescaling is largely benign in aggregate AUROC terms on both evaluated backbones.

2. Aggregate stability does not imply decision consistency.

At the separately recalibrated nominal 5% ID-rejection operating point, across 20 random scalings, approximately 48 ID images (0.59% ± 0.08 pp) and 920 OOD images (1.79% ± 0.28 pp) change their decisions per draw, out of 8,000 ID and 51,518 OOD images. The aggregate ID false-rejection rate remains nearly unchanged (5.9% at identity versus 5.8% averaged across random scalings), yet the identities of rejected samples change.

Importantly, the OOD disagreements are directionally asymmetric: 86% of flips correspond to initially detected OOD samples becoming missed, whereas 14% correspond to initially missed samples becoming detected. Direct computation across the 20 draws gives a net OOD miss-rate increase of approximately 1.3 pp, despite a mean AUROC decrease of only 0.04 pp.

Table L6. Decision disagreement with identity for two selected scaling cases.

PopulationRandomID-calibrated
ID (8,000)61 (0.76%)77 (0.96%)
iNaturalist444 (4.44%)430 (4.30%)
SSB-hard155 (1.55%)316 (3.16%)
NINCO133 (2.26%)115 (1.96%)
OpenImages-O263 (2.63%)260 (2.60%)
Textures144 (2.55%)190 (3.37%)
Places365205 (2.05%)458 (4.58%)
All OOD (51,518)1,344 (2.61%)1,769 (3.43%)

Our empirical margin analysis further shows that affected samples concentrate near the original operating threshold. The identity-basis margin thus provides an empirical indicator of susceptibility, rather than a certified predictor of future decision changes.

These findings distinguish aggregate ranking robustness from operating-point and individual-decision robustness. We establish the magnitude of these effects under controlled function-preserving transformations, not their prevalence in deployment. The corresponding accuracy–consistency trade-off and additional bounded-rescaling experiments are discussed in our next comment.

AuthorsOpenReview ↗

Official Comment 3 — Practical significance and the accuracy–consistency trade-off

Following the ResNet-50 results above, we further quantify the accuracy cost of enforcing native equivariance and examine sensitivity across different rescaling strengths.

1. What is gained by enforcing native consistency?

The native-equivariant alternative retains useful detection gains, although its accuracy cost depends on the backbone.

Table L7. Six-suite mean AUROC and the cost of native equivariance.

SettingEnergy (%)ReAct (%)PC-ReAct (%)Cost (pp)
ViT-B/16 (fixed random D)68.9377.1376.520.61
ResNet-50 (identity)78.982.881.01.8

PC-ReAct retains approximately 93% and 54% of ReAct's respective gains over energy, while providing exact native equivariance. These results complement the theoretical construction in Proposition 2: consistency across positive channel rescalings does not require discarding learned coordinate axes or reverting to raw energy.

2. Sensitivity across rescaling strengths.

We additionally examined bounded positive rescalings at different condition numbers, refitting the detectors under each transformation.

Table L8. Bounded-rescaling dose-response on ResNet-50 (mean AUROC change, pp).

kappaReActPC-ReAct
1.25 (Table L5)-0.040.00
1.5-0.110.00
2.0-0.120.00
3.0-0.260.00
κΔAUC (pp)ID FRR (%)OOD flips (%)total (pp)capability (pp)op-shift (pp)FPR95 (%)
identity05.88000059.63
1.25−0.035.46 ± 0.101.79 ± 0.28−1.28−0.27−1.0160.91
1.5−0.114.88 ± 0.143.74 ± 0.67−3.36−0.61−2.7462.98
2.0−0.124.12 ± 0.236.92 ± 1.26−6.68−1.00−5.6866.31
3.0−0.263.49 ± 0.249.89 ± 1.83−9.72−1.31−8.4169.35

These experiments show increasing average degradation for ReAct across the tested condition numbers, although its magnitude remains modest. PC-ReAct remains numerically unchanged at the reported precision, consistent with its exact theoretical guarantee.

3. Practical implications and revised interpretation.

We therefore do not claim that native equivariance is necessary to prevent a large average accuracy collapse under mild rescaling. Rather, our experiments establish three distinct findings:

  • Aggregate ranking performance: ReAct is largely robust to the tested mild native rescalings.
  • Operating-point and decision consistency: Despite the small AUROC changes, individual OOD decisions change, with a net increase of approximately 1.3 pp in the OOD miss rate on ResNet-50.
  • Constructive guarantee: PC-ReAct removes native parameterization dependence by design while retaining useful detection gains, at a measurable, backbone-dependent accuracy cost.

When aggregate detection accuracy is the sole objective, ReAct may remain preferable. When consistency of individual decisions across equivalent native parameterizations is required, PC-ReAct offers an explicit alternative.

We agree that the practical importance of this guarantee depends on the application. Our experiments quantify sensitivity under controlled function-preserving transformations rather than establishing their prevalence in real deployments.

Together with the characterization results discussed in our first follow-up comment, these findings separate what follows from detector definitions, what is guaranteed by theory, and what must be measured empirically. We will revise the manuscript accordingly, avoiding blanket claims about the undesirability of learned coordinate dependence.

The full per-draw results, dose-response experiments, and individual decision lists are provided in the anonymous supplementary repository.

Reviewer_U8tmOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies issues in post-hoc out-of-distribution (OOD) detection: several activation-shaping methods, including REACT, ASH, and DICE, can produce different OOD decisions for functionally identical classifiers simply because the penultimate representation is expressed in a different hidden basis. The authors formalize this through function-preserving reparameterizations and introduce a hierarchy of invariance/equivariance requirements. They show theoretically that a global REACT threshold is not invariant even to positive channel rescaling, that continuous coordinate-wise shaping cannot be invariant to arbitrary orthogonal basis changes except for uniform scaling, and that orthogonally equivariant per-sample transformations must have a radial form.

Based on these observations, the paper develops PC-REACT, Radial-ReAct, and Maha-Radial, corresponding to progressively stronger symmetry requirements. The empirical analysis is extensive and spans small models, pretrained ResNet backbones, an ImageNet ResNet-50, and a ViT-B/16. The central finding is almost convincing: coordinate-wise activation shaping can exhibit substantial variation across function-preserving parameterizations despite unchanged classifier logits, whereas appropriately geometric variants remove the corresponding dependence. Particularly striking is the ImageNet experiment where REACT improves over energy in the trained basis but loses this advantage under alternative parameterizations of the same classifier.

Overall, I find the paper interesting. My main reservation is conceptual rather than about the correctness of the demonstrated phenomenon: the paper largely treats dependence on hidden coordinates as undesirable, whereas learned coordinate structure may itself encode useful properties of the training distribution. The experiments convincingly establish representation sensitivity, but establish less clearly when this sensitivity should be regarded as an artifact rather than useful inductive structure.

Critical strengths
  1. The paper identifies a real and underappreciated ambiguity in feature-based OOD detection. The strongest aspect of the work is that it isolates parameterization sensitivity while holding the classifier function fixed. This makes the experimental comparison unusually clean: changes in OOD behavior cannot simply be attributed to changes in classifier accuracy or logits. The ImageNet experiments, per-image verdict flips, and adversarial basis results make the phenomenon difficult to dismiss as a small-model or numerical artifact.
  2. The theoretical and empirical parts reinforce each other unusually well. The results do more than observe instability. They characterize which symmetry requirements different classes of transformations can satisfy and construct PC-REACT, Radial-ReAct, and Maha-Radial as concrete examples. The extensive controls over model size, representation dimension, calibration-set size, normalization choices, pretrained backbones, and alternative OOD methods strengthen the main conclusion considerably.
Critical weaknesses
  1. The paper does not completely justify why invariance to arbitrary hidden-basis transformations should be the desired criterion for OOD detection. The argument is strongest for architecture-native parameter symmetries such as positive channel rescaling: if two parameter settings represent the same network function within the same architecture, differing safety decisions are genuinely concerning. The case for arbitrary orthogonal or affine basis invariance is more normative. A trained network may organize information into directions or coordinates that encode regularities of the training distribution even when those coordinates are not identifiable from the final classifier function alone. Discarding this structure can therefore improve invariance while simultaneously discarding potentially useful OOD information, as the behavior of Radial-ReAct itself illustrates. I would like the paper to distinguish more sharply between invariance that follows from genuine model symmetries and invariance deliberately imposed as a design prior. This issue connects to work showing that useful object information can persist in latent representations even under severe transformations of the input. Malik, Crowder, and Mingolla (2023a) specifically study robust latent object representations under extreme image transformations, while Malik, Crowder, and Mingolla (2023b) compare how such transformations affect recognition in humans and machines. These results suggest that representation geometry can contain useful information that is not captured simply by the final classifier decision. That does not contradict the present paper, but it makes the distinction between “parameterization artifact” and “meaningful representational structure” important.
  2. The empirical evaluation establishes sensitivity much more strongly than it establishes that naturally occurring representation changes cause practically harmful instability. Random Haar rotations and especially adversarially optimized bases are effective existence tests, but they are not representative of how independently trained networks or fine-tuned checkpoints move through representation space. The authors acknowledge that full fine-tuning is untested and that the adversarial search is a white-box existence result. This leaves an important gap between “a function-preserving basis exists that changes the decision” and “realistic model evolution produces this failure.” An experiment aligning representations from independently trained or fine-tuned models, followed by analysis of the corresponding OOD decisions, would substantially strengthen the practical interpretation.

Relevant references

Malik, G., Crowder, D., & Mingolla, E. (2023a). Extreme image transformations facilitate robust latent object representations. arXiv preprint arXiv:2310.07725.

Malik, G., Crowder, D., & Mingolla, E. (2023b). Extreme image transformations affect humans and machines differently. Biological Cybernetics, 117(4), 331–343.

Questions for authors
  1. Clarification that could affect my assessment: Do the authors view invariance under the architecture's native symmetry group and invariance under arbitrary orthogonal/affine reparameterizations as equally necessary properties, or is the latter intended only as an optional design principle? I think the current discussion occasionally blurs these two claims. A sharper separation would change how strongly I interpret the practical conclusion.
  2. Clarification that could affect my assessment: Can the authors quantify how the random/adversarial basis transformations studied here compare with representation changes observed between independently trained seeds or before/after realistic fine-tuning? If naturally occurring representation changes occupy a much smaller subset of the investigated transformations, the deployment implications would be more limited, although the theoretical result would remain interesting.
  3. Clarification that could affect my assessment: Is there a way to separate coordinate dependence that is arbitrary under reparameterization from coordinate structure that reliably reflects properties of the ID distribution? Radial-ReAct's loss of REACT's ImageNet gain seems particularly informative here: it suggests that enforcing basis invariance removes both undesirable dependence and potentially useful signal. I would be interested in the authors' interpretation of this tradeoff.

A useful future direction, rather than a requirement for acceptance, would be to investigate detectors that exploit representation geometry while being invariant only to symmetries that are demonstrably non-identifiable or naturally encountered during training.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for highlighting the value of fixing the classifier and suggesting an analysis of real-model representations. We have now connected independently trained and fully finetuned representations to their OOD scores and decisions.

Q1/W1: Which invariances are intended as requirements, and which are design choices?

Architecture-native consistency removes dependence on channel units and ordering while retaining learned axes. Arbitrary orthogonal and affine invariance are optional design requirements for representation-dependent detectors, not consequences of classifier-function equivalence. Orthogonal consistency specifies orientation-independent geometry; the Mahalanobis construction requires full-rank, covariantly estimated covariance for its exact affine guarantee.

Our contribution characterizes admissible shaping rules and constructs corresponding solutions. For a positive clipping threshold, Proposition 1 rules out scalar recalibration as a general remedy for nonuniform positive rescaling; Proposition 2 provides the per-channel solution. Within the fixed continuous operator classes considered, Proposition 3 restricts orthogonally equivariant coordinate-wise maps to uniform scaling, and Theorem 2 characterizes sample-wise equivariant maps as radial. This explains the design choices, beyond observing rotation sensitivity.

The native positive-scaling/permutation and orthogonal groups are not nested. We will make their distinct roles explicit in the abstract, contributions, and symmetry figure, replacing the blanket "artifact dominates benefit" interpretation with measured sensitivity and accuracy trade-offs.

Q3: Can useful learned structure be retained while removing arbitrary coordinate dependence?

Yes, for the native rescaling requirement there is a concrete construction and measurable retained benefit. Positive scaling with head compensation preserves channel identity, within-channel ordering, and the pre-shaping products W_cj h_j. PC-ReAct recalibrates each channel, making its shaped logits invariant to these changes while continuing to exploit channel-specific activations. The paper's ImageNet ResNet-50 results illustrate the choice:

DetectorSix-suite mean AUROC (%)Positive-scaling consistency (refit)Orthogonal consistency (refit)
Energy78.8YesYes
ReAct82.8No, in generalNo, in general
PC-ReAct81.5YesNo, in general
Radial-ReAct78.9No, in generalYes

These are rounded trained-basis means. PC-ReAct retains approximately 2.7 pp over Energy with native equivariance, at a measured accuracy cost relative to ReAct. Radial-ReAct's near-Energy performance shows that this particular orientation-independent construction does not retain ReAct's gain. PC-ReAct provides a concrete intermediate choice that preserves channel-specific processing.

We additionally validated native scaling through actual parameter rewrites of the ViT's final LayerNorm and classifier. For two preselected transformations with condition-number bound 1.25, all 63,518 images were re-forwarded and detectors refitted. ReAct's six-suite mean AUROC decreased by 0.03 pp for the random scaling and 0.52 pp for ID-calibrated range balancing; Energy and PC-ReAct remained numerically unchanged, with zero raw-classifier top-1 disagreements. These are individual-transformation results. They demonstrate sensitivity to channel units without mixing the trained axes, at the measured scale.

The new alignment experiment makes the design trade-off concrete: PC-ReAct has smaller median alignment-induced changes than ReAct, while its exact guarantee remains specific to native scaling/permutation. Both methods have negative median gains over Energy under the new CIFAR-10/SVHN protocol. Alongside their positive ImageNet gains, this result shows why detection benefit and coordinate stability should be assessed jointly, providing a practical basis for detector selection. A matched-statistics intervention on high-tail channels leaves ReAct's gain unchanged on ViT (|change| <= 0.16 pp) — tail statistics are not supported as the source of the gain.

These are comparisons of specified detectors and interventions, not a causal partition of ReAct's gain into useful signal and artifact. They identify consistency guarantees compatible with channel-wise processing and quantify the associated performance trade-offs. We will discuss the cited studies of extreme input transformations by Malik et al. (2023a) and Malik et al. (2023b), distinguishing their input-robustness setting from hidden-coordinate consistency.

Together, the added alignment experiment, native parameter rewrites, and measured trade-offs strengthen the paper's practical contribution: they connect the theoretical operator characterizations to real trained representations and concrete detector choices. We welcome any specific remaining concern and would be happy to provide further clarification or targeted analysis.

AuthorsOpenReview ↗

Official Comment by Authors

Q2/W2: How do model-derived coordinate changes compare with random transformations?

On CIFAR-10 versus SVHN, we evaluated 90 directed pairs among 10 independently trained final models and 21 later-checkpoint-to-initial-checkpoint pairs from three full-network finetuning trajectories. The training units are the 10 models and three trajectories. We split 10,000 ID images into disjoint sets for alignment (2,000), detector fitting (1,000), operational-threshold calibration (1,000), and evaluation (6,000). All 26,032 SVHN test images are evaluation-only. An uncentered orthogonal Procrustes map Q is fitted using ID-align alone. For row-stacked target features, we apply H'=HQ and W'=WQ, leaving the bias unchanged. Thus the target classifier is preserved; maximum recorded raw-logit error is at most 2e-14 in float64, with zero top-1 disagreements. Shaping uses the fixed 90th percentile. Every detector is refitted in each representation, and its operating threshold is recalibrated to nominal 5% ID rejection on the separate threshold set.

The geometry distinguishes independent training from finetuning.

Model pairsRaw displacementAligned residualQ-induced target displacement
Independent models1.1020.4111.097
Finetuning checkpoints0.3400.3110.147

Entries are medians over directed pairs on ID-eval. Raw displacement and aligned residual are normalized by the reference feature norm; Q-induced displacement is normalized by the target feature norm. Alignment reduces the independent-model discrepancy substantially, while much of the finetuning discrepancy remains.

Model-derived alignments change detection even after recalibration. For each pair, we also evaluate 20 random orthogonal Cayley transformations matched to its Q-induced displacement on ID-align (within 1%), plus full-Haar controls without displacement matching.

Model pairs / detectorQ: abs. AUROC change (pp)Matched randomFull HaarOOD decision flips (%)
Independent / ReAct1.211.8051.8163.8
Independent / PC-ReAct0.181.4071.5071.3
Finetuning / ReAct0.2320.2051.9772.6
Finetuning / PC-ReAct0.1650.0660.5110.5

AUROC changes are in pp; summaries use per-target aggregation followed by medians across targets. Flips count decision changes in either direction between the target's original and aligned representations, each with its recalibrated threshold. Energy and Radial-ReAct remain unchanged to numerical precision under the orthogonal intervention. The natural direction's |change| sits at the 57.5th percentile of its matched controls for ReAct (60th for finetuning) — indistinguishable from equal-displacement random directions. For ReAct, the pooled mean signed AUROC changes (aligned minus original) are -0.804 pp for independent-model pairs and +0.186 pp for finetuning pairs. Image-level paired 95% bootstrap intervals are [-0.849,-0.760] and [0.133,0.235], respectively (1,000 resamples, conditional on these checkpoints and alignments). These means are distinct from the median absolute changes above. Repeating the ID splits gives signed median changes of -0.71/-0.84/-0.85 pp for independent-model pairs and +0.15/+0.13/+0.15 pp for finetuning pairs. A full replication on 10 freshly retrained checkpoints (a separate training family) reproduces the medians (ReAct -0.80 pp; geometry 1.11/0.44/1.09).

The main addition is a direct test using coordinates inferred from actual trained models, with no OOD information used to choose the alignment. ReAct's response is of comparable magnitude to the displacement-matched controls in these settings. The finetuning result also shows why full-Haar rotations should not be treated as representative of every model update: their effect is substantially larger here. PC-ReAct exhibits smaller alignment-induced changes, although it has no general orthogonal-invariance guarantee.

Actual training also changes the classifier. We report cross-model differences separately: the reported absolute ReAct AUROC gap changes from 5.542 to 5.090 pp after alignment for independent models, and from 3.860 to 3.909 pp for finetuning. Thus alignment does not consistently close the cross-model gap. The new CIFAR-10/SVHN experiment connects real trained representations to fixed-function coordinate sensitivity; it does not reconstruct complete training updates or estimate deployment failure rates. Adversarial optimization remains a worst-case existence test.

For more detail More data details at this anonymous Github: https://anonymous.4open.science/r/react_invariance-A33F/rebuttal_data/README.md

i thank the authors for clarifications and raising my score.

Thank you for your positive feedback and for reconsidering your score. We sincerely appreciate your time and thoughtful evaluation.

If you have any remaining concerns or questions, we would be more than happy to provide further clarifications or conduct additional experiments to address them.

Reviewer_RdpdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper identifies a weakness of existing, simple OOD detectors based on clipping methods such as ReACT, which, paired with maximum logit or maximum energy detection, applies a global clipping operation to all activations in a hidden layer of the network, aiming to reduce over-confidence on OOD data samples. The authors demonstrate different types of transformations that effectively skew the hidden layer signals (to be clipped by ReACT), even though downstream processing can perfectly correct for such transformations, leaving the classifier output the same. They consider transformations that may induce hidden activation scaling, rotation of the hidden vector, and general affine transformations, with, respectively, per-channel scaling/clipping, Radial-ReAct, and Maha-Radial invariant to these transformations. Experiments, involving an ensemble of random transformations, demonstrate a wide detection performance swing when the detection approach is not invariant to the applied transformation.

What the paper does not show (unless I missed it) is not only whether "natural training approaches the adversarial bases" -- unless I misread, the paper does not even in fact show whether natural training approaches (different realizations of it, or different training configurations) exhibit a performance "swing" similar to what is observed for their controlled, random transformations. While the paper is interesting and has some compelling results, this appears to be its limitation.

Critical strengths

The paper establishes sound theoretical results for what hidden signal representations are needed in order for the detector to be invariant to "usual suspect" transformations such as coordinate scaling, vector rotation, and affine transformations.

The experiments demonstrate that methods such as ReACT can be quite unreliable, in the presence of these transformations.

The experiments also clarify how detection performance depends on the hidden layer dimensionality and that the results hold up with frozen pre-trained (and not small-dimensioned) backbones.

Critical weaknesses

It's unclear what types of neural network training/training configurations have ``predilections'' toward producing hidden signal representations consistent with the transformations considered in this paper.

Short of that, the authors do not (it appears) conduct experiments showing that methods such as ReACT exhibit large swings in detection performance, across an ensemble of model trainings or training configurations.

Some variables are not very clearly defined, along with some terminology: Q_p() ? h(X_cal) ? "Global thresholds cannot be recalibrated back" -- I didn't understand what is meant here.

Questions for authors

Aside from complexity, what is the situation where the Maha-radial method will underperform ?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for recognizing the paper's contributions. Your questions motivated additional experiments on independent training runs, full-network finetuning, and Maha-Radial's calibration conditions, connecting the controlled symmetry analysis to ordinary model development.

W1-W2: Does substantial detector variation occur under actual training?

We now directly measure this under the paper's SGD training protocol and full-network continuation training at one-tenth of the original learning rate. Across ten independently trained pipelines on each of five tasks, ReAct's across-seed AUROC range is 5.18–15.93 pp; on four tasks it exceeds the average within-model range over 100 Haar bases. These comparisons follow the paper's seed-specific calibration protocol (digits: seed-specific ID split too).

To examine training dynamics, we additionally ran ten fresh CIFAR-10/SVHN trainings with nine prespecified checkpoints and continued training three cached final models, updating all network parameters. The following table separates across-run variation from variation within a trajectory. Every range is max minus min AUROC, in pp.

ComparisonReActEnergy
Across 10 fresh final models24.3621.96
Within each fresh run, epochs {12,20,30}4.46–24.013.69–21.38
Full-network finetuning, seed 04.412.39
Full-network finetuning, seed 16.154.67
Full-network finetuning, seed 214.9312.37

Late-checkpoint median: 16.47 pp (ReAct). Finetuning uses eight prespecified snapshots over 15 epochs. Across seeds, calibration follows the paper's sampling protocol. Evaluation images are identical across these CIFAR-10/SVHN runs, and calibration samples are fixed within every trajectory. Thus, the within-trajectory results do not arise from changing evaluation or calibration samples.The relative benefit of shaping also varies. Among the ten fresh final models, ReAct-minus-Energy ranges from −8.62 to +1.65 pp, with four positive and six negative values. During the three finetuning trajectories, this paired difference spans 3.31, 3.22, and 2.93 pp, respectively. These results show variation in both absolute detection performance and the incremental benefit of ReAct under ordinary training.

What the fixed-function experiments add. We separately retain the paper's ten cached models for controlled reparameterization. Across 100 Haar bases per model, ReAct's within-model AUROC range is 2.50–10.49 pp. Positive diagonal scaling also changes recalibrated ReAct: the mean within-draw scaling-sweep range is 4.26–8.90 pp across models. Each of 30 draws is evaluated at seven strengths (sigma in {0,...,1.5}); its across-strength range is averaged over the 30 draws. Compensating the classifier head preserves raw logits, with maximum recorded error at most 2.2e-14.

The training experiments establish that substantial detector variation occurs in ordinary model development; the compensated transformations isolate coordinate dependence while holding the classifier function fixed. Energy also varies under training, as the table shows, whereas it remains unchanged in the compensated control. These experiments therefore answer complementary questions rather than attributing all training variation to a change of basis. This directly addresses the ensemble evidence requested in W2. For W1, it establishes occurrence under standard training and full-network finetuning, without claiming that a particular optimizer systematically produces a particular rotation. Together with the paper's characterization results, these additions give a practical framework for evaluating both cross-pipeline variation and within-fixed-function coordinate sensitivity.

What we can and cannot claim about training-config "predilections." The transformation→effect half is mathematical and measured: any non-negative, sparse representation rotated to a Haar basis acquires dead channels and heavy-tailed scales, and by Proposition 1 a single global quantile threshold is well-posed only when per-channel scales are homogeneous. The training→statistics half is now measured, not hypothesized: along 10 fresh retrainings, per-channel log-Q90 heterogeneity falls from 1.37±0.11 at init to 0.36±0.04 at convergence, 10/10 seeds — the untrained network is 3.8× more heterogeneous and has no gain, so training manufactures the homogeneous coordinates that make a global threshold well-posed. What we do not claim is a causal mapping "optimizer/config X → preference for transformation family Y": the basis is not identifiable from the function (our own Thm-level point), so such a mapping has no group-level meaning, only empirical tendencies — and the reviewer's explicitly offered alternative — performance-fluctuation evidence across training instances — is fully provided by the experiments above. Trajectories, grid results, and manifests are at this anonymous Github: https://anonymous.4open.science/r/react_invariance-A33F/rebuttal_data/README.md

AuthorsOpenReview ↗

Official Comment by Authors

Q: When can Maha-Radial underperform, beyond computational cost?

The new grid identifies when covariance estimation is unreliable and measures how regularization helps. We evaluated the paper's pretrained ResNet-50 (d=2048) on six OOD suites. At each calibration size, arms and bases share the same sample splits; covariance fitting, operational-threshold calibration, and clean-ID evaluation use disjoint subsets.

Calibration samplesPaper's spectral cutoffShrinkage, lambda=0.1
25080.3579.31
51281.3381.44
1,02481.2581.53
2,04866.8080.88
2,56080.6780.77
4,09681.5580.47

Entries are six-suite mean AUROC (%), in the original basis. The spectral cutoff is 1e-12 times the largest eigenvalue; shrinkage uses Σλ=(1−λ)Σ^+λI\Sigma_\lambda=(1-\lambda)\hat\Sigma+\lambda I. A single shrinkage coefficient achieves 79.31–81.53% across the entire tested calibration-size grid. At n_cal=2048, it improves the spectral-cutoff version from 66.80% to 80.88%. This provides a concrete alternative when unregularized covariance estimation is unstable. At n_cal=4096, the unregularized full-rank version performs better, illustrating the regularization trade-off.

At n_cal=d, centering limits covariance rank to n_cal−1; the retained spectrum remains severely ill-conditioned even after cutoff. This illustrates why effective rank and conditioning, rather than floating-point positivity alone, matter for reliable estimation.

At n_cal=2560 and 4096, the unregularized covariance is full rank with no eigenvalues removed. Across the five Haar and five positive-diagonal transformations tested per setting (diagonal sigma=1), every suite's AUROC and FPR95 remain unchanged. This supports the theoretical affine guarantee in its applicable regime: a positive-definite covariance estimated covariantly, without shrinkage or spectral truncation. The observed sample sizes are properties of this experiment, not a universal sample-size threshold. Isotropic shrinkage retains orthogonal equivariance while trading away exact general-affine covariance.

Corrupted-input robustness at a controlled clean-ID operating point. We added an evaluation with disjoint sets for covariance fitting, operational-threshold calibration, and clean-ID testing. This allows us to assess corrupted-input rejection while checking that the intended clean-ID operating point is achieved: realized clean-ID false-OOD rates are 4.8–6.0% across the grid. At n_cal=4096, the unregularized full-rank detector rejects 30.45%, 19.05%, and 13.35% of the noise, blur, and pixelation populations, respectively, averaging 20.95%. Thus, elevated rejection of corrupted ID inputs persists even with separately calibrated thresholds and approximately 5% clean-ID rejection, providing additional evidence that affine invariance does not by itself ensure robustness to input distribution shift.

Table 18 used an in-sample operational threshold, so its extreme rejection rates reflect calibration effects as well as input corruption and should not be interpreted as corruption effects alone. We will make this protocol distinction explicit and add the disjoint-split results, providing a more informative assessment of Maha-Radial's behavior at a controlled clean-ID operating point.

These findings extend the symmetry hierarchy with practical guidance: check covariance rank and conditioning, use regularization when needed, and calibrate the operational threshold separately from fitting the geometry.

W3: Notation and the meaning of recalibration.

We will define h(Xcal)h(X_{\mathrm{cal}}) at first use as the ncal×dn_{\mathrm{cal}}\times d matrix of calibration features, and QpQ_p as the empirical p-quantile. Global ReAct applies this quantile to all entries of that matrix; Radial-ReAct applies it to the vector of per-sample feature norms.

“Global thresholds cannot be recalibrated back” refers to the clipping operator. For positive diagonal DD, pulling a recalibrated global clipper back to the original coordinates gives $ D^{-1}R_{c'}(Dh)]_j=\min(h_j,c'/d_j). $ Restoring the original positive global threshold c would require c′=djcc'=d_j c for every channel, which is impossible when the djd_j differ. The threshold can be recalibrated, but one scalar cannot restore the original clipping map; per-channel calibration resolves precisely this native-rescaling mismatch.

We believe these additions strengthen the paper's practical contribution: actual training variation is now measured, fixed-function controls isolate coordinate dependence, and the geometric alternatives have explicit empirical operating conditions. If a specific concern remains, we would welcome your guidance and would be happy to provide further clarification or targeted experiments.

I have read all the reviews and author responses and affirm my score for this paper.

Reviewer_qDSyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the hidden-basis dependence of post-hoc, training-free OOD detectors that shape the penultimate representation coordinate-wise (REACT, ASH, DICE). It shows their verdicts are not a function of the classifier alone: under function-preserving reparameterizations — positive diagonal rescaling (the ReLU architecture's native symmetry) and orthogonal rotations — the same classifier yields different OOD scores.

Critical strengths
  1. A crisp, novel theoretical characterization of what basis-equivariance demands. The three-tier hierarchy (architecture's natural group → O(d) → affine), the sharp impossibility results (Props. 1, 3; Thm. 2), and the exactly-invariant constructions at each tier are mathematically clean, compact, and checkable. As far as I can verify against the literature, this precise equivariance analysis of coordinate-wise OOD shaping is new, and it gives the community a principled language for a fragility that earlier work (e.g., ActSub) only observed symptomatically.

  2. Exceptionally rigorous methodology behind a consequential, credible negative result. The ImageNet-scale demonstration — REACT's documented +1.7 to +7.9 AUROC gain over raw energy at the trained basis is erased on all six suites by reparameterization alone, with ASH-B swinging up to 77 points — is backed by per-basis re-calibration, machine-precision invariance checks, and an adversarial basis search that inverts REACT (0.97→0.003). This elevates the paper beyond "coordinate-wise methods are basis-dependent" (which is arguably folklore) to a quantified, worst-case-aware robustness statement that practitioners should take seriously.

Critical weaknesses
  1. The interpretive leap from "sensitivity exists" to "the benefit is an artifact" is not fully established. The paper shows worst-case sensitivity under adversarial/random reparameterizations, but it does not show that the trained basis is arbitrary or typical. On the contrary, Table 12 shows the identity (trained) basis is the best of 99 draws on all six ImageNet suites, with the median basis sitting at the raw-energy level — which is consistent with natural training tying the trained basis to OOD-relevant structure. Under that reading, the correct conclusion is "the reported REACT numbers have unbounded worst-case sensitivity under reparameterization" (a serious robustness/reporting problem) rather than "the benefit is an artifact." The paper itself concedes in Limitations that "the symmetry choice is a value judgement," which undercuts the strength of its own headline claim.
  2. The proposed fixes do not preserve the benefit. Radial-ReAct collapses to raw energy at ImageNet scale (mean 0.789 vs 0.788), and Maha-Radial is below REACT at the identity basis on several ImageNet suites (0.921 vs 0.948 on iNaturalist; 0.736 vs 0.878 on OpenImages-O), in addition to its 12–13 FPR95 residual at n_cal<d on ResNet-101 (mitigated only by shrinkage or n_cal≥d). So the paper convincingly demonstrates the fragility, but its recommended replacements largely recover invariance by discarding the coordinate signal that produced REACT's gains — the practical upshot approaches "use raw energy," which is already basis-invariant. This weakens the constructive contribution relative to the diagnostic one.
Questions for authors
  1. Is the trained basis special, and does it matter for the "artifact" claim? Table 12 shows identity is the best draw of 99 Haar bases on all six ImageNet suites, with median REACT AUROC at the energy level. If natural training systematically yields bases with atypical coordinate statistics (e.g., heterogeneous channel scales that make the trained basis informative for OOD), then the trained-basis gain is not an artifact — the correct claim is unbounded worst-case sensitivity. Do you have any evidence (e.g., comparing coordinate statistics of trained vs. random bases, or basis sensitivity under randomly-initialized vs. trained networks) on which interpretation holds?
  2. What should a practitioner actually deploy? Given that Radial-ReAct collapses to raw energy at ImageNet scale and Maha-Radial underperforms REACT at the identity basis on several sets, is there any regime where an exactly-invariant detector strictly dominates REACT at the trained basis (not merely on average)? How does your recommendation differ for near-OOD vs. far-OOD regimes?
  3. Do adversarial bases transfer across OOD suites? The search optimizes per task on held-out halves. Is a single basis adversarial for all six ImageNet OOD suites simultaneously, or must the basis be suite-specific? This determines whether the inversion result is a per-dataset existence proof or a deployment-facing threat.
  4. Please reconcile the headline swing numbers. The abstract's "up to 15.5 points" appears to come from the pretrained-backbone regime (§4.4: 11–16), while §4.2 reports per-seed ranges of 2.0–9.4 for small models. State explicitly which number corresponds to which setting, and whether "swing" is range, RS, or WD.
2 comments
AuthorsOpenReview ↗

Official Comment 1 - Is the trained basis special?

Thank you for the careful and constructive review. We agree with your distinction: the trained-basis gain is genuine, but it is not invariant under function-preserving reparameterizations and can be entirely erased or reversed in the worst case. We will revise the headline claim accordingly. The architecture-native result (positive diagonal rescaling; Propositions 1-2) remains central, as it concerns an exact symmetry of the original architecture rather than an additional invariance requirement.

We conducted four complementary analyses to address whether the trained basis is special.

Random initialization. On a randomly initialized ResNet-50 of the same architecture, ReAct's mean gain over energy decreases from +4.0 to +0.1 AUROC points, with all evaluated detectors (including ASH, DICE, and ViM) achieving 0.49–0.57 AUROC. This supports the interpretation that ReAct's observed benefit depends on the learned representation, although this control alone does not identify the mechanism by which training selects favorable coordinates.

Continuous basis interpolation (all six ImageNet suites). Along six paths from the trained basis toward random orthogonal bases, ReAct's gain on iNaturalist decreases monotonically, crossing zero at approximately 60–75% of the interpolation path. The gains on all six suites approach zero at the endpoints:

OOD suiteInitial gain (pt)Endpoint gain (pt)
iNaturalist+7.9-0.5
Places365+4.5+0.1
Textures+4.4-0.2
SSB-hard+2.9-0.2
OpenImages-O+2.5-0.1
NINCO+1.7+0.1

OpenImages-O and NINCO exhibit small intermediate increases (≤0.5 points). The endpoints are consistent with the earlier Haar-basis sweep (Table 12). These results show that the favorable trained-basis behavior progressively disappears along the sampled paths, rather than only under isolated adversarial transformations.

Cross-seed variability (10 independent trainings × 5 tasks). ReAct's identity-basis AUROC varies by 5.2–15.9 points across training seeds. On four of five tasks, this exceeds the average within-model range across 100 bases. On CIFAR-10/SVHN, the cross-seed range (15.9 points) exceeds every individual seed's across-basis range (≤10.5 points). Raw energy also varies by 4.4–11.7 points across training seeds while remaining exactly invariant across bases. Thus, variability across independently trained classifiers and sensitivity within a fixed classifier are both important, distinct phenomena.

Coordinate statistics. We additionally examined per-channel 90th-percentile activation statistics on ImageNet ID data. The trained representation exhibits substantially lower scale heterogeneity than the Haar-transformed representations (H = 0.223 versus 0.89 ± 0.03). This provides direct evidence that the trained representation has atypical coordinate statistics, consistent with the identity outperforming all 99 sampled Haar bases on every suite (Table 12). Interestingly, the trained coordinates exhibit greater scale homogeneity, rather than greater heterogeneity. These observations characterize the statistical difference between trained and rotated representations, without establishing heterogeneity as the sole causal mechanism.

Revised interpretation. We therefore agree that the trained basis is neither arbitrary in its empirical behavior nor uninformative for OOD detection. Its advantage is real and associated with the learned representation. Our contribution is to show that this advantage is not determined by the classifier function alone, to characterize the symmetry constraints precisely, and to quantify the consequences of violating them. We have revised the abstract and discussion to distinguish genuine trained-basis utility from parameterization-dependent robustness.

The new experimental results are provided in the anonymous supplementary repository: https://anonymous.4open.science/r/react_invariance-A33F/rebuttal_data/README.md

AuthorsOpenReview ↗

Official Comment 2 - Do the fixes discard the benefit? The adversarial basis; swing-number map

Q2 — What should a practitioner actually deploy?

We agree that invariance does not guarantee higher detection accuracy. Radial-ReAct essentially matches raw energy on ImageNet ResNet-50 (0.789 vs. 0.788), but this limitation does not extend to all symmetry tiers. PC-ReAct and full-rank Maha-Radial retain +2.7 and +2.6 AUROC points over energy, respectively (0.815 and 0.814), compared with ReAct's +4.0 points.

Increasing the ID calibration size from 2,000 to 10,000 improves Maha-Radial's mean AUROC from 0.725 to 0.814 and eliminates the measured rescaling residual on all six suites (Appendix: the full-rank repair). Exact affine equivariance requires a nonsingular covariance estimate; shrinkage provides an approximate alternative under rank deficiency.

Strict improvements over trained-basis ReAct do exist. On pretrained ImageNet ViT-B/16 (Appendix Table 21), orthogonally invariant Maha-Radial achieves 0.805 mean identity-basis AUROC versus ReAct's 0.773, a +3.2-point improvement. On MNIST versus Fashion-MNIST, the corresponding values are 0.98 versus 0.95 (Table 1). Thus, invariance need not entail a loss in detection performance, although no universal dominance is established.

Near- versus far-OOD. On ImageNet ResNet-50, PC-ReAct improves over ReAct on Textures (0.920 vs. 0.874) and OpenImages-O (0.887 vs. 0.878), while trailing on iNaturalist (0.929 vs. 0.948). Its average difference is +1.2 points across these three far-OOD suites, but -3.2 points across the two near-OOD suites (SSB-hard and NINCO). The additional Places365 suite also favors ReAct.

Our recommendation is consequently conditional: PC-ReAct is a low-cost alternative when architecture-native symmetry is required; Maha-Radial offers stronger invariance when sufficient ID calibration is available; and raw energy remains a useful invariant baseline. For near-OOD, retaining coordinate information can be beneficial, and the accuracy–invariance trade-off should be reported explicitly. Maha-Radial's computational cost and sensitivity to corrupted ID inputs also preclude a universal deployment recommendation.

Q3 — Do adversarial bases transfer across OOD suites? Yes.

Using the original Cayley-parameterized optimization procedure, pairwise-logistic surrogate, and per-step threshold recalibration, we optimized one basis for 1,500 steps using only the ID and iNaturalist optimization halves. We then froze this basis and evaluated its transfer to all six suites:

OOD suiteIdentityAdversarial basisAUROC decrease (pt)
iNaturalist (held-out)0.9500.01793.3
OpenImages-O (unseen)0.8780.29358.5
Textures (unseen)0.8740.45941.5
NINCO (unseen)0.7610.36140.0
Places365 (unseen)0.8150.57424.1
SSB-hard (unseen)0.6910.56113.0

The same basis nearly reverses the iNaturalist ranking (0.950 → 0.017) and degrades AUROC by 13.0–58.5 points on the five unseen OOD suites. Orthogonality is preserved to 2.6e-14, and raw energy remains invariant.

We additionally optimized a single basis jointly across all six suites, obtaining decreases of 8.3, 4.8, 4.4, 2.8, 2.6, and 1.8 points on iNaturalist, Textures, Places365, SSB-hard, OpenImages-O, and NINCO, respectively. This joint objective produced a weaker solution than the iNaturalist-only objective under the evaluated settings. Warm-starting the joint objective from the converged iNat-specialized basis (three perturbed restarts) converges to the same per-suite envelope as the cold start (iNat -7.6/-8.1/-7.7 pt vs -8.3), so the weaker joint solution reflects a genuine trade-off between suites' adversarial directions rather than an optimization failure.

These results establish cross-suite transferability without requiring suite-specific bases. The scope remains a white-box worst-case reparameterization result, rather than evidence of naturally occurring adversarial bases.

Q4 — Reconciling the headline swing numbers.

The submitted abstract reports 15.8 points. We will explicitly distinguish the following settings:

AUROC range (pt)SettingAggregation
15.8Frozen ResNet-18 + CIFAR-10 linear head, SVHN OOD (Table 20)Across 100 bases
11–16Frozen pretrained-backbone regime (§4.4)Across 100 bases, three head seeds
2.0–9.4Five from-scratch settings (§4.2)Per-seed range across 100 bases
23.3MNIST near-OOD, d=8 (§4.3)Range across 50 bases, averaged over two seeds

The new experimental results, including basis interpolation, initialization controls, cross-seed variability, coordinate statistics, and adversarial transfer, are provided in the anonymous supplementary repository.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.