- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies how post-hoc OOD detectors depend on the coordinate representation of the penultimate feature. It considers three symmetry levels: architecture-native positive channel rescaling/permutation, arbitrary orthogonal changes of basis before the classifier head, and a stronger affine-covariant tier. The paper analyzes ReAct, ASH, DICE and geometric baselines, proposes PC-ReAct, Radial-ReAct and Maha-Radial, and evaluates the resulting sensitivity on several small-scale tasks, pretrained ResNets, ImageNet, and a ViT backbone. The main empirical message is that coordinate-wise shaping can change substantially across function-preserving reparameterizations, whereas geometric constructions can be exactly invariant by design. I find the architecture-native rescaling analysis and the careful quantitative study useful, but I have a central concern about the motivation and novelty of the orthogonal-basis results.
Critical strengths
- The paper is careful and systematic. It distinguishes architecture-native symmetries from stronger basis-agnostic design requirements, checks logit preservation numerically, and evaluates the effect over many models, tasks, and reparameterizations. The positive channel-rescaling result is particularly relevant because it concerns a genuine symmetry of common ReLU-style parameterizations rather than an externally inserted rotation.
- The paper does more than diagnose sensitivity: it gives a clean hierarchy of equivariant alternatives (PC-ReAct, Radial-ReAct, Maha-Radial) and quantifies the robustness/accuracy trade-off at ImageNet scale. This makes the study practically informative even where the qualitative transformation behavior is mathematically expected.
Critical weaknesses
- My main concern is that much of the qualitative invariant/non-invariant pattern under arbitrary orthogonal changes of basis is already implied by the detector definitions. Coordinate-wise operators such as ReAct and ASH are generically non-equivariant to coordinate mixing; DICE ranks coordinate-wise products before masking; in contrast, Euclidean distance, a re-fitted PCA residual norm, and a properly transformed Mahalanobis quadratic form are orthogonally invariant by construction. Therefore, the qualitative distinction in Table 5 is largely predictable a priori. The nontrivial empirical contribution is the magnitude of the effect, its worst-case severity, and whether it occurs under practically relevant reparameterizations. The paper would be stronger if it more sharply separated definition-level consequences from the genuinely new theoretical characterizations and empirical findings, rather than presenting the qualitative rotation sensitivity itself as a central discovery.
| Method | Core dependence | Orthogonal | Positive diagonal | Definition-level reason |
|---|---|---|---|---|
| ReAct | Global coordinate clipping | Generally changes | Generally changes for nonuniform | in general; one global threshold cannot match heterogeneous . |
| PC-ReAct | Per-channel clipping | Generally changes | Exact after per-channel recalibration | , so clipping commutes with ; mixes channels. |
| ASH-B | Coordinate ranking / pruning | Generally changes | Generally changes for nonuniform | Ranking, survivors, and reassignment statistics change. |
| DICE | Coordinate contribution ranking + head mask | Generally changes | Exact for positive diagonal | ; mixes contributions before ranking. |
| kNN | Euclidean distance | Invariant | Generally changes for nonuniform | Orthogonal maps preserve Euclidean distances. |
| ViM | Logits + re-fit PCA residual | Invariant | Not generally invariant | Under the residual subspace co-rotates; general changes Euclidean PCA geometry. |
| Mahalanobis | Mean/covariance quadratic form | Invariant | Exact if covariance is full-rank and re-estimated covariantly; otherwise not guaranteed | Full-rank Mahalanobis geometry is invariant to invertible linear coordinates; rank deficiency/truncation is the caveat. |
| Radial-ReAct | Euclidean norm | Invariant | Generally changes for nonuniform | ; heterogeneous changes the norm. |
| Maha-Radial | Mahalanobis radius | Invariant | Affine-covariant only in the full-rank unshrunk case | Full-rank transform covariantly; rank deficiency/shrinkage weakens nonorthogonal invariance. |
- A second concern is the normative step from classifier-function equivalence to treating arbitrary basis changes as nuisance variation for a representation-dependent detector. The paper explicitly acknowledges that arbitrary basis invariance is a design desideratum, not an architecture fact (Fig. 5), and later calls the symmetry choice a value judgment. End-to-end training jointly shapes the representation and the linear head, so the trained coordinates can contain axis-dependent statistics (e.g., channel-wise sparsity, marginal tails, or coordinate-specific contribution patterns) that ReAct/ASH/DICE intentionally exploit. Applying an arbitrary Q mixes these statistics even though W is compensated so that the raw classifier function is unchanged. The paper's own ImageNet result—that the trained basis is the best of the sampled Haar bases on all six OOD sets, while Radial-ReAct largely falls back to raw-energy performance—is at least consistent with this alternative interpretation. Thus, a changed feature-based OOD score is not by itself evidence of an undesirable parameterization artifact. A stronger justification is needed for treating the trained coordinate system as nuisance variation rather than potentially useful learned structure.
Questions for authors
- The paper correctly states that arbitrary basis invariance is a design desideratum rather than an architecture fact. What is the strongest justification for imposing this desideratum on representation-dependent OOD detectors? Can the authors provide evidence that the trained coordinate statistics exploited by ReAct/ASH/DICE are nuisance variation rather than useful learned structure? A direct analysis linking trained-basis statistics to OOD gains would materially affect my assessment.
- Can the authors separate more explicitly three components of the contribution: (i) invariance/non-invariance that follows directly from the detector definitions, (ii) the nontrivial characterization results (e.g., restrictions imposed by orthogonal equivariance), and (iii) empirical findings about effect size and deployment relevance? In particular, which main conclusions remain if the evaluation is restricted to architecture-native or otherwise deployment-plausible reparameterizations rather than arbitrary orthogonal changes of basis?
If the authors’ response can adequately address the two concerns above, I would be willing to reconsider my current assessment and potentially raise the score accordingly.
4 comments
Official Comment 1 — Beyond definition-level invariance
Thank you for the follow-up. We appreciate your distinction between predictable invariance properties and nontrivial contributions, and agree that the qualitative taxonomy alone does not establish theoretical novelty.
The distinction we would like to clarify is between verifying the transformation behavior of a particular detector and characterizing the class of shaping rules that can satisfy a prescribed symmetry.
-
Impossibility of recalibration (Propositions 1 and 4). Definition-level non-equivariance does not establish whether recalibration can recover the original shaping. Proposition 1 rules out every replacement global threshold as an exact operator-level remedy under nonuniform positive rescaling. Proposition 4 further establishes that, for a fixed input satisfying its active-clipping assumptions, almost every orthogonal transformation admits no scalar threshold recovering the original clipped representation, even if that threshold is optimized rather than percentile-calibrated.
-
Characterization of admissible shapers (Proposition 3 and Theorem 2). Orthogonal equivariance restricts fixed continuous coordinate-wise maps to uniform linear scaling, and fixed continuous per-sample maps, without auxiliary data-dependent structures, to radial forms. Consequently, within these classes, nontrivial coordinate-wise clipping or sparsification cannot coexist with full orthogonal equivariance.
We do not claim novelty for the underlying group-equivariance principles themselves. Our contribution is their formulation as explicit restrictions on OOD activation shaping, together with the impossibility of scalar recalibration, the corresponding constructive alternatives, and their measured accuracy–invariance trade-offs.
Importantly, even without arbitrary orthogonal transformations, Propositions 1 and 2 establish an operator-level obstruction and an exactly equivariant alternative under the architecture's native symmetry, while retaining the learned coordinate axes.
We will restructure the abstract, contribution list, Table 5, and symmetry figure to explicitly distinguish (i) definition-level consequences, (ii) impossibility and characterization results, and (iii) empirical findings about effect size and practical relevance. We agree that the first component should not be presented as an independent discovery.
Official Comment 2 — Native rescaling: aggregate robustness versus decision consistency
Thank you for highlighting the practical significance of native rescaling. We agree that the modest aggregate effects under mild transformations should not be overstated. We extended the ViT experiment to ResNet-50 and investigated whether aggregate stability translates into consistency of individual decisions.
1. Architecture-native parameter rewrite on ResNet-50.
We now implement positive channel rescaling directly in the existing ResNet-50 parameters, without inserting any layer. Scaling is propagated through the final residual stage via the BatchNorm affine parameters, compensated bottleneck input weights, and inverse classifier-head scaling. Positive homogeneity of ReLU preserves the classifier function in exact arithmetic. A 512-image forward audit confirms numerical agreement with the cached-feature computation and zero top-1 disagreements.
Using the validated cached-feature protocol, we evaluated 20 random scalings and one ID-calibrated scaling at kappa=1.25, with detector fitting and operating-threshold calibration on disjoint ID splits.
Table L5. ReAct AUROC changes (pp relative to identity).
| OOD suite | Random mean | Random min | Random max | ID-calibrated |
|---|---|---|---|---|
| iNaturalist | -0.11 | -0.45 | +0.24 | -0.23 |
| SSB-hard | -0.09 | -0.64 | +0.32 | -1.15 |
| NINCO | -0.01 | -0.29 | +0.32 | -0.03 |
| OpenImages-O | -0.02 | -0.24 | +0.18 | +0.35 |
| Textures | 0.00 | -0.26 | +0.28 | +1.68 |
| Places365 | -0.02 | -0.27 | +0.55 | -0.52 |
| Six-suite mean | -0.04 | -0.20 | +0.09 | +0.02 |
PC-ReAct and raw energy remain numerically unchanged at the reported AUROC precision. These results confirm that mild native rescaling is largely benign in aggregate AUROC terms on both evaluated backbones.
2. Aggregate stability does not imply decision consistency.
At the separately recalibrated nominal 5% ID-rejection operating point, across 20 random scalings, approximately 48 ID images (0.59% ± 0.08 pp) and 920 OOD images (1.79% ± 0.28 pp) change their decisions per draw, out of 8,000 ID and 51,518 OOD images. The aggregate ID false-rejection rate remains nearly unchanged (5.9% at identity versus 5.8% averaged across random scalings), yet the identities of rejected samples change.
Importantly, the OOD disagreements are directionally asymmetric: 86% of flips correspond to initially detected OOD samples becoming missed, whereas 14% correspond to initially missed samples becoming detected. Direct computation across the 20 draws gives a net OOD miss-rate increase of approximately 1.3 pp, despite a mean AUROC decrease of only 0.04 pp.
Table L6. Decision disagreement with identity for two selected scaling cases.
| Population | Random | ID-calibrated |
|---|---|---|
| ID (8,000) | 61 (0.76%) | 77 (0.96%) |
| iNaturalist | 444 (4.44%) | 430 (4.30%) |
| SSB-hard | 155 (1.55%) | 316 (3.16%) |
| NINCO | 133 (2.26%) | 115 (1.96%) |
| OpenImages-O | 263 (2.63%) | 260 (2.60%) |
| Textures | 144 (2.55%) | 190 (3.37%) |
| Places365 | 205 (2.05%) | 458 (4.58%) |
| All OOD (51,518) | 1,344 (2.61%) | 1,769 (3.43%) |
Our empirical margin analysis further shows that affected samples concentrate near the original operating threshold. The identity-basis margin thus provides an empirical indicator of susceptibility, rather than a certified predictor of future decision changes.
These findings distinguish aggregate ranking robustness from operating-point and individual-decision robustness. We establish the magnitude of these effects under controlled function-preserving transformations, not their prevalence in deployment. The corresponding accuracy–consistency trade-off and additional bounded-rescaling experiments are discussed in our next comment.
Official Comment 3 — Practical significance and the accuracy–consistency trade-off
Following the ResNet-50 results above, we further quantify the accuracy cost of enforcing native equivariance and examine sensitivity across different rescaling strengths.
1. What is gained by enforcing native consistency?
The native-equivariant alternative retains useful detection gains, although its accuracy cost depends on the backbone.
Table L7. Six-suite mean AUROC and the cost of native equivariance.
| Setting | Energy (%) | ReAct (%) | PC-ReAct (%) | Cost (pp) |
|---|---|---|---|---|
| ViT-B/16 (fixed random D) | 68.93 | 77.13 | 76.52 | 0.61 |
| ResNet-50 (identity) | 78.9 | 82.8 | 81.0 | 1.8 |
PC-ReAct retains approximately 93% and 54% of ReAct's respective gains over energy, while providing exact native equivariance. These results complement the theoretical construction in Proposition 2: consistency across positive channel rescalings does not require discarding learned coordinate axes or reverting to raw energy.
2. Sensitivity across rescaling strengths.
We additionally examined bounded positive rescalings at different condition numbers, refitting the detectors under each transformation.
Table L8. Bounded-rescaling dose-response on ResNet-50 (mean AUROC change, pp).
| kappa | ReAct | PC-ReAct |
|---|---|---|
| 1.25 (Table L5) | -0.04 | 0.00 |
| 1.5 | -0.11 | 0.00 |
| 2.0 | -0.12 | 0.00 |
| 3.0 | -0.26 | 0.00 |
| κ | ΔAUC (pp) | ID FRR (%) | OOD flips (%) | total (pp) | capability (pp) | op-shift (pp) | FPR95 (%) |
|---|---|---|---|---|---|---|---|
| identity | 0 | 5.88 | 0 | 0 | 0 | 0 | 59.63 |
| 1.25 | −0.03 | 5.46 ± 0.10 | 1.79 ± 0.28 | −1.28 | −0.27 | −1.01 | 60.91 |
| 1.5 | −0.11 | 4.88 ± 0.14 | 3.74 ± 0.67 | −3.36 | −0.61 | −2.74 | 62.98 |
| 2.0 | −0.12 | 4.12 ± 0.23 | 6.92 ± 1.26 | −6.68 | −1.00 | −5.68 | 66.31 |
| 3.0 | −0.26 | 3.49 ± 0.24 | 9.89 ± 1.83 | −9.72 | −1.31 | −8.41 | 69.35 |
These experiments show increasing average degradation for ReAct across the tested condition numbers, although its magnitude remains modest. PC-ReAct remains numerically unchanged at the reported precision, consistent with its exact theoretical guarantee.
3. Practical implications and revised interpretation.
We therefore do not claim that native equivariance is necessary to prevent a large average accuracy collapse under mild rescaling. Rather, our experiments establish three distinct findings:
- Aggregate ranking performance: ReAct is largely robust to the tested mild native rescalings.
- Operating-point and decision consistency: Despite the small AUROC changes, individual OOD decisions change, with a net increase of approximately 1.3 pp in the OOD miss rate on ResNet-50.
- Constructive guarantee: PC-ReAct removes native parameterization dependence by design while retaining useful detection gains, at a measurable, backbone-dependent accuracy cost.
When aggregate detection accuracy is the sole objective, ReAct may remain preferable. When consistency of individual decisions across equivalent native parameterizations is required, PC-ReAct offers an explicit alternative.
We agree that the practical importance of this guarantee depends on the application. Our experiments quantify sensitivity under controlled function-preserving transformations rather than establishing their prevalence in real deployments.
Together with the characterization results discussed in our first follow-up comment, these findings separate what follows from detector definitions, what is guaranteed by theory, and what must be measured empirically. We will revise the manuscript accordingly, avoiding blanket claims about the undesirability of learned coordinate dependence.
The full per-draw results, dose-response experiments, and individual decision lists are provided in the anonymous supplementary repository.
Response to Rebuttal: Remaining Concerns
Thank you for the detailed rebuttal and additional experiments. We appreciate the authors' efforts to address our concerns.
While the rebuttal provides useful clarifications and additional evidence, we remain unconvinced about the novelty of the theoretical contributions beyond the definition-level observations. Moreover, although the architecture-native scaling experiments validate the reported sensitivity, the observed performance changes under the tested mild rescalings are modest, leaving their broader practical significance unclear.
Therefore, we maintain our original score.