acceptodds
Under review as a conference paper at ICLR 2027

Anti-coherence Regularized Absolute Scaling for Out-of-Distribution Detection

Abstract

Out-of-distribution (OOD) detection is recognized as a critical problem in AI safety. SCALE is an effective post-hoc OOD detection method in the activation-shaping family that requires no training data and rescales penultimate activations using a percentile-based magnitude ratio. However, this ratio is ill-defined when negative activations are present, since positive and negative contributions cancel within the underlying sum, causing SCALE to degenerate on backbones such as ViT. We address this with an Absolute Fix that computes the ratio over absolute-valued activations rather than discarding negative activations through ReLU. Building on this corrected scale, we propose Anti-coherence regularization, which penalizes the scaling factor by the fraction of a sample's max-logit contributions to the max logit that are sign-mismatched and thereby suppresses unreliable evidence for the predicted class. Despite requiring no access to training data or training labels, our method ties or outperforms activation-shaping methods that rely on training data in terms of AUROC and FPR@95 on CIFAR-10 and ImageNet under the OpenOOD v1.5 protocol.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Out-of-distribution Detection, Activation Shaping Method, Post-hoc OOD detection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_xTe8OpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper makes the improvements in activation shaping based methods. For the ASH/SCALE paper, the negative activations have detrimental effect on scaling. These methods dont generalize to architectures (ex. ViTs) which produce negative activations. Those methods primarily work well on architectures with non-negative activations such as ResNets and using batch normalization.

The paper proposes two fixes in the heuristics formula used by ASH / SCALE. Instead of using the original activations, they replace that with absolute activations. Hence, it measures relative strength of absolute top-k activations with respect to absolute total activations. Apart from that, paper proposes anti-coherence fix which determines disagreement of constituent dimensions with the prediction. ID samples have relatively less disagreement among dimensions while OOD samples have more anti-coherence and the scaling factor is penalized more. Combining these two heuristics, paper improves over the scaling the activation for separable energy score between ID and OOD.

Critical strengths

The findings of the paper is genuinely novel. The absolute fix is very simple, just two operation in existing scaling heuristics. And, the anti coherence term introduced by the paper is also simple and genuinely novel to the best of my knowledge. It's not been used in the literature so far to my understanding. The additive contribution of the paper doesn't introduce non-trivial computational overhead.

The performance of the method across architectures is robust. The paper makes meaningful advance in scaling based OOD detection. The paper has good presentation. There's no concern of hyperparameter sensitivity too.

Critical weaknesses

There are no critical weakness as such in the paper. This paper is a clear accept for me. However, a few clarifications could be useful.

Performance seems largely attributed to the absolute fix. How much does Anti-coherence alone contribute to the OOD detection? Does anti-coherence fix alone added to SCALE bring improvement over the average performance across all architectures in ImageNet + CIFAR datasets?

Or does anti-coherence contribute only when absolute fix is present? A discussion on this would be useful. And, could authors separate its result alone on relu-activations architectures vs others?

Furthermore, I am curious whether both of the components of the paper ( absolute fix and anti coherence) can be applied to AdaSCALE for performance boost similarly to SCALE? Results on these would be interesting.

The paper seems to have two hyperparameters which needs exhaustive tuning or could authors provide discussions on that? How much would the average performance become if lambda is fixed to one value (determined from one settting) across all settings? Does it still outperform all methods on average performance?

What do authors mean by SCALE + Relu. Is this LTS?

Questions for authors

Please see weaknesses.

Reviewer_E9d7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the limitations of SCALE on backbones with negative activations by proposing Absolute Fix, which computes the scaling factor using absolute-valued activations to avoid cancellation between positive and negative values while preserving the magnitude information of negative activations. Building on this, the authors introduce Anti-coherence, which decomposes the predicted class logit into positive and negative feature contributions and adjusts the scaling factor based on the proportion of negative contributions relative to the total contribution magnitude. Experimental observations show that OOD samples generally exhibit higher Anti-coherence values, motivating stronger suppression of their scaling factors to improve ID/OOD discrimination. The proposed method requires neither additional training nor gradient computation at inference time and achieves strong OOD detection performance across a variety of backbone architectures.

Critical strengths
  1. The paper identifies a clear limitation of SCALE when handling negative activations and proposes an effective solution. Extensive experiments across multiple datasets and backbone architectures demonstrate the effectiveness of the proposed method.
  2. The paper is well-organized and clearly written. The research motivation and core ideas are easy to understand, and the proposed method is intuitively motivated.
  3. The proposed method requires neither additional training nor gradient computation, relying only on the features and classifier weights of a pretrained model. This results in low computational overhead and makes the method practical for real-world deployment.
Critical weaknesses

1.The motivation behind the Anti-coherence integration strategy is not sufficiently explained.

Questions for authors
  1. Figure 3 illustrates the distributional differences in the Anti-coherence metric A(z) between ID and OOD samples, while Table 5 demonstrates that incorporating A(z) into SCALE+Abs improves detection performance. I am curious about the specific motivation behind the design . Given that A(z) itself exhibits some OOD discriminative ability, why not use it directly as an OOD detection score or combine it with the Energy score of SCALE+Abs? Does incorporating A(z) into the scaling factor offer any particular advantages over these alternatives?
  2. Although the proposed method achieves promising results on CIFAR-10 and ImageNet, Table 11 shows that its performance on CIFAR-100 is even worse than that of the original SCALE. The authors suggest that this may be due to insufficient feature representation quality, but direct experimental evidence supporting this explanation is lacking. Since ResNet-18 produces non-negative features, the Absolute Fix does not change the original SCALE scaling factor, suggesting that the performance degradation may be related to the Anti-coherence regularization. Could the authors further analyze the distributions of A(z) for ID and OOD samples on CIFAR-100 and explain why introducing this regularization term leads to a decrease in detection performance?
  3. Table 3 shows that the original SCALE outperforms AdaSCALE-L on CIFAR-10. Given that AdaSCALE was proposed as an improvement over SCALE, could the authors briefly explain this observation and clarify whether the performance difference might be related to experimental settings, hyperparameter selection, or other factors?
Reviewer_JiTpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper aims to address limitations of SCALE, an effective post-hoc OOD detection method in the activation-shaping family that requires no training data and rescales penultimate activations using a percentile-based magnitude ratio. The proposed method begins with computing a percentile-based magnitude ratio over absolute-valued activations, followed by penalizes the scaling factor by the fraction of a sample's max-logit contributions to the max logit that are sign-mismatched and thereby suppresses unreliable evidence for the predicted class.

Critical strengths
  1. this paper is well writeen and easy to follow
  2. figures are informative
  3. experiment results are extensive
Critical weaknesses
  1. According to Eq. (6), A(z)A(z) in Eq. (7) can turn into a simpler form: $ A(z)=\frac{1}{2}\left(1-\frac{\ell_{c^*}} {\sum_j |W_{c^*,j}z_j|}\right)$. As a result, the so-called anti-coherence is essentially an affine transformation of a normalized predicted-class logit. However, the authors fails to analysis why this particular normalization provides information beyond simpler confidence measures.

  2. This paper does not disclose: 1) Why ID data is expected to have smaller anti-coherence, 2) why subtracting λA(z)\lambda A(z) in Eq. (8) is expected to improve detection, and 3) when this operation will damage ID/OOD ranking. As a result, it is unclear why the proposed method can work.

  3. While Table 1 shows that MaxLogit can dominance after applying rabsr_{\mathrm{abs}}, whereas the final ood detector based on rabs−λA(z)r_{\mathrm{abs}}-\lambda A(z), which can be potentially much smaller than rabsr_{\mathrm{abs}} especially when λ\lambda is as large as 12. As a result, there can be a significant gap between between the motivation and the implementation.

  4. The zero-bias assumption in Lemma 4.1 is insufficiently justified. While the authors state in appendix F that removing bias did not affect performance, this is not supported by quantitative ablation.

Questions for authors

see weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.