acceptodds
Under review as a conference paper at ICLR 2027

One Score for All: Robust OOD Detection across Pretrained Models

Abstract

Post-hoc out-of-distribution (OOD) detection should remain useful across pretrained classifiers and calibration budgets. In practice, however, changes in the pretrained model can reverse the in-distribution (ID)/OOD ordering of feature statistics, while class-relative references can become unreliable when estimated from few samples. We propose ONE, a common scoring and fitting procedure that uses pretrained class information both to combine detection evidence and to estimate its reference. Raw logits and Gaussian-form fit in normalized feature space are combined within corresponding classes before pooling. The resulting score retains each branch's overall evidence together with their soft class overlap, rather than discarding class correspondence through scalar fusion. We estimate the feature reference by shrinking empirical class centers toward classifier-derived priors according to sample counts and estimated mismatch, while regularizing covariance according to sample availability. The reference and separate evidence scales are fitted using labeled ID data, with the network frozen. Across nine benchmark settings, ONE achieves 86.3 mean AUROC with full calibration. On 30 public ImageNet checkpoints, it achieves 85.1 mean AUROC and a 6.8-point maximum gap to the best evaluated score on each checkpoint, improving both criteria over the evaluated alternatives. With one labeled ID calibration sample per class, it achieves higher mean AUROC (84.2) than the classifier-only GEN baseline (83.5).

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Out-of-distribution detection, post-hoc detection, robustness, architectures, training recipes

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_b1zNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes proper fusion of logit and feature contribution to final OOD detection score which works across different pretrained models. As logits and features live in different space, paper proposes scaling them with different scales and then combine them. If logit and feature information are not pointing towards same prediction then, the sample is more likely to be OOD. The paper also proposes to supplement the condition where the no of images available are less and then, it borrows the classifier information to get the class mean representation blended with images' mean representation.

Critical strengths

The paper provides the extensive experimental results. And the paper proposes to use the logit and feature agreement in the final OOD detection score. This might be a novel contribution of the paper. It also provides a way of estimating class center in limited ID data regime by leveraging the classifier weights. This is also useful thing when data is limited.

Critical weaknesses

The gains are near-trivial and the idea of the paper doesn't add much performance. The novelty is highly incremental and is additive of well known observations. Model sensitivity in OOD detection has been well documented. Performance variability across models is well documented and has also been attributed to feature norm. Though the evaluation is broader, there’s is not significant novel insights in the paper. ComboOOD also outperforms ONE in FPR metric in Table 7. The main text doesn’t discuss FPR metric and just centers the discussion around AUROC metric only as it doesn’t outperform ComboOOD in FPR metric. In table 3, again, comboood outperforms one in far column substantially.

Questions for authors

None.

3 comments

We thank the reviewer for the constructive feedback. We address the novelty, the size of the gains, and FPR95 in turn; the revised PDF marks changes in red.

1. Novelty. "The novelty is highly incremental … Model sensitivity in OOD detection has been well documented." We agree that model dependence is well documented (Section 2); prior work mainly reports it or recovers performance by fine-tuning or retraining the network. Among post-hoc approaches, Mahalanobis++ traced the variation of Mahalanobis scores to the feature norm and removed it by normalizing the features; ONE adopts this normalization for its feature evidence. Beyond this normalization, ONE combines the classifier and the feature evidence within each class, checking that both favor the same class, and estimates the class references from classifier-derived priors. Both components have a measurable effect:

  • Class agreement: with both branches fixed, adding the class-overlap term (Eq. 3) to scalar fusion raises the 30-checkpoint panel mean AUROC from 84.6 to 85.6 and improves 26 of the 30 checkpoints (Table 5); the gain is largest where the branches disagree on OOD inputs (ResNet-50 AugMix: +4.7; Appendix C). MME checks the same correspondence with a hard classifier–nearest-mean consistency factor (Table 1); on the four checkpoints of Appendix C.1, adding only this factor to scalar fusion raises mean AUROC by 0.30 on average (at most 0.91), against 1.42 (at most 4.70) for the class-overlap term.
  • Classifier-informed centers: at one image per class, they raise the feature evidence alone from 77.1 to 83.4 AUROC, and ONE from 83.6 to 84.6 (Table 7; Appendix C, Table 16).

2. Gains. "The gains are near-trivial and the idea of the paper doesn't add much performance."

  • Few calibration images: at one image per class, ONE and NCI are nearly tied in nine-setting mean AUROC (84.6 and 84.5), while NCI's max regret is 6.1 against 2.9 for ONE. At two images per class, ONE reaches 85.3 versus 84.4 for NCI, and at five, 86.0 versus 84.7 for CTM; NCI and CTM are the strongest competing scores at these budgets (Table 6, now in the main text). ONE stays above the best baseline in each of four random draws at every budget. We emphasize this regime because it is directly linked to the labeling cost of deploying a detector for user-defined ID classes.
  • Consistency: with full calibration, ONE has the smallest maximum regret among the 25 scores (General response, point 2), and across all evaluated classifiers, including the 30-checkpoint panel, paired comparisons favor ONE (Appendix C, Table 13):
ONE minusHEATCOMBOODRMDS++MDS++
ΔAUROC [95% CI]+1.7 [1.1, 2.3]+1.7 [0.9, 2.6]+2.0 [1.3, 2.7]+2.2 [1.2, 3.3]
Classifiers where ONE is higher76%74%79%79%

3. FPR95 and COMBOOD. "ComboOOD also outperforms ONE in FPR metric in Table 7 … In table 3, comboood outperforms one in far column substantially." FPR95 was in the appendix only for space; the revision places it next to AUROC (Table 3). On the 30-checkpoint panel, ONE has the lowest mean FPR95 and the smallest mean and maximum FPR95 regret; over all evaluated classifiers, including the panel, it has the lowest mean FPR95 (53.2; COMBOOD 55.2) and the lowest mean FPR95 regret (6.2; COMBOOD 8.2). COMBOOD's advantage in the far column comes from CIFAR far-OOD (AUROC 92.3 vs. 86.8), where it also uses intermediate-layer activations; on ImageNet, ONE is ahead on both near-OOD (84.1 vs. 81.9) and far-OOD (89.6 vs. 86.4; Table 4).

The gains are very trivial and the studies of eg. one image per class isn't a very realistic scenario. I thank authors for the response but I keep my original score.

Thank you for the feedback. We respectfully disagree on both points, for the reasons below.

Regarding the low-budget settings, one labeled image per class is an established evaluation setting: few-shot OOD detection methods are evaluated on ImageNet with one to sixteen labeled images per class, e.g., the prompt-learning method LoCoOp (Miyai et al., NeurIPS 2023). For a frozen classifier, such budgets arise when its training data cannot be used for calibration, for example, for a released model trained on proprietary data. Against the four baselines with the highest mean AUROC (HEAT, COMBOOD, RMDS++, and MDS++), ONE has the highest mean AUROC at every budget on 9 benchmark settings, and with two images per class it stays within 1.6 points of its full-calibration value, whereas these baselines lose 8 to 14 points (Table 6):

Images per class12520Full
HEAT79.276.084.585.986.3
COMBOOD77.777.778.077.886.2
RMDS++--71.779.785.285.4
MDS++--77.683.384.785.5
ONE84.685.386.086.686.9

--: undefined at one image per class.

Regarding the gains, the table below compares ONE with these four baselines, which have the highest mean AUROC, with full calibration (Table 2):

9 benchmark settings30 ImageNet models
Mean AUROC ↑Mean regret ↓Max regret ↓Mean AUROC ↑Mean regret ↓Max regret ↓
HEAT86.32.44.683.73.510.1
COMBOOD86.22.510.083.43.710.6
RMDS++85.43.49.383.33.810.9
MDS++85.53.37.583.14.016.0
ONE86.91.84.485.61.66.7

Regret: AUROC gap to the best of the 25 scores on each setting or checkpoint (Eq. 11).

ONE has the smallest mean and maximum regret in both groups. On the 30-checkpoint panel, its mean regret is less than half that of each baseline, and each baseline falls 10 or more points behind the best score on some checkpoint, whereas ONE stays within 6.7.

We would be glad to address any further questions or concerns. Thank you.

Reviewer_4248OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents a post-hoc scoring function for OOD detection, specifically targeted at achieving competitive performance across different classifier architectures and sample budgets for tuning the scoring function. This is accomplished by a formulation using both classifier logits and normalized features, aggregated across all classes, i.e. not only the predicted class or scalar fusion of pooled information. The feature contribution is based on the available sample budget, where the distance to feature sample mean is predominantly used if many samples are available, and if few samples are available information is pulled from the classifier matrix to act as prior. Experiments demonstrate how the proposed method shows less performance variation across different architectures and sample budgets, with average performance setting a new SOTA compared to a large range of previous methods.

Critical strengths
  • Overall an interesting focus on OOD detection robustness across different settings. While sensitivity to architecture is a somewhat common problem for previous methods, it is also interesting to see the investigation of different sample budgets. The proposed combination of logits and distance to empirical mean vs. classifier prior comes across as an effective strategy for increasing the robustness.

  • The paper is of empirical nature, but it is rigorous in its experiments, comparing a large range of architectural variations, datasets, sample budgets, and previous OOD scoring methods, as well as ablations of the different design choices used to formulate the proposed method.

Critical weaknesses
  • The paper requires some effort to comprehend, where I believe that some of the explanations could be formulated differently to facilitate understanding. As an example, in Tab. 1 it is difficult to understand the sentence "whether and how the combination uses which class each source supports". But this also goes for some of the formulations in the text. Also, it requires some effort to follow the equations, where central variables go without direct explanation. Instead, the narrative is top-down, where the variables are explained in some subsequent section, requiring the reader to go back and forth to a large degree. One direct way of facilitating understanding would be to first provide a short summary of the method, followed by the detailed explanations. An algorithm would also facilitate connecting the dots.
Questions for authors
  • Only AUROC is reported in the main text, whereas FPR95 is reported in the appendix with the proposed method placing 2nd on average. Have you noticed a trend in terms of the proposed method being favored more when using AUROC compared to FPR95? How about also using the same "regret" measure as in Eq. 11, but based on FPR95?
  • The proposed method relies on labelled ID data for calibration/tuning. Would it be possible to adapt it for label-free tuning?
4 comments

We thank the reviewer for the positive assessment and for the concrete suggestions on presentation.

Presentation (W). We followed your suggestions to make the method easier to follow. The revision (changes in red) (i) rewrites the Table 1 sentence you quote as "whether, and how, the combination checks that the classifier evidence and the feature evidence favor the same class"; (ii) opens Section 4 with an overview that defines the score and its central variables and states how each part is estimated, before the details; and (iii) adds Algorithm 1 (Appendix B), which lists the calibration and scoring steps.

Q1. AUROC and FPR95. On the nine benchmark settings, ONE ranks first in mean AUROC and second in mean FPR95, as you note (52.8 versus 50.6 for COMBOOD). On the 30-checkpoint ImageNet panel, ONE leads on both metrics, with mean FPR95 of 55.4 versus 59.0 for COMBOOD (Tables 2 and 3). FPR95 is now in the main text with the same layout as AUROC, including the FPR95 regret you suggest (Table 3: on each setting or checkpoint, a score's FPR95 minus the lowest FPR95 among the 25 scores). Over all evaluated classifiers, including the 30-checkpoint panel, ONE has the lowest mean FPR95 and the lowest mean and median FPR95 regret:

All evaluated classifiersONECOMBOODHEATMDS++
Mean FPR95 ↓53.255.256.656.8
Mean FPR95 regret ↓6.28.29.79.9
Median FPR95 regret ↓2.97.18.97.5

COMBOOD's lead on the nine settings comes from the three CIFAR settings.

Q2. Label-free calibration. Yes. ONE uses the labels in three places: the correct-classification filter, the grouping of calibration images into classes, and the fit of the two scales. We replace the labels by the classifier's predictions in all three and fix κ_l = 1, the classifier's own temperature (with predicted labels the likelihood of κ_l has no finite maximum). On the nine settings, this label-free ONE reaches 86.8 mean AUROC with full calibration (86.9 with labels) and keeps the best mean AUROC and mean regret among the 25 scores; at every balanced budget it stays within 0.4 AUROC of the labeled version (Section 5.3; Appendix C, Table 14).

Thank you again for the positive assessment and the concrete suggestions. We hope the revised presentation (the Table 1 caption, the method overview, and Algorithm 1), the FPR95 results and FPR95 regret in the main text, and the label-free results address your comments. We would be glad to clarify anything further before the discussion closes.

Thanks for the additional explanations and clarifications. Also interesting to see the potential of the proposed method in the label-free setting. Overall, I remain positive and keep my original score (the revision makes clarifications, but do not add anything substantial that would justify increasing the score).

Thank you for the follow-up. Your suggestions on the presentation and the FPR95 regret, and your question on label-free calibration, have improved the paper.

Reviewer_nJY3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ONE, a post-hoc OOD detection method that combines classifier logits and class-conditional feature fit before aggregating over classes. Its feature references use normalized features, classifier-derived class-direction priors, and calibration-dependent shrinkage of both class centers and covariance. The authors evaluate ONE on nine benchmark settings, 30 public ImageNet checkpoints, and controlled ResNet-50 variants, including experiments with small per-class calibration budgets.

The paper is clearly organized, and its central motivation—that OOD score behavior varies across models and calibration budgets—is relevant. The results are promising, especially for the broad ImageNet checkpoint panel and low-budget calibration. My main reservation is whether the empirical gains establish a sufficiently general advance over strong post-hoc baselines, given the limited uncertainty analysis and the method’s reliance on correctly classified calibration examples.

Critical strengths
  1. The paper addresses an important practical issue: OOD scores can behave differently across pretrained models and calibration budgets.

  2. Matched controls show that class-wise fusion improves over scalar fusion on the 30-checkpoint ImageNet panel.

  3. The low-budget results are promising: ONE reports higher mean AUROC than GEN with one labeled calibration sample per class.

  4. The evaluation spans a broad set of models, and the paper provides useful implementation details.

Critical weaknesses
  1. The reported gains over strong baselines are point estimates. Uncertainty estimates or paired significance tests would help establish whether the improvements are reliable across OOD datasets and calibration draws.

  2. The reference estimator uses only correctly classified calibration examples. The paper should quantify how many examples are excluded and how often classes have no usable examples, especially at low budgets.

  3. The prior strength is estimated from the same calibration data used to fit the references. Its sensitivity to small-sample noise and to the estimation procedure is not fully established.

  4. The low-budget comparison deserves a more matched protocol. At low budgets, ONE fits evidence scales using ID labels, while GEN does not require reference fitting. The comparison would be clearer with ablations separating the benefit of the proposed reference from the benefit of supervised scale fitting.

Minor:

  1. The main tables average AUROC equally across OOD datasets. This is transparent, but a few easier or larger datasets may have substantial influence on the aggregate. Reporting per-dataset variation or a complementary median would help readers interpret the mean.

  2. The paper discusses both AUROC and FPR95, but the central claims focus on AUROC. Since operating-point behavior matters for rejection systems, the relationship between ONE’s AUROC improvements and its FPR95 results should be emphasized more in the main text.

  3. Please make explicit whether the ID examples used to fit the evidence scales are disjoint from the ID examples used to estimate centers and covariance. The paper states that both use the calibration set; if they overlap, it would be useful to discuss the consequences for small-budget fitting.

Questions for authors
  1. What fraction of labeled calibration examples is excluded by the correct-classification filter, and how often does a class lack usable examples?

  2. Are the improvements over the strongest baselines statistically reliable across OOD datasets and calibration draws?

  3. How much of ONE’s low-budget performance comes from classifier-derived centers versus fitting the two evidence scales with labels?

3 comments
AuthorsOpenReview ↗

Comment 1 of 2

We thank the reviewer for the careful reading and the concrete suggestions.

A. Reliability across OOD sets and calibration draws (W1, Q2)

ONE minusHEATCOMBOODRMDS++MDS++
ΔAUROC [95% CI]+1.7 [1.1, 2.3]+1.7 [0.9, 2.6]+2.0 [1.3, 2.7]+2.2 [1.2, 3.3]
Classifiers where ONE is higher76%74%79%79%
(Classifier, OOD set) pairs where ONE is higher75%65%68%68%

Over all evaluated classifiers, including the 30-checkpoint panel (Appendix C, Table 13). The CIs bootstrap over classifiers, resampling all OOD sets of a classifier together; a paired Wilcoxon test over classifiers gives p < 0.001 against each baseline (Holm-corrected).

Across calibration draws: at every budget from one to fifty images per class, ONE's nine-setting mean is above the best baseline in each of the four draws (28 of 28), and its standard deviation over draws is at most 0.13.

B. Correct-classification filter (W2, Q1)

Our budget k counts labeled calibration images per class before the filter, so a class has no usable image when all k of its images are misclassified. Across all evaluated classifiers (Appendix A, Table 9), the filter excludes 8.7–9.0% of the calibration images on average at every budget (at most 22.9%, MobileNetV2 V2). Classes with no usable image become rare quickly:

Images per class1235≥ 10
Classes with no usable image, mean (max)8.7% (22.9%)1.6% (6.9%)0.4% (2.2%)0.06% (0.35%)≤ 0.05%

ONE does not drop such a class: with n_c = 0 its center is the classifier prior, the same estimate it uses for a class with no calibration image, and its misclassified images still enter the global mean and the fitted scales. At one image per class the centers are the priors for every class anyway (mean blend weight 0.00, Table 7). Part C also shows that ONE does not depend on this filter: with the classifier's predictions in place of the labels no image is excluded, and the nine-setting mean AUROC changes by at most 0.4 (Table 14).

C. Classifier-derived centers versus supervised scales at low budgets (Q3, W4)

To separate the two, we replace the calibration labels by the classifier's predictions everywhere ONE uses them (the correct-classification filter, the class grouping and the fit of κ_m) and fix κ_l = 1, the classifier's own temperature (with predicted labels the likelihood of κ_l has no finite maximum), so that no label enters the calibration. On the same calibration subsets (Table 14), this label-free ONE stays within 0.4 AUROC of the labeled version at every budget, and it is above GEN at every budget in each of the four draws (28 of 28). The low-budget performance therefore does not come from fitting the scales with labels. The classifier-derived centers are what lift ONE above GEN at one image per class: with sample-mean centers it reaches 83.6, which does not exceed GEN (83.9), against 84.6 with the classifier priors (Table 7).

Prior strength (W3). The prior strength (the pseudo-count n_0 in ρ_c) matters only between the two ends of the budget range: at one image per class every center is its classifier prior, and at full calibration every center is its sample mean (mean blend weight 0.00 and 1.00, Table 7). In between, Table 7 brackets the estimation procedure by its two limits, n_0 → ∞ (classifier priors) and n_0 → 0 (sample means): the estimated n_0 matches or exceeds the better limit at every budget and exceeds both at two images per class (85.3 vs. 84.8 and 85.2). Its small-sample noise is part of the draw-to-draw variation in A: n_0 is re-estimated on each draw, and ONE's nine-setting mean varies by at most 0.13 (standard deviation over draws).

AuthorsOpenReview ↗

Comment 2 of 2

Per-dataset variation (Minor 1). Equal averaging over the OOD sets does not drive the ranking: with the median over each checkpoint's OOD sets in place of the mean, ONE again has the highest average over all evaluated classifiers (87.7; next HEAT 85.8), as reported in Appendix C (Statistical Reliability).

FPR95 (Minor 2). FPR95 is now in the main text (Table 3), with the same layout and regret as AUROC. Over all evaluated classifiers, including the 30-checkpoint panel, ONE has the lowest mean FPR95 (53.2; next COMBOOD 55.2) and the lowest mean FPR95 regret (6.2; COMBOOD 8.2).

Overlap of calibration images (Minor 3). The two sets overlap: the centers and covariance are estimated from the correctly classified calibration images, and the two scales are fitted on all calibration images, so the first set is contained in the second and no image is held out when only a few are available. The overlap affects only the feature scale κ_m, since the classifier scale κ_l does not use the centers. A calibration image contributes to its own class center, so its distance to that center is shorter than for a new ID image of the class; the feature evidence thus appears more discriminative than on new images, which favors a larger κ_m. The bias is set mainly by the weight of an image in its own center, ρ_c/n_c, which follows from the blend weights in Table 7. This weight is near zero at one image per class, where the centers are almost entirely the classifier priors (ρ̄ = 0.00) and the covariance is almost fully shrunk to the isotropic reference (λ̄ = 1.00); it is largest at a few images per class, where the prior reduces it (about 0.24 at two images per class, against 0.5 for sample means); and it vanishes as the budget grows (0.04 at 20 images per class, below 0.001 with the full training sets). The estimates shared by all classes (the global mean, the coefficient of the classifier prior, and the covariance) use the same images at every budget. Appendix B of the revision now states this overlap and its effect.

Thank you again for the detailed review and the concrete suggestions. We hope our response addresses your questions on statistical reliability across OOD sets and calibration draws, the correct-classification filter, and the contribution of the classifier-derived centers versus the label-fitted scales at small budgets. We would be glad to clarify anything further before the discussion closes.

Reviewer_rt4QOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ONE, a score used to find out-of-distribution (OOD) data for a fixed image classifier. It is meant to work with many different pretrained models. To get the score, it looks at every class. It combines the classifier's logit with the distance between the input's feature and that class's center. Then, it mixes these values for all classes. If there is very little labeled data, it uses the classifier's weights as class centers. In tests on nine CIFAR and ImageNet setups and 30 public ImageNet checkpoints, ONE worked very well. It had the best average AUROC compared to 24 other methods. It was always close to the best method on every checkpoint. It also works even with only one labeled example per class.

Critical strengths
  1. The study tests 30 ImageNet checkpoints, eight ResNet-50s with different training settings, and 24 other methods. ONE stays close to the best method on every checkpoint. The code provided matches the math in the paper. The experiment is generally comprehensive
  2. The method combines the classifier's score and the feature fit. It rewards when both agree on a class. The class centers and spreads adjust themselves based on the data, so you don't need to tune them for every model.
Critical weaknesses

W1. While the study is comprehensive within its scope, it does not cover everything the title claims as "ALL Pretrained Models". The title says the score works for "pretrained classifiers," but all tested models are small CIFAR or ImageNet-1K models. It does not test large models like CLIP or SigLIP, or vision-language models used for zero-shot tasks, even though the method could work on them. Also, ONE needs a linear head, access to features and weights, and labeled data, but the paper does not mention these needs. Without testing large models, we don't know if the score works for today's most important models.

W2. ONE is only the best in a few cases. On transformer and ConvNeXt models, it is always worse than the best baseline. Also, its robustness was tested on the same 30 checkpoints used to build the method, not on new ones. The results show it is "rarely far from the best on small ImageNet models," not "one score for all pretrained models."

W3. The idea makes sense, but the paper's own tests show the new parts don't add much. Adding the two scores without class-wise pooling already beats all baselines. Class-wise pooling only adds a tiny bit more. The classifier-informed centers don't change anything when there is plenty of data. Also, the paper compares ONE with GEN when there is one example per class, but NCI works just as well there. Finally, a similar study on how training affects OOD detection was already published (Xiong et al., ICCV 2025) but is not cited.

Questions for authors
  1. Would ONE work on large models and vision-language models (like CLIP ViT-L/14, SigLIP, or EVA02-L)?
  2. I used Claude Code to run parts of your code on public data and found two issues. Please check them: First, the feature extraction uses ImageNet statistics for every input. However, three timm ViT-B/16 checkpoints (including the ViT-B setting) expect a mean and std of 0.5. Using your method, their top-1 accuracy on ImageNet validation images drops by 4–5 points. Since the paper doesn't report accuracy, this is hidden. Second, on the OpenOOD CIFAR-10 checkpoints, the training set is classified almost perfectly. This causes the two fitted scales to keep growing as more steps are used (in my tests, this changed AUROC by at most 0.1). It might be worth it to fix the ViT preprocessing, report the accuracy and the fitted scales for every checkpoint, and re-run Tables 2, 4 and 9.
5 comments
AuthorsOpenReview ↗

Comment 1 of 2

We thank the reviewer for the constructive comments, and for the effort of running our code on public data.

1. Scope and large models (W1, Q1)

Scope. We agree that the title claims more than we evaluate, and the revision changes it to "One Score for All: Robust OOD Detection across Pretrained Classifiers". The revision also states what ONE requires: a linear classification head trained on the model's features, access to its weights and to the penultimate features, and a set of ID calibration images. The labels of these images are not necessary: with the classifier's predictions in place of the labels and κ_l fixed to 1, ONE's nine-setting mean AUROC is 86.8 (86.9 with labels), and it keeps the best mean AUROC and mean regret among the 25 scores (details in our response to Reviewer 4248).

Large models (Q1). We added the three models, each fine-tuned on ImageNet-1k, with 300M to 430M parameters: CLIP ViT-L/14 (timm vit_large_patch14_clip_224.openai_ft_in1k), SigLIP SO400M (vit_so400m_patch14_siglip_378.webli_ft_in1k) and EVA02-L (eva02_large_patch14_448.mim_m38m_ft_in1k). All 25 scores use the same frozen features. ONE's regret, with the full training set or k labeled samples per class as calibration:

ModelTop-1Fullk = 1k = 2k = 10k = 50
CLIP ViT-L/1487.80.12.10.70.0(1st)0.0(1st)
SigLIP SO400M89.70.0(1st)0.50.0(1st)0.0(1st)0.0(1st)
EVA02-L89.70.52.51.00.70.6

From two samples per class onward, ONE is within 1.0 AUROC of the best of the 25 scores on all three models. With the three models added to the 30-checkpoint panel, ONE remains first in mean AUROC (86.3; HEAT 84.3), in mean and maximum regret (1.4 and 6.7; next, HEAT 3.4 and fDBD 7.9), and in mean FPR95 (53.0; COMBOOD 56.2).

2. Best on few checkpoints (W2)

ONE is not designed to be the best score in a particular setting, but to stay close to the best in every setting. A post-hoc detector is attached to a classifier whose architecture and training recipe, as well as the OOD data it will meet, are unknown when the score is chosen, so the score cannot be selected for each model; the revision states this meaning of "one score for all" explicitly (Section 1). Across all evaluated classifiers, including the 30-checkpoint panel, ONE has the smallest maximum regret (6.7 AUROC) and is within 1 AUROC of the best on 22 classifiers, more than any other score (Appendix C, Figure 5). Even so, ONE is also the best score more often than any other (on 9 classifiers, against 6 for ActSub and COMBOOD), including three transformers (both Swin-B checkpoints and SigLIP).

New checkpoints. The three larger models in our answer to Q1 were not part of the submission and differ from the panel in scale and pretraining; on each, ONE stays within 0.6 AUROC of the best of the 25 scores and is the best on SigLIP SO400M.

3. Components and related work (W3)

Class-wise combination. As you note, on the nine settings the gain over scalar fusion is small (86.69 to 86.89). Its effect is on the checkpoints where the scores disagree most: on the 30-checkpoint panel it raises mean AUROC from 84.6 to 85.6 and lowers the max regret from 9.0 to 6.7. With scalar fusion the max regret would exceed fDBD's 7.9, so the class-wise combination is what gives ONE the smallest max regret among the 25 scores.

Classifier-informed centers and small budgets. That the centers do not change with plenty of data is expected: the class means are then well estimated, and the weight of the prior vanishes. The prior is meant for the opposite case, which we consider the one that matters most in deployment. When the training data of a released classifier cannot be used, for example because they are proprietary, the user must collect calibration images for each ID class. The number needed then sets the cost of deploying the detector. With one image per class, the classifier prior raises ONE's nine-setting mean AUROC from 83.6 (sample means) to 84.6, 2.3 below its value with the full training sets. We compared against GEN as the strongest score that uses no calibration data. You are right that NCI reaches the same mean at this budget (84.5), but not the same reliability: its max regret is 6.1 (ResNet-50), against 2.9 for ONE.

Xiong et al. (ICCV 2025). Thank you for highlighting this related work; the revision cites it in Section 2. It traces the lower OOD performance of the newer torchvision recipes (V2) to augmentations such as label smoothing and mixup, and recovers it by changing the network: fine-tuning the last layer without these augmentations, or retraining with a revised recipe. ONE addresses the complementary setting in which the network is used as released: no weight is changed, and the score adapts to each model through two scales fitted on the ID calibration images, starting from one image per class.

AuthorsOpenReview ↗

Comment 2 of 2

4. Reproducibility (Q2)

We confirmed the ViT normalization error, corrected it, and reran all 25 scores on the affected checkpoints. We also quantified how the fitted scales and AUROC change with optimization length. The revised results include both the preprocessing correction and the covariance update (General response, point 4).

(a) ViT preprocessing. Three timm ViT-B/16 checkpoints expect mean = std = 0.5 (augreg2_in21k_ft_in1k, our ViT-B benchmark setting, and augreg_in1k and sam_in1k on the panel), and we had applied the ImageNet statistics to them. With the correct normalization, their top-1 accuracy on our ID test set rises from 80.7 to 85.1, from 75.0 to 77.5 and from 73.5 to 78.8. Every table and figure of the revised PDF uses the corrected features. ONE remains first among the 25 scores in mean AUROC and in mean and maximum regret, on both the nine settings and the 30-checkpoint panel:

ONE (rank among 25)Nine settings, submittedNine settings, revised30-checkpoint panel, submitted30-checkpoint panel, revised
Mean AUROC ↑86.3 (1st)86.9 (1st)85.1 (1st)85.6 (1st)
Mean regret ↓1.9 (1st)1.8 (1st)1.7 (1st)1.6 (1st)
Max regret ↓4.5 (1st)4.4 (1st)6.8 (1st)6.7 (1st)

(b) Accuracy and fitted scales. The table lists the top-1 accuracy on our ID test set and the two fitted scales for the nine benchmark settings; Table 10 of the revised appendix gives them for every checkpoint, including the 30 panel models, the eight controlled ResNet-50s and the three larger models of Q1.

SettingTop-1κ_lκ_m
CIFAR-10, ResNet-1894.71.660.0222
CIFAR-100, ResNet-1877.31.880.0330
CIFAR-100, WRN76.31.980.0703
ImageNet, ResNet-5074.41.090.0122
ImageNet, ViT-B85.11.370.0067
ImageNet, Swin-B84.81.340.0063
ImageNet, ConvNeXt-B83.91.270.0022
ImageNet, DeiT-B81.71.460.0088
ImageNet, EfficientNet-B076.21.440.0144

Across all checkpoints, κ_l lies between 0.96 and 1.98 and κ_m between 0.0021 and 0.070.

(c) Fitted scales on CIFAR-10. When the calibration set is classified perfectly (100.00% for the CIFAR-10 ResNet-18), the label likelihood keeps increasing with the scales, so the maximum-likelihood estimate is unbounded and the fitted value is set by our fixed budget of 300 Adam steps. With longer runs both scales grow (κ_l from 1.66 at 300 steps to 3.32 at 3,000 and 16.1 at 10,000), while AUROC is 92.48 up to 3,000 steps and 92.22 at 10,000, consistent with your observation. When some training images are misclassified the estimate is finite: on CIFAR-100 (99.97% and 98.35% training accuracy) the scales converge and AUROC changes by at most 0.02. Appendix B of the revision states the 300-step budget and reports this analysis.

Thank you again for the detailed feedback and for the effort of running our code. We have updated the supplementary code with the two corrections described in our response: input normalization fixes for the three ViT-B/16 checkpoints, and the covariance estimated around the class centers. It also includes the three larger models (CLIP ViT-L/14, SigLIP SO400M, and EVA02-L). If our responses have addressed your concerns, we would appreciate it if you would consider updating your assessment, and we would be glad to answer any remaining questions before the discussion closes.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We thank all reviewers for their constructive feedback and for recognizing the relevance of the problem (nJY3, 4248), the breadth of the evaluation (all reviewers), and the class-wise combination and classifier-derived centers (rt4Q, nJY3, 4248, b1zN). We address the common points here; the revisions described in our responses are in the revised PDF, marked in red, and each response points to the section where they appear.

1. One score for all. In deployment, the pretrained classifier and the OOD data are not known in advance, and labeled ID images must be collected for each class for calibration. No existing score is safe across classifiers: on our 30-checkpoint panel, 11 different baselines are each the best on some checkpoint, and each falls at least 7.9 AUROC behind the best on another. ONE aims to stay close to the best on every checkpoint, which we measure by regret, the AUROC gap to the best of the 25 scores (Eq. 11).

2. Main results. On both the nine settings and the 30-checkpoint panel, ONE has the smallest mean and maximum regret (nine settings: 1.8 and 4.4; panel: 1.6 and 6.7) and the highest mean AUROC (86.9 and 85.6; Table 2). On three larger models, it stays within 0.6 AUROC of the best score. Over all evaluated classifiers, it has the lowest mean FPR95 (53.2; next: COMBOOD 55.2), and paired 95% confidence intervals of its AUROC gain lie above zero against each strong baseline (Appendix C, Table 13).

3. Small calibration budgets. We emphasize this regime because, when the training data of a released classifier cannot be used or are limited, for example because they are proprietary, the number of ID images collected per class sets the cost of deploying the detector. Nine-setting mean AUROC (Table 6, now in the main text):

Images per class12520Full
ONE84.685.386.086.686.9
Best budget-sensitive baseline79.2 (HEAT)82.6 (CTM)84.7 (CTM)85.9 (HEAT)86.3 (HEAT)
Budget-insensitive (NCI)84.584.484.584.484.4
Budget-independent (GEN)83.983.983.983.983.9

4. Revision.

  • Title "…across Pretrained Classifiers"; ONE's requirements stated (Sections 1–2); related work on variation across models (Section 2).
  • Method overview, Algorithm 1 and variable definitions (Section 4, Appendix B).
  • FPR95 (Table 3) and calibration budgets (Table 6) in the main text; three larger models (Section 5.1, Appendix C).
  • New appendices: statistical reliability, accuracy and fitted scales, the correct-classification filter, label-free calibration.
  • Covariance estimate: following the principle stated in Section 4.3, the covariance is now estimated around the class centers that the distance uses, rather than around the sample means. All numbers use this estimate and the corrected ViT preprocessing found by Reviewer rt4Q.

We thank all reviewers for their constructive feedback. During the rebuttal, we added results on three larger pretrained models (CLIP ViT-L/14, SigLIP SO400M, and EVA02-L), statistical tests, and a label-free variant, moved FPR95 and the calibration-budget results into the main text, and clarified the method description. With these revisions, ONE keeps the smallest mean and maximum regret across classifiers and the highest mean AUROC at every calibration budget, from one image per class to full calibration. We are grateful to the reviewers and the Area Chair for their time and the constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.