acceptodds
Under review as a conference paper at ICLR 2027

TASTE: Task-Aware Out-of-Distribution Detection via Stein Operators

Abstract

Out-of-distribution detection methods are often either data-centric, detecting deviations from the training input distribution irrespective of their effect on a trained model, or model-centric, relying on classifier outputs without explicit reference to data geometry. We propose TASTE (Task-Aware STEin operators): a task-aware framework which links distribution shift to the input sensitivity of the model. The resulting operators admit a geometric interpretation as a projection of distribution shift onto the sensitivity field of the model, yielding theoretical guarantees. Beyond detecting a shift, the construction enables its localisation through a coordinate-wise decomposition, and - for image data - provides interpretable per-pixel diagnostics. Experiments on controlled Gaussian shifts, MNIST under geometric perturbations, and CIFAR-10/ImageNet-1K perturbed benchmarks demonstrate that TASTE aligns closely with task degradation while showing competitive performance compared to established baselines, with gains that depend on the shift regime.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Out-of-distribution detection, Task-aware anomaly detection, Stein operators, model-agnostic, adversarial attack detection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CzGgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TASTE, which constructs a task-aware out-of-distribution (OOD) signal by incorporating the score of the training distribution—along with the input gradients of a fixed predictor—into a Langevin Stein operator. This approach ensures that detection focuses on distribution shifts that impact model performance, rather than on all statistical variations. Furthermore, the authors use the training set to calibrate score errors and extend this signal to both sample-level detection and pixel-level localization.

Critical strengths
  1. The proposed methodology offers some novelty. TASTE employs a fixed predictor as a test function for a Stein operator, integrating the score of the training distribution with the model’s sensitivity to its input. It further introduces a projection identity, a score-error correction mechanism, and both sample-wise and pixel-wise residuals.

  2. The experiments are comprehensive and provide meaningful insights. The paper evaluates the approach across a diverse range of datasets, including Gaussian, MNIST, CIFAR-10, ImageNet-1K, and MVTec AD—assessing its out-of-distribution (OOD) detection performance while demonstrating its ability to capture the directionality of task-relevant shifts and perform pixel-level localization.

Critical weaknesses
  1. The main methodological idea is to combine the score of the training distribution with the input sensitivity of a fixed predictor, with the full operator additionally using the predictor’s Laplacian. This is a meaningful and potentially novel construction. However, the broader goal of task-aware OOD detection has been explored in prior work, including Task-Driven OOD Detection, Sequential Harmful Shift Detection Without Labels, and Towards a Certificate of Trust. The paper should therefore clarify which aspects are genuinely new beyond this existing line of work. The experimental section includes several general-purpose OOD baselines, but the discussion and evaluation of the closest task-aware and joint input/output likelihood methods remain limited.

  2. The multiclass experiments use the predicted-class softmax as the default probe. Although a pre-softmax logit probe appears in one auxiliary experiment, the paper does not explain how sensitive the method is to this choice. A brief comparison with a logit probe, or a clearer justification for using the softmax probe, would make the methodology easier to assess. The different variants should also be separated more clearly in the main results. Full TASTE uses the complete Langevin Stein operator; the no-Laplacian version is a cheaper first-order variant that still depends on the predictor gradient; score-only removes this predictor-sensitivity term; and TASTEext uses a classwise first-order construction for ImageNet. These variants have different interpretations and should be labeled consistently in the tables and captions.

  3. The paper uses signed residuals in some experiments and ∣r(x)∣|r(x)| in others. It would help to state explicitly how score direction, aggregation, and thresholding are defined for AUROC/FPR95, dataset-level shift detection, and pointwise OOD testing. A compact implementation table would also improve reproducibility by collecting the diffusion timestep, noise sampling, Hutchinson settings, calibration split, random seeds, repetitions, preprocessing, feature layers, and baseline hyperparameter choices. Most of this information may already be present in the appendix; the main need is to make it easier to find. As a minor optional analysis, the authors could report rank correlations between TASTE residuals and task performance in the controlled Gaussian or MNIST experiments.

Questions for authors

See the “Critical Weaknesses” above for details.

Reviewer_eWoSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TASTE which is an out-of-distribution detection algorithm that utilizes both model and data for detection. Specifically, they apply the Langevin Stein operator to a fixed model, and derive a projection identity that links distribution shift to the sensitivity field of the model. Empirical results show that TASTE achieves strong performance against standard OOD baselines.

Critical strengths

This paper studies an well-motivated problem of OOD detection problem and it clearly describes its contribution and related work in this space.

Authors provide theoretical justification for their approach. Further, the directional-shift experiment (Figure 2) is demonstrate their approach clearly.

The method needs no retraining of the classifier and no OOD or negative samples.

Critical weaknesses

My main question is about the note of authors on model-centric stating that they do "not explicitly connect fθ to the geometry of the input distribution". Why does an OOD detection method needs the geometry of input data and cannot just rely on the outputs? Is that because of the high confidence of model when wrongly predicting classes?

One of the main claim of the paper is that TASTE responds to harmful shifts and ignores benign ones. This has only been verified in Toy setups and MNIST. Is there any way to confirm this in more realistic setups? For example, in the MNIST setup you have used one model with two different shifts. What if we train two models on the same data but during the training of one model, we rotate the samples (augment). Then, we pass the same rotated data to both models.

Questions for authors

Please see weaknesses above.

Reviewer_HEHmOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a task-aware OOD detection method. It combines the score of the training distribution with the input derivatives of a deployed predictor through a Stein operator. The method also provides coordinate-wise scores for local diagnosis. Experiments cover several shift types and show competitive results in some settings, although performance varies across scenarios.

Critical strengths
  1. The paper provides a useful perspective. Distribution changes should be considered together with the sensitivity of the deployed predictor. The connection between distribution geometry and model sensitivity offers insight for further research.

  2. The Stein formulation gives a clear mathematical interpretation of the proposed statistic. Its coordinate-wise decomposition also supports local diagnosis beyond a single detection score.

Critical weaknesses
  1. The current theory supports predictor-aware sensitivity, but does not directly establish harmful task-shift detection. A change in model output does not necessarily imply an increase in prediction error. The paper should clarify this distinction and provide more direct evidence connecting the proposed score to task performance degradation.

  2. Some reported results need clarification. Are the classical OOD and adversarial AUROC values for TASTE in Table 1 calculated from the results in Tables 7 and 4, respectively? The reported averages do not appear to match. Also, ODIN’s Overall FPR95 in Table 1 is 0.9486, which is higher than all four group averages. This is not possible under the stated averaging procedure. Please check the results and explain the aggregation.

  3. The absolute performance limits the deployment claims. On CIFAR-10, the Overall AUROC is about 0.63 and FPR95 is about 0.82. On MVTec, the overall image AUROC is about 0.65, and localisation on object categories is relatively weak. These results support competitive performance in some settings, but are insufficient to establish reliable deployment. MVTec also lacks quantitative comparisons with anomaly detection methods under the same setting, so it mainly provides evidence of feasibility.

  4. The Overall results should be interpreted with respect to the benchmark composition. The ImageNet evaluation contains more adversarial attack settings, giving this regime greater weight in the Overall average. The authors appropriately acknowledge that no method performs best in every regime. However, on classical OOD, ViM achieves an AUROC of 0.9428, compared with 0.8485 for TASTE_ext. The best Overall result under this aggregation should therefore not be interpreted as a general advantage across scenarios.

Questions for authors

Please see the corresponding Critical Weaknesses for details.

  1. What evidence connects the proposed score to task performance degradation?

  2. Could you reconcile the results in Table 1 with the appendix and clarify the averaging procedure?

  3. Could you provide MVTec comparisons under the same evaluation setting?

  4. How would the ImageNet Overall ranking change if the two shift regimes received equal weight?

4 comments

Q1. Label-preserving input shifts provide a natural setting for connecting predictor sensitivity to task performance. Sensor noise, blur, compression artefacts, illumination changes and class-preserving rotations or translations replace (X,Y)(X,Y) by (T(X),Y)(T(X),Y). Adversarial perturbations that preserve the true label fit the same setting: a successful attack on an initially correct prediction creates an error, potentially with high confidence. This makes adversarial examples a concrete motivation for relating input geometry to model sensitivity.

Sensitivity alone, however, does not imply degradation: confidence can change without changing the decision, and a class change can correct an existing error. Additionally, a changed labelling rule can increase error without changing the input distribution. We will make this distinction explicit in the revised submission.

Viewed as a hypothesis test, TASTE asks whether a new image is consistent with the reference distribution (Section 4). For a new image drawn from that distribution, rank-based thresholds calibrated on held-out images control the false-positive rate. Under a shift, the mean TASTE score is the expected inner product of the density-score difference and the model gradient (Eq. (5)). This explains why the same shift can produce different mean residuals across classifiers. We test whether this sensitivity tracks task degradation by asking whether TASTE AUROC falls when augmentation reduces classification error on the same corrupted images.

We add an empirical test using label-preserving MNIST rotations and CIFAR-10 Gaussian noise, with robustness varied through training augmentation. For each dataset, we train three standard/augmented classifier pairs. Within each pair, classifiers share the initialization and training data order; the augmented arm additionally receives rotations sampled uniformly from −30∘,30∘-30^\circ,30^\circ for MNIST or Gaussian noise for CIFAR-10. We evaluate six rotation angles and five noise severities, giving 18 rotation and 15 noise seed-condition comparisons. We also evaluate translations of 4, 8, 12 and 16 pixels on padded MNIST images as a control. Both arms are evaluated on the same 2,000 clean images and their transformed counterparts. We measure excess classification error (shifted minus clean classification error) and TASTE AUROC for distinguishing shifted from clean images. This tests whether TASTE responds less strongly when the same shift becomes less harmful to the classifier. The table reports the −30∘-30^\circ rotation, 16-pixel translation and noise-severity-5 results (mean ±\pm sample SD across three training seeds).

ShiftExcess error (pp), standard → augmentedTASTE AUROC, standard → augmented
MNIST, −30° rotation25.80 ± 2.17 → 3.27 ± 1.060.755 ± 0.011 → 0.565 ± 0.020
MNIST, 16-pixel translation−0.03 ± 0.06 → 0.23 ± 0.060.513 ± 0.009 → 0.518 ± 0.006
CIFAR-10, Gaussian noise (severity 5)15.65 ± 0.97 → 2.85 ± 0.260.678 ± 0.010 → 0.585 ± 0.007

Excess error and TASTE AUROC fell together at every rotation angle and noise severity for all three seeds. With the images and density-score estimator held fixed, TASTE separated the same shifts less strongly after the classifier became more robust to them. For the 16-pixel translation control, excess error was close to zero and AUROC was near chance for both classifiers.

Q3. MVTec demonstrates zero-shot localisation: neither the classifier nor the score model was trained or fine-tuned on MVTec data. We have not evaluated specialised anomaly detectors under the same zero-shot conditions and will present these results as a supplementary feasibility study.

Tables 1 and 2 show that performance varies across shifts for every detector evaluated. TASTE leads on both AUROC and FPR95 overall and on adversarial attacks on CIFAR-10. For classical OOD, Energy leads on both metrics on CIFAR-10 and ViM on ImageNet. Even the lowest overall FPR95 on CIFAR-10 corresponds to frequent false alarms on clean inputs. We will revise the phrase “well suited to post-deployment monitoring” in Section 6. TASTE complements existing detectors, particularly for adversarial inputs. Its use for monitoring would require validation on the shifts of interest at a false-alarm rate the application can tolerate.

Q4. Equally weighted ImageNet results.

Giving the classical and adversarial regimes equal weight, using the results from Table 2, changes the leading results as follows:

MethodAUROC / rankFPR95 / rank
TASTEextTASTE_{ext}0.81130 / 10.51165 / 2
Mahalanobis0.80245 / 20.46005 / 1
kNN+0.76370 / 30.52950 / 3

Under this alternative weighting, TASTEextTASTE_{ext} remains first by AUROC and moves to second by FPR95, behind Mahalanobis. On classical OOD, ViM achieves 0.9428 AUROC versus 0.8485 for TASTEextTASTE_{ext}. We will emphasise that these results should not be interpreted as a regime-independent advantage.

Q2. First, we clarify the averaging procedure. Each group column averages its constituent datasets or shift settings. Overall gives equal weight to each setting in the table. Let mim_i denote the value for setting ii, mˉg\bar{m}_g the mean for group gg, ngn_g its number of settings, and N=∑gngN = \sum_g n_g the total number of settings. Then

mˉOverall=1N∑imi=∑gngNmˉg.\bar{m}_{Overall} = \frac{1}{N}\sum_i m_i = \sum_g \frac{n_g}{N}\bar{m}_g.

Each group therefore contributes to Overall in proportion to its number of settings. Giving the groups equal weight would give individual settings in smaller groups more influence on the result.

Thank you for flagging the inconsistencies in Table 1. When we assembled the table, we included some values from earlier research runs alongside those from the final evaluation.

We have regenerated Table 1 using the final procedure described in the paper and include it below. Seventeen of the 20 Overall values match the submitted table to four decimal places, including both values for TASTE. We will use the same procedure to update Tables 4–7 in the revised manuscript.

Table 1. Each cell reports AUROC / FPR95.

MethodAdversarialCIFAR-10-CCIFAR-10-PClassical OODOverall
MSP0.3633 / 0.96140.6131 / 0.87230.5982 / 0.86640.8033 / 0.52570.5751 ± 0.0081 / 0.8521 ± 0.0161
ODIN0.5141 / 1.00000.5679 / 0.98620.5663 / 1.00000.7664 / 0.59000.5776 ± 0.0084 / 0.9486 ± 0.0070
Mahalanobis0.5822 / 0.93360.5880 / 0.90000.5620 / 0.89320.8189 / 0.54590.6060 ± 0.0084 / 0.8664 ± 0.0116
Energy0.3551 / 0.95860.6256 / 0.87020.6111 / 0.85800.8520 / 0.46710.5871 ± 0.0081 / 0.8421 ± 0.0125
kNN+0.4646 / 0.93920.6152 / 0.87960.5955 / 0.86680.8291 / 0.54570.6007 ± 0.0083 / 0.8526 ± 0.0123
ReAct0.5140 / 0.96200.5405 / 0.91830.5184 / 0.92180.7079 / 0.65220.5478 ± 0.0087 / 0.8993 ± 0.0091
ViM0.5497 / 0.95400.6045 / 0.88070.5715 / 0.87620.8175 / 0.51450.6079 ± 0.0084 / 0.8552 ± 0.0111
ASH0.3614 / 0.95480.6250 / 0.87500.6122 / 0.86310.8498 / 0.48030.5882 ± 0.0081 / 0.8459 ± 0.0126
PRO0.3523 / 0.96320.6047 / 0.89610.5842 / 0.89100.7160 / 0.65310.5560 ± 0.0082 / 0.8827 ± 0.0160
TASTE0.6274 / 0.81350.6172 / 0.86840.5956 / 0.86070.7600 / 0.56610.6301 ± 0.0082 / 0.8207 ± 0.0127

Our conclusions remain unchanged: TASTE achieves strongest performance in the AA setting and overall, with strong FPR95 across corruptions and perturbations.

Thank you for the response and additional experiments. The revised Tab. 1 address my main concern about the reported numbers. Please ensure that Tab. 4–7 will be updated consistently.

AuthorsOpenReview ↗

The tables are updated in the uploaded revision

Thank you for the reminder. In the uploaded revised version, the tables are updated. As we added three tables before these appear (Table 1 in the main text on task score degradation, Table 5 in the appendix on probe choice, and Table 6 in the appendix on equal weighting results), they are now Tables 7-10, on pages 29-31. All changes from the original submitted paper are marked in blue.

Comments on the paper

2 comments

We have uploaded a revised manuscript incorporating the reviewers’ feedback and the clarifications, additional experiments and corrections discussed in our responses. The changes from the original paper are in blue.

The key changes are

P.1 rephrased the second paragraph to avoid the impression that geometry would be necessary for OOD

P.2 added a paragraph on related work regarding task-aware OOD

P.3 clarified the novelty of TASTE

P.8 added an experiment in which TASTE can be seen to respond less strongly to less harmful shifts

P.8 clarified the averaging procedure for Tables 2 and 3

P.9 updated Table 2 (the former Table 1)

P.10 clarified that MVTec is a supplementary feasibility study

P.22 augmented Table 4 with more details of the scoring and aggregation protocols

P.26 added Table 5, addressing probe choice

P.25 added results on equal weighing of the classical and adversarial regimes, with Table 6 on p.27 showing the results

P.28 added details for the settings used in Table 2

P.29-31 Tables 7-10 are the updated Tables 4-7

P.35 added Table 11 with the full results for the new experiment on task degradation in Section 5.

We would like to thank all reviewers for the comments and suggestions which in our view have led to a substantial improvement of the paper, We hope that all of you will now be able to recommend acceptance,

We have uploaded a further revision of the manuscript. Table 3 (ImageNet-1K results) now includes a comparison with an SHSD baseline, following a request for comparisons with closely related task-aware methods.

Thank you for your time and engagement throughout the rebuttal discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.