acceptodds
Under review as a conference paper at ICLR 2027

From Flow Matching to Type I Error Guarantees in Anomaly Detection

Abstract

Anomaly detection asks whether a new observation is likely to be generated by a reference distribution that characterizes normal behavior. We address this problem by learning a transport, via flow matching, that maps the reference distribution to a standard Gaussian. Once trained, this map lets us test a new sample by checking how far its transported representation lies from the Gaussian: under the reference distribution, its squared norm follows a chi-squared statistic, yielding a threshold with an explicit type 1 error rate. We further derive upper bounds that link the training loss of the underlying model to the resulting detection guarantees, showing that a well-trained model yields a test whose behavior remains close to this ideal. These results are made explicit in two cases, Gaussian and spherically symmetric reference distributions, and validated numerically. We evaluate the resulting detector on three distinct applications: detecting images that do not belong to a reference set, detecting targets in radar signals against background clutter, and classifying imbalanced biological data. Across these settings, our method is competitive against existing detectors.

Primary area:
generative models
Keywords:
flow matching, anomaly detection, type 1 error, transport

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hS3KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an anomaly detection framework built on conditional flow matching. The core idea is to learn an invertible transport map that projects the reference distribution onto a standard Gaussian distribution. Under an ideal transport, the squared norm of transformed normal samples follows a chi-squared distribution, which enables analytically calibrated thresholds with explicit Type I error guarantees.

Critical strengths
  1. This work establishes a rigorous theoretical chain from model training loss to statistical detection guarantees, which is a contribution to deep learning-based anomaly detection.
  2. The paper creatively combines flow matching with classical statistical hypothesis testing for anomaly detection.
Critical weaknesses
  1. Theoretical bounds lack tightness and are misaligned with the practical pipeline. The derived upper bounds include the exponential Lipschitz constant, which can be extremely large in high-dimensional settings such as images. This makes the bounds too loose for quantitative practical guidance, and they function more as existence guarantees than usable error estimates.
  2. The paper acknowledges that independent coupling introduces an irreducible variance D that prevents loss from converging to zero, and proposes Reflow to mitigate this issue. However, the theoretical bounds are not extended to the Reflow setting, creating a gap between the theoretical analysis and the actual algorithm used in experiments.
  3. Limited competitiveness on mainstream vision OOD benchmarks. While the method ranks first among methods trained exclusively on in-distribution data, it lags significantly behind pretrained-backbone methods on image OOD tasks. Although the authors correctly note that pretrained methods do not transfer to non-image domains, the paper omits comparisons with more recent non-pretrained flow matching/diffusion OOD methods, making it hard to assess the relative advantage of this framework within the same method category.
  4. The paper's central selling point is explicit Type I error control. However, empirical validation of false alarm alignment is only conducted on synthetic K-distributed data. No corresponding experiments verify how closely the actual false alarm rate matches the nominal level when using the analytic chi-squared threshold on image or biological data. This weakens the theory-practice connection for real-world high-dimensional applications.
  5. The related work section only briefly mentions a small number of flow matching detection works. No direct experimental comparison is performed with these methods, so it remains unclear what specific advantages the proposed calibrated-test framework offers over other flow-based anomaly detection approaches.
Questions for authors

Please see Weaknesses.

2 comments

We thank the reviewer for the careful and constructive review. Tables R1–R8 are in the common comment.

W1: the theoretical bounds lack tightness, with an exponential Lipschitz constant.

We agree that the exponential Lipschitz constant is a worst case. In the revision, instead of bounding how fast two trajectories can drift apart anywhere in space (via Grönwall), we write the error exactly as the velocity error along the true trajectory, carried back to t=0t=0 by the Jacobian of the learned flow (the classical Alekseev–Gröbner formula). The constant is then the size of this Jacobian where the data lie, which we measure by automatic differentiation. The bound, however, can be evaluated numerically only when the true transport is known (Gaussian and spherical distributions), since LFM\mathcal{L}_{\rm FM} requires the true velocity field.

On Gaussian references (d=4,8d=4,8), the bound of Lemma 1 is within a factor of 9 to 12 of the measured transport error (Table R8). We have no such evidence in high dimension: the bound cannot be evaluated on images. The bounds have a supporting role and are not meant to certify a level numerically; the revised title, introduction and conclusion no longer speak of guarantees.

W2: the theoretical bounds are not extended to the Reflow setting.

We now state explicitly that Lemma 1 controls the transport towards the distribution a model is trained on: PH0P_{\mathcal H_0} for the first round, the samples of the previous model afterwards. A bound with respect to PH0P_{\mathcal H_0} across rounds remains open. Proposition 2, however, depends only on the transport error of the deployed map, so it also applies to reflowed models.

Besides, Reflow is not needed for the bound to vanish. DD does not depend on the model, and our proof already bounded the transport error by the excess loss LFM=LCFM−D\mathcal L_{\rm FM}=\mathcal L_{\rm CFM}-D (Eq. 26). Only the statement used LCFM\mathcal L_{\rm CFM}, because it can be computed, which is why the bound had a floor. We made two changes:

  • Lemma 1 is stated with LFM\mathcal{L}_{\rm FM}: the bound now vanishes for a perfect model, without Reflow.
  • The effect of Reflow is quantified: for ideal Reflow, the gaps of successive rounds are summable, ∑k=0RD(k)≤E∥x1−x0∥2\sum_{k=0}^{R} D^{(k)} \le \mathbb{E}\lVert x_1-x_0\rVert^2 (as in Liu et al., 2023), so Reflow tightens the computable bound.

Empirically, Reflow is not uniformly beneficial: at a 1% type-I error with SVHN as ID, the model without Reflow detects more (Table R5).

W3, W5: limited competitiveness on vision OOD benchmarks; no comparison with recent flow matching/diffusion detection methods.

The revision removes the "Avg" column and presents this scenario as a limitation, whose cause is discussed in our response to Reviewer uus6 (W2): our statistic is the flow's likelihood without its volume term and partly inherits the failure of likelihood-based detection (Nalisnick et al., 2019). The detection requires the shift of ∣∣z∣∣2||z||^2 between the 2 hypotheses to exceed the spread of its χ2\chi^2 distribution (2d\sqrt{2d}, d=3072d=3072 for image OOD scenario); otherwise the score distributions overlap. Future work can mitigate this limitation by feature extraction: in preliminary results, Table R1, our method ranks first or tied for first as well from Table R4 in three of the six OOD scenarios among pretrained-backbone methods

Regarding methods of the same category, Table 1 already included DDPM-OOD (Graham et al., 2023), a diffusion model trained on ID data only; we could not add further non-pretrained image baselines during the rebuttal. We added TCCM (Li et al., 2025), a flow-matching baseline designed for anomaly detection, on ADBench, where our method obtains the best AUROC on six of seven datasets (Tables R2 and R7). The other flow matching works address different problems: failure detection for robot policies (Sun et al., 2026; Xu et al., 2025) and generalization of conditional flow matching to unseen conditions (Pham, 2026). To our knowledge, none of these works provides a test statistic with a known distribution under H0\mathcal H_0, nor do they address radar detection.

W4: false alarm alignment is only validated on synthetic data, not on image or biological data.

We concede this point: the agreement between the realised and nominal false-alarm rates is only verified on synthetic data. As stated in our response to Reviewer uus6 (Q2), the analytic threshold is not yet reliable on images, and we have not evaluated it on the biological data. However, first, K-distributed clutter is a standard model of heavy-tailed clutter in radar detection. Second, the study added for Reviewer uus6 (Table R6) shows when this threshold is useful in practice: at the false-alarm rates used in radar (α=10−5\alpha=10^{-5}), an empirical threshold needs about 10710^{7} samples, and none can even be computed from our 5×1045\times10^{4} training samples, whereas the analytic threshold gives a realised rate on Gaussian data (d=4,8d=4,8).

Thanks for your reponnse.

Reviewer_YpuXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper introduced a new flow-matching based method for semi-supervised anomaly detection (i.e., only normal points are in the training set). The key idea is to map the "normal data distribution" to the Gaussian via a trained flow-matching model, and when a new test point comes in and that point is really from the normal data distribution, then the norm after mapping it back to gaussian will follow a chi-square distribution, and hence a type-I error guarantee can be derived.

Critical strengths

I like the theoretical part, especailly Lemma 1, which derives the connection between the flow matching training loss and the test statistic.

The paper is very well-written and easy to follow.

Critical weaknesses

The empirical evaluation does not support that the proposed method can work well in practice.

  • In table 1 where CIFAR-10 (ID), SVHN (ID) are used as training set, and the task is to tell if the algorithm can detect points from another image dataset that is not used in training. Six (train, test) pairs were reported there, (CIFAR, SVHN), (CIFAR, Celeb), etc. Over these 6 folds, the authors report that the proposed method achieved highest average ROC-AUC, yet that result is very fragile if we look inside the table. Specifically, the proposed method has

0.503 0.695 0.511 0.897 0.993 0.963, while the best baseline in each column achieves

0.830 0.928 0.620 0.990 0.999 0.992;

that is, the prposed method is never the best, and the ROC-AUC of the proposed method is at least 0.1 smaller than the best performing baselines in 4 out 6 pairs.

  • Table 2 report experiments on a single biomedical dataset from a KDD Cup. The experiment seems not to be the most suitable and relevant one here as most baselines seem to be supervised learning methods (yet the paper is about semi-supervised anomaly detection).

As table 1 and 2 are the main results in the paper, I am not fully convinced that they demonstrated strong empirical performance of the proposed method.

  • Flow-Matching baselines like Li (2025) were discussed in the related work but was not included in the experiment section.

  • A critical ablation is missing: as flow matching can give you likelihood, the author need to justify their proposed anomaly score (test statistic) by comparing it against the likelihood from the same trained flow matching models.

Questions for authors
  • How is the proposed method applied to the bioinformatics data?
  • Why does the averages repored in Table 12 not match the corresponding cell in table 1 (e.g., 0.945 ± 0.015 over seeds for SVHN → CIFAR-100, while Table 1 reports 0.963)?
5 comments

We thank the reviewer for the careful and constructive review. Tables R1–R8 are in the common comment

W1: the empirical evaluation does not support that the method can work well in practice.

We agree with this reading of Table 1. We removed the "Avg" column and now present image OOD detection as a limitation (details in reviewer uus6 response to W2). The ablation you requested in W4 identifies its cause: our statistic is the flow's likelihood without its volume term, so it partly inherits the failure of likelihood-based detection reported by Nalisnick et al. (2019). With CIFAR-10 as ID, SVHN images do not shift ∥z∥2\lVert z\rVert^2 beyond the spread of its χd2\chi^2_d distribution; with SVHN as ID, at the same dimension, AUROC reaches 0.90–0.99. Future work can mitigate this by reducing the dimension through feature extraction: in preliminary results (TableR1), our method then ranks first or tied for first

The empirical case for the method rests on 3 other settings: (i) one-class tabular data, where it obtains the best AUROC on six of seven ADBench datasets against 14 detectors, including the flow-matching baseline TCCM (TableR2); (ii) radar detection, where it is competitive with detectors that know the target steering vector (Fig.5 in the paper); (iii) small type-I errors: at α=10−5\alpha=10^{-5}, the analytic threshold remains comparable to 2α2\alpha on Gaussian data, where no empirical threshold exists with the same samples (TableR6 & details in reviewer uus6 response to Q2)

W2, W3: a single biomedical dataset with mostly supervised baselines; FM baselines like Li (2025) not included.

The baselines in Table2 are imbalanced-learning methods trained on labeled samples from both classes, whereas our method uses only majority-class (normal) samples; we included them as upper-reference comparisons. We agree that one-class detectors provide more relevant comparisons and that a single dataset is insufficient. We have extended our evaluation to seven tabular datasets from ADBench (Han et al., 2022), with 14 anomaly detection baselines (TableR2). They include TCCM (Li et al., 2025), a flow-matching baseline designed for anomaly detection. TableR7 reports results across random seeds for TCCM and our method on three of these datasets, with paired significance tests: the AUROC and AUPR differences are significant, p≤0.03p\le0.03

W4: the anomaly score should be compared against the likelihood from the same trained flow matching models.

We added this ablation in TableR3. By the change of variables, the negative log-likelihood of the flow decomposes as −log⁡pθ(x)=12S(x)+I(x)+d2log⁡2π-\log p_\theta(x)=\tfrac12 S(x)+I(x)+\tfrac d2\log 2\pi, where S(x)=∥Tθ−1(x)∥2S(x)=\lVert T_\theta^{-1}(x)\rVert^2 is our statistic, and I(x)=∫01div⁡vθ(xt,t)dt=−log⁡∣det⁡JTθ−1(x)∣I(x)=\int_0^1\operatorname{div}v_\theta(x_t,t)\mathrm{d}t=-\log\lvert\det J_{T_\theta^{-1}}(x)\rvert is the logarithm of the factor by which the learned transport changes volumes around xx. Our statistic is therefore the likelihood without this volume term

When H0\mathcal H_0 is Gaussian, the ideal transport is affine, its Jacobian does not depend on xx, and both statistics define the same test; on Gaussian clutter, their detection rates are 0.382 and 0.371. On images, the volume term has opposite effects depending on the in-distribution set: it improves detection with SVHN as ID, but degrades it with CIFAR-10 as ID (0.095 vs. 0.445 against SVHN), which is the failure of likelihood-based detection reported by Nalisnick et al. (2019). We keep SS for two reasons. First, if the transport is exact, SS follows a χd2\chi^2_d distribution under H0\mathcal H_0 whatever the data distribution, which gives an analytic threshold, whereas the distribution of II depends on the data and the model, so a threshold on the likelihood must be calibrated empirically. Second, SS does not require the divergence of vθv_\theta, whose exact computation costs dd backward passes per integration step, or a noisy Hutchinson estimate

Q1: how is the method applied to the bioinformatics data?

In KDD Cup'04(BIO) Protein dataset, each protein sample is represented as a feature vector. We train the Rectified Flow model exclusively on non-homologous protein samples (y=0). During inference, both homologous and non-homologous test samples are transformed into the latent space through the learned transport. The anomaly score is computed as the squared Euclidean norm of the latent representation. Samples with higher anomaly scores are considered more likely to be homologous (y=1)

Q2: the averages reported in Table 12 do not match the corresponding cell in Table 1.

Table1 reports a single training run (0.963 for SVHN→CIFAR-100), whereas Table12 reports the mean ± std over five seeds of the same configuration (0.945 ± 0.015): the Table1 value lies 1.2 std above the seed mean. We agree that mixing the two is confusing. In the revision, the entries of our method are reported as mean ± std wherever seeds are available, and single-run values are identified as such

I thank the authors for the detailed response.

My concerns about the original Table 2 and about the asked abalation study have been addressed, though 1) for table 2, the evidence would be stronger if more ADBench datasets are included (I think 15 - 20 is common nowadays), and 2) for the ablation, it's a pity that the proposed score did not beat the likelihood --- I thank the authors for the honest report.

Regarding Table 1: it's nice to see that ViT pre-trained features can improve the performance, but this seems a bit unfair for the baselines there as they could also benefit from the ViT pre-trained features?

Overall, the authors provided a helpful rebuttal. I think it's a boardline now, and I am slightly towards the accept: the concern is about the evidence of the overall empirical performance, but the strong points are the nice theoretical part and the newly added empirical results (especially to address my concern for the original table 2). My final score depends on the next phase AC-reviewer discussion,

We sincerely thank for the comment.

Table R1 (Table 3 of the revised version). We agree: the baselines could also benefit from the ViT features, so this table does not support a claim of superiority. It only illustrates a possible way to mitigate the limitation on images; feature extractors without pre-training could also be considered, which we could not test within the rebuttal period.

Running all methods on the same features is not straightforward within the rebuttal period: ASH and ReAct already operate on the features of a pre-trained network, while most other baselines are designed for raw images and would have to be modified to take feature vectors as input, so they would no longer be the published methods. In the final version, the caption of Table 3 will present it as an illustration rather than a comparison.

ADBench. We have now extended the comparison in Table R2 to 15 datasets, as suggested. The results on the 8 newly added datasets are provided below. Due to the response length limitations, we cannot include the complete results here; however, the full results across all 15 datasets will be included in the final version of the manuscript.

Dataset (#Features)PCAOCSVMLOFCBLOFCOFHBOSKNNSODCOPODECODDeepSVDDLODAIForestTCCMOurs
InternetAds (1555)0.7200.7060.8350.7210.4920.5030.7940.4840.6720.6730.7080.5330.4380.8580.881
letter (32)0.5100.9270.8730.7720.8240.5800.9000.7760.5500.5540.5230.4850.5860.9090.973
satimage-2 (36)0.9790.9830.9970.9980.5490.9730.9990.7200.9760.9670.9770.9850.9950.9980.999
vertebral (6)0.4050.4940.4740.4890.4690.4210.4390.3350.3680.3930.5350.3460.4220.5680.599
vowels (12)0.6700.8290.9630.9070.8420.7130.9760.7840.5120.6100.7390.7650.7940.9650.995
WBC (9)0.8940.8750.8570.8320.6740.8490.8720.7560.9050.9060.9070.8770.8760.9000.981
Wilt (5)0.2500.8210.8760.4710.5090.3860.7960.5140.3450.3870.3180.3740.4510.9430.890
wine (13)0.8831.0001.0001.0000.9830.7941.0000.7530.8580.7030.9080.9220.8330.9511.000
Reviewer_uus6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a flow-matching-based method for anomaly detection that admits an explicitly calculable type I error rate. The work is based on the observation that well-trained flow matching should map samples from the reference (training) data distribution to a unit Gaussian, and that the the squared norm of transformed samples should thus follow a chi-squared distribution. Under a perfect transport map, cutting on a transformed sample's squared norm yields a hypothesis test with a type I error rate that can be computed exactly from the chi-squared distribution. In practice, trained flow-matching algorithms are not perfect, and the authors derive bounds on the type-I error rate based on the flow-matching training loss. The authors test their method in three domains: out-of-distribution image detection, radar target detection, and imbalanced biological data classification. They observe generally competitive performance with baseline methods.

Critical strengths
  • The paper's main idea (obtaining reliable analytic bounds on type-I error for ML-based anomaly detection) is clear, well-motivated, and conceptually sound. The derived error bounds are a concrete and valuable theory contribution
  • The experimental evaluation of the proposed method is sound, even though the method does not generally outperform baselines. The case is strongest for radar detection, where controlling false alarm rates is clearly important.
Critical weaknesses
  • The theoretical bounds on type-I error are not studied in the experiments section. The bounds depend on training loss and the Lipschitz constant of the flow-matching network, which might be hard to control in practice. It would be illuminating to understand if these bounds are useful in practice or not. As it stands, there seems to be a disconnect between the theory and experiment sections of the paper.
  • The proposed method does not consistently outperform any baselines. In the absence of any advantage derived from the theoretical type-I error results, the paper does not make a strong case for using the method.
Questions for authors
  • What is the "avg" column in table 1 - the average of the whole row? This feels like an odd column to include, given that the two ID scenarios (column blocks) are trained on different datasets and very different average performance across methods. The proposed method is not best on any individual column, but is best on "average" -- this feels slightly misleading. The method performs essentially no better than chance in 2/3 of the OOD scenarios for a CIFAR-10 reference distribution
  • What purpose does having a known type-I error rate serve for the OOD image detection task? It does not seem to be discussed, and the reported AUROC values are a fairly coarse measurement of performance. It might be more interesting to e.g. fix the type-I error rate and compare different models' performance. The ability to calculate/control this rate seems to be the main selling point of the method (and indeed the radar study is a great example of a scenario where it's important to do so!)
  • How are so many AUROC values in table 1 below 0.5 (random chance)? Does this indicate a badly tuned baseline? And wouldn't simply inverting the classifier actually make many of these perform much better?
  • Have the authors evaluated their theoretical bounds on any of the trained models, and how useful are they in practice? It would be interesting to get a sense of how loose/tight they are in a realistic scenario.
  • Is there any understanding of why the method performs so poorly for the CIFAR-10 ID case?
  • Could the authors include the seed variation error bars for all methods / (ID, OOD) pairs in the appendix? Table 12 only includes a couple, and it would be nice to see where there is a statistically significant difference in performance.
3 comments

We thank the reviewer for the careful and constructive review. Tables R1–R8 are in the common comment.

W1: the theoretical bounds on type-I error are not studied in the experiments.

We agree that the bounds should not carry more weight than they can bear. Their role is a supporting one (showing that the nominal level is recovered as the excess flow matching loss vanishes, and at what rate). They are not meant to certify a level numerically. To avoid overstating them, the revised title (From Flow Matching to Anomaly Detection with Upper Bounded Type I Error), introduction and conclusion no longer speak of guarantees. We also strengthened Lemma 1: the Lipschitz constant is replaced by an effective factor (the norm of the Jacobian of the learned flow, which can be estimated by automatic differentiation). The bound is now evaluated numerically (Table R8).

W2, Q1, Q5: the method does not consistently outperform baselines, is best only on "average", and performs poorly for the CIFAR-10 ID case.

We agree and have removed the "Avg" column. The image OOD scenario is now presented as a limitation. Detection requires the shift of ∥z∥2\lVert z\rVert^2 between the two hypotheses to exceed the spread of its χd2\chi^2_d distribution (of order 2d\sqrt{2d}, about 78 for d=3072d=3072); otherwise the score distributions overlap. With CIFAR-10 as ID, this shift is absent for SVHN. This is the failure of likelihood-based detection reported by Nalisnick et al. (2019): generative models trained on CIFAR-10 assign a higher likelihood to SVHN than to CIFAR-10, which Serrà et al. (2020) attribute to image complexity. Our statistic is the flow's likelihood without its volume term (see our response to Reviewer YpuX), so it partly inherits this failure: removing this term reduces it but does not remove it (0.095 → 0.445, Table R3).

Future work can mitigate this limitation by feature extraction, which reduces the dimension and encodes semantic content: with features of a pretrained ViT (dimension 768), preliminary results are given in Tables R1 and R4.

Finally, we added experiments on ADBench datasets, where our method obtains the best AUROC on six of seven datasets (Table R2).

Q2: what purpose does a known type-I error rate serve for OOD image detection?

A known type-I error rate lets the detector meet a false-alarm constraint set in advance. We agree with the reviewer that AUROC is a coarse measure: since detection power cannot be guaranteed without knowing the OOD distribution, methods should be compared at the same type-I error. We thus add this comparison, as suggested (Table R5). With SVHN as ID, our flow without reflow detects 90/89/97% of CIFAR-10/CIFAR-100/CelebA at a 1% type-I error.

We also agree that computing this rate is the main selling point of the method. It matters at small α\alpha, as in radar: an accurate empirical threshold needs about 100/α100/\alpha ID samples, i.e. 10710^{7} at α=10−5\alpha=10^{-5}. With our 5×1045\times10^{4} samples, none even exists at this level, while the analytic one is given in Table R6. This requires an accurate transport, so far only obtained in low dimension: on images (d=3072d=3072), it is not reliable, hence the empirical thresholds in Table R5.

Q3: how are so many AUROC values in Table 1 below 0.5?

As usual in OOD detection, each score's orientation is fixed in advance (higher means more anomalous). Inverting it requires knowing which OOD set is coming, and the right orientation depends on it: for WAIC with CIFAR-10 as ID, inverting raises the AUROC on SVHN (0.143 → 0.857) but lowers it on CelebA (0.928 → 0.072). The CIFAR-10 → SVHN case is the failure of likelihood-based generative models discussed above.

Q4: have the bounds been evaluated on trained models, and how loose/tight are they in practice?

Yes. The bound can be evaluated only when the true transport is known, as for Gaussian and spherical data, both classical models in radar detection. In the Gaussian case, the bound of Lemma 1 holds but is conservative by about one order of magnitude, as expected from a worst-case bound (Table R1). We concede that it cannot be evaluated on images, where the true transport is unknown.

Q6: seed variation error bars for all methods / (ID, OOD) pairs.

We have added seed-variation results for all evaluated (ID, OOD) pairs using our method with feature extraction stage (Table R4). We also report seed variation and paired statistical significance tests comparing our method with TCCM on ADBench healthcare datasets (Table R7).

Reviewer_uus6OpenReview ↗

Raising score to 3

I thank the authors for their detailed replies and the inclusion of follow-up studies. I appreciate the clearer distinction between theoretical vs empirical results, and agree with the softening of the language around "guarantees" versus "bounds". I am also happy to see the new results with ViT feature extraction and on ADBench. Taken together, these changes have substantially strengthened the paper's empirical basis and clarified the theory vs experiment divide. I will raise my score to 3.

Thank you very much for your comment, we appreciate your positive feedback. And thank you for raising your score!

Comments on the paper

4 comments
AuthorsOpenReview ↗

Tables R1–R7 supporting the responses

Table R1. OOD detection performance comparison in terms of AUROC (higher is better). All methods are trained or pretrained on ID data and evaluated on OOD data. ``-'' indicates that the corresponding result is unavailable. Best value per column in bold. ''Ours'' refers to our original scheme applied to the images as input, whereas, "Ours+feature extraction" refers to the original scheme applied to the features of images as input (here, we used ViT for feature extraction)

MethodCIFAR-10 (ID): SVHNCIFAR-10 (ID): CelebACIFAR-10 (ID): C100SVHN (ID): C10SVHN (ID): CelebASVHN (ID): C100
IGEBM0.6340.7000.500–––
VAEBM0.8300.7700.620–––
WAIC0.1430.9280.5320.8020.9910.831
LR0.0640.9140.5200.8190.9120.779
NLL0.0910.5740.5210.9900.9990.992
DDPM-OOD0.3900.6590.5360.9510.9860.945
ReAct0.3340.3430.4750.9320.9480.930
ASH0.5280.3450.4930.3200.8040.338
Ours0.5030.6950.5110.8970.9930.963
Ours+feature extraction0.7350.9790.8210.9991.0000.997

Table R2. Anomaly detection performance comparison on selected ADBench datasets in terms of AUROC (higher is better). For CIFAR-10, the reported AUROC is averaged over the ten class-wise anomaly detection settings (CIFAR-10_0 to CIFAR-10_9).

Dataset (#Features)PCAOCSVMLOFCBLOFCOFHBOSKNNSODCOPODECODDeepSVDDLODAIForestTCCMOurs
cardio (21)0.9700.9640.9530.9500.4760.8850.9280.5090.9310.9460.9130.9070.9610.9220.981
Cardiotocography (21)0.8340.8190.8020.7170.4840.5820.7790.4070.6700.7920.8270.7920.8180.8030.871
mammography (6)0.8940.8740.8570.8320.6730.8480.8710.7560.9050.9060.9070.8770.8750.9000.911
musk (166)1.0000.5001.0001.0000.4030.9991.0000.1840.9450.9521.0000.9480.9851.0001.000
Pima (8)0.7170.6380.6610.6770.5090.7110.6930.5670.6800.6080.7110.6240.7300.7280.761
WDBC (30)0.9880.9720.9950.9950.9970.9550.9950.9250.9890.9690.9730.9810.9870.9911.000
CIFAR-10 (512)0.9220.9220.8590.9310.4480.8870.9500.7300.9000.8810.8430.8930.8860.9400.955

Table R3. SS vs. the negative log-likelihood (NLL) of the same flow (no reflow), AUROC. On Gaussian clutter, the detection rates are 0.382 ± 0.009 (SS) and 0.371 ± 0.007 (NLL).

ID → OODSSNLL
SVHN → CIFAR-100.9620.991
SVHN → CIFAR-1000.9510.990
SVHN → CelebA0.9960.999
CIFAR-10 → SVHN0.4450.095
CIFAR-10 → CIFAR-1000.5140.521
CIFAR-10 → CelebA0.6950.534

Table R4. OOD detection performance (AUROC) against methods relying on a pretrained backbone (SCOPED, GEPC: pretrained on CelebA-32). "Ours, pixels": row of Table 1. "Ours, ViT features": our detector on features of a pretrained ViT (dimension 768), mean ± std over seeds.

MethodC10 → SVHNC10 → CelebAC10 → C100SVHN → C10SVHN → CelebASVHN → C100
SCOPED0.8140.9400.4770.9710.9960.959
GEPC0.8420.9990.5540.8801.0000.897
Ours, pixels0.5030.6950.5110.8970.9930.963
Ours, ViT features0.732 ± 0.0030.977 ± 0.0020.820 ± 0.0010.998 ± 0.0010.999 ± 0.0010.997 ± 0.000

Table R5. Detection rate at a fixed type-I error α=1%\alpha=1\% (ID: SVHN). The threshold is the empirical quantile of the scores on half of the ID test set; the realised rate is measured on the other half.

ScoreRealised type-I errorCIFAR-10CIFAR-100CelebA
Ours, no reflow0.88%0.900.890.97
Ours, reflow1.05%0.540.590.63
ReAct1.07%0.500.510.70

We sincerely thank the reviewers for their careful and constructive reviews. The main changes to the revised paper are the following.

Title. Now "From Flow Matching to Anomaly Detection with Upper Bounded Type I Error". The term "Guarantees" overstated our bounds: they upper-bound the type-I error and give the rate at which the nominal level is recovered as the excess flow matching loss vanishes, without certifying a level numerically.

Limitations. Now stated in the paper, notably for vision OOD. We identify the cause of this limitation and give an alternative with preliminary results (Table R1), pending a deeper numerical and theoretical study.

Lemma 1. The bound involved the Lipschitz constant of the flow matching network, hard to control in practice. It is replaced by a factor estimated by automatic differentiation (which can also be evaluated exactly when the exact transport is known).

Experiments. We added a more thorough ablation study and a comparison on ADBench datasets, where our method ranks first or ties against 14 baselines.

The additional numerical results are gathered in Tables R1–R7, in the following comment; several of them answer points raised by two or three reviewers.

The revised PDF will be uploaded shortly; until then, these changes are described in our responses.

AuthorsOpenReview ↗

Tables R6–R8 supporting the responses

Table R6. Realised type-I error as α^/α\hat\alpha/\alpha (target: 1) with the analytic threshold, for flows trained on n=5×104n=5\times10^{4} Gaussian H0\mathcal H_0 samples and evaluated on 10710^{7} draws; mean [min, max] over five seeds. Last row: empirical threshold from nn samples, mean [5%, 95%]; it does not exist when α<1/(n+1)\alpha<1/(n+1).

α=10−2\alpha=10^{-2}α=10−3\alpha=10^{-3}α=10−4\alpha=10^{-4}α=10−5\alpha=10^{-5}
d=4d=41.03 [1.00, 1.10]1.09 [0.96, 1.20]1.24 [1.01, 1.36]1.77 [1.51, 2.13]
d=8d=81.10 [1.08, 1.14]1.24 [1.15, 1.33]1.49 [1.19, 1.67]2.09 [1.39, 2.45]
Empirical threshold, same nn1.00 [0.93, 1.07]1.00 [0.78, 1.24]1.00 [0.39, 1.83]does not exist

Table R7. TCCM vs. ours on ADBench (Han et al., 2022): mean ± std over seeds and p-values of paired tests.

DatasetAUROC TCCMAUROC OurspAUPR TCCMAUPR Oursp
Cardiotocography0.798 ± 0.0110.861 ± 0.010<0.0010.708 ± 0.0090.822 ± 0.017<0.001
Mammography0.867 ± 0.0310.910 ± 0.0030.0290.424 ± 0.0600.576 ± 0.0240.002
WDBC0.993 ± 0.0031.000 ± 0.0000.0070.872 ± 0.0611.000 ± 0.0000.009

Table R8. Bound of Lemma 1 on trained flows, Gaussian reference (η=10−2\eta = 10^{-2}). Factor = bound / measured value. α^\hat\alpha: realised false-alarm rate with the analytic threshold at α=10−2\alpha = 10^{-2}.

ddnnε\varepsilon measuredBoundα^−α\hat\alpha-\alpha
450k0.0870.88≈ 0
41M0.0520.60≈ 0
850k0.1741.52+0.0014
81M0.0931.09+0.0005

Dear Reviewers,

Please respond to the authors' rebuttal. Thank you.

Regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.