acceptodds
Under review as a conference paper at ICLR 2027

The Signal in the Noise: OOD Detection Through Goodness-of-Fit Testing in Factorised Latent Spaces

Abstract

Deep generative models offer a natural foundation for out-of-distribution (OOD) detection, yet prior work has shown that their assigned likelihoods are notoriously unreliable indicators for in- vs out-of-distribution data. In this paper, we address this problem by leveraging the diffeomorphic and mass-preserving properties of continuous normalising flows. Our analysis shows that OOD samples are mapped to noise samples that are highly atypical under the noise prior in ways not captured by the likelihood. Based on this observation, we propose a new method—Signal in the Noise (SITN)—for OOD detection on the single-sample level. SITN requires no access to OOD data, incurs minimal computational overhead, and provides strict control of false positive rates. Comprehensive evaluations through standard benchmarks and synthetic perturbations highlight the method's effectiveness and the absence of the complexity bias inherent to likelihood-based methods.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
OOD Detection, AI Safety, Robustness, Reliability

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_552tOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method called SITN for detecting out-of-distribution (OOD) data. It uses continuous normalizing flows trained on in-distribution (ID) data to map observations into a latent space. The method calculates two statistics: an Anderson–Darling test to check for deviations from a standard normal distribution, and a power spectrum-based test to check for specific correlations between coordinates. These are combined using empirical quantiles. A separate calibration step then sets the detection threshold based on a desired false positive rate. Experiments compare SITN to several existing methods on standard OOD detection and image corruption tasks.

Critical strengths

The method is relatively simple and easy to interpret. The two statistics analyze different properties of the latent representation and the experiments show they respond to different types of errors. Importantly, the method does not need OoD examples to train or calibrate. The experimental results are good in several cases. For instance, on CIFAR-10 → SVHN, SITN achieves a score of 0.946, while the DoSE method only gets 0.802. On CIFAR-10-C, SITN averages 0.85 compared to 0.78 for DoSE. These results suggest that looking at latent statistics is a useful alternative to just using likelihoods.

The comparison experiments are fair because they use the same underlying generative models, which helps to understand the contribution of the detection method. The supplement baselines and visualizations of where the method fails are also helpful.

Critical weaknesses

1. Lack of novelty compared to related work. Testing observations against a reference distribution has been done before. Specifically, the paper does not discuss Wang et al. (2020), which uses white-noise tests for single-sample detection, or KLODS, which uses latent factorization. Since SITN seems similar to these approaches, its novelty is unclear to me.

2. Unclear contribution of combining statistics. While combining an Anderson–Darling and a spectral statistic might be useful, the paper does not clearly explain what this combination adds beyond the individual methods, either theoretically or in the experiments. Direct comparisons with the closest existing methods are needed to prove the contribution is significant.

3. Mathematical mistake in the theoretical analysis. If i'm not wrong, there are some erreors in the Type II error analysis in Appendix K.4. Equation 10 does does not necessarily define a valid probability density. For example, if we set specific parameters D=1D=1, (identity flow), the formula produces a value that grows to infinirt as xx goes to infinity, meaning it cannot be a probability measure. Because of this, Lemma K.7 does not hold as stated. Moreover, the jump from Equation 14 to 15 is unjustified to me. The fact that a statistic is "scale invariant" does not mean its distribution remains unchanged when conditioned on another statistic (like Anderson–Darling) that is affected by scaling. These mathematical errors invalidate the theoretical derivation of the Type II error limits. While the algorithm and experiments might still work, the theory needs major correction.

4. Theoretical motivation is too much wide. The theory assumes an exact invertible flow, meaning the ID distribution matches a Gaussian latent distribution. However, even if the latent distributions are different, the AD and spectral tests might not see the difference. For example, if one take a normal distribution GG and flip the sign of its first coordinate (Z=sign⁡(G1)GZ=\operatorname{sign}(G_1)G), the new distribution is not Gaussian, but a simple sign flip does not change the Anderson–Darling or power spectrum values. Is this example shows that the theory does not prove the method can detect all types of deviations ? The paper should clarify exactly what the method can and cannot detect.

5. Overstated conclusions. The results show performance varies a lot. In Appendix J.3, SITN gets a low score (0.572) on CIFAR-10 to CIFAR-100 and 0.690 on CIFAR-10 to Tiny ImageNet. This suggests the method struggles with semantic shifts (which is quite expected...) Also, comparing against discriminative models is tricky because those models have labels at their disposition, which is an advantage. The paper should explain these limitations more carefully. Similarly, the claim that the method is not biased by image complexity relies on a weak correlation with image compression size in just one experiment. This only supports a narrow conclusion.

6. Evaluation metrics and reliability. Figure 12 shows calibration, but the main tables only report AUROC. Since the goal is to control false positives, reporting detection rates at specific low rates (like 1% or 5%) is vital. A high AUROC does not guarantee good performance at low false positive rates. Also, the current guarantees include an estimation error term, so they are not exact. Furthermore, the reported confidence intervals only measure uncertainty from the test samples. They do not account for the variability that happens when training the flow model repeadly.

  1. Wang, Z., Dai, B., Wipf, D. P., & Zhu, J. (2020). Further Analysis of Outlier Detection with Deep Generative Models. Advances in Neural Information Processing Systems (NeurIPS).

  2. Zhang, Y., Pan, J., Liu, W., Chen, Z., Li, K., Wang, J., Liu, Z., & Wei, H. (2024). Kullback-Leibler Divergence-Based Out-of-Distribution Detection With Flow-Based Generative Models. IEEE Transactions on Knowledge and Data Engineering, 36(4), 1683–1697. Published online in 2023.

  3. Jiang, D., Sun, S., & Yu, Y. (2022). Revisiting Flow Generative Models for Group-wise Out-of-Distribution Detection. International Conference on Learning Representations (ICLR).

Questions for authors
  1. How does SITN differ from the white-noise testing of Wang et al. or the latent-factorization of KLODS? Could you provide a comparison using similar models and data if time/ressources allows it?
  2. Could you fix the definition in Equation 10, and the resulting Lemma K.7, and the logical bridge between Equations 14 and 15. I'm interested to know what valid theoretical conclusion about Type II error remains after these corrections?
  3. What detection rates does SITN achieve at 1% and 5% TNR ? Also, please could you clarifiy if the data used for calibration is completely separate from the data used to select the final model checkpoint ?
4 comments

Thank you for your thorough feedback. We are glad you liked the simplicity of our method and found the paper easy to follow.

Discussion and differentiation from additional related work

Thank you for sharing these additional references, we have included them in the related work discussion of the updated manuscript.

While the work by Wang et al. superficially resembles the independence-testing component of SITN, the two methods differ fundamentally. First, their approach assesses the prediction residuals of autoregressive models in the data (or VAE reconstruction-error) space. SITN, on the other hand, uses CNFs to map samples to the latent space, where (as discussed in Section 3.1) OOD samples manifest as deviations from the Gaussian noise prior. Furthermore, while Wang et al. only test the white noise property (via a lag-based Box-Pierce test) of their constructed sequences, SITN exploits the factorised Gaussian noise prior to simultaneously test both dimensional independence (globally in the frequency domain) and marginal normality. As Appendix A demonstrates, these two components have strong complementary value.

KLODS is motivated from a group OOD detection setting. The idea is to fit a high-dimensional Gaussian distribution to the latent representations of multiple samples, i.e. N(μ~,Σ~)\mathcal{N}(\tilde{\mu}, \tilde{\Sigma}) where μ~\tilde{\mu} and Σ~\tilde{\Sigma} are the mean vector and covariance matrix across samples. A KL divergence between the fitted Gaussian and the prior is then used as OOD score. To apply this to a single sample, one splits the latent representations along the channel dimension and treats that like individual samples. Furthermore, estimation of the sample covariance matrix requires the number of channels in the latent representation to exceed the spatial dimensions (C>H×WC > H \times W); otherwise, the sample covariance matrix is singular. The authors work around this by using only the latents from the last scale in Glow, but the method could not be applied to single samples using standard CNF latents as in SITN. SITN, on the other hand, is natively a single-sample method that leverages the complete factorisation of the noise prior. Instead of fitting a multivariate Gaussian across samples, SITN directly performs a goodness of fit test across the elements of a single latent representation. Moreover, through the Gaussian fit, KLODS only captures first- and second-order moments (mean μ~\tilde{\mu} and linear pairwise pixel covariance Σ~\tilde{\Sigma} across channels in the single-sample case) whereas SITN's AD component is also sensitive to higher-order shape distortions, skewness, heavy tails.

Value of combining statistics

We provide theoretical justification and motivation for the individual statistics and their combination in section 3.2 of the manuscript. The two statistics have different purposes; they allow us to check for normality and independence, respectively.

Empirical evidence that these statistics have complementary value is presented in Section 5.4 (Ablations) and further in Appendix A. Their combination in SITN is superior to using either of the statistics individually.

Theoretical analysis

Equation (10) is correct, as ∥X∥22\|X\|_2^2 is in the superscript of the exponential. We have changed the typesetting of this Equation in the updated submission to make this clearer. We have also simplified the derivation around Equations (14)-(15), removing the incorrect conditional probability step in the process. The overall conclusions remain the same.

Captured types of deviations

We do not claim that our method can detect all types of dependence. In fact, we specifically acknowledge prior work that showed that no statistical test can do so comprehensively in Section 3.2. As explained, our method specifically tests for two properties of the Gaussian prior: marginal normality and autocorrelation. We have added a new dedicated limitations section where we reiterate this.

Results comparing to discriminative methods and image complexity analysis

We agree and in fact already discuss the caveats of comparing to discriminative methods in a dedicated section (J.1) in the appendix.

Semantic OOD detection is discussed in section J.3. SITN performs best among generative methods and even outperforms discriminative methods in some dataset pairings (CIFAR->SVHN, CIFAR->MNIST). Moreover, as discussed and highlighted by the reviewer, discriminative methods have an unfair advantage in this setting given their access to class labels.

Regarding the analysis of image complexity, we follow the same methodology using compression size as the original work by Serrà et al. that established the complexity bias. Furthermore, this is not the only evidence presented in the paper. The analysis in Section 5.3 and further visualisations of the 150 samples with highest OOD scores for each method in the appendix provide another stark illustration of the complexity bias in likelihood-based methods.

Evaluation metrics and reliability

Thank you for the recommendation. We have amended the manuscript with an additional appendix (Appendix L) showing actual FPR on ID test data as well as TPR across thresholds calibrated to FPRs between 0.001 and 0.1 on validation data for the different CIFAR-10 dataset pairings. SITN scores were calibrated as described in the paper, while the rest of the methods do not require validation data to be split and were calibrated directly on the full validation set through their empirical CDF. SITN consistently shows higher TPRs across thresholds.

The finite-sample slack in Corollary K.2 is to be expected. It is not generally possible for finite sample estimates to be guaranteed to exactly recover a population statistic.

Regarding the confidence intervals, we compare methods on the same, fixed flow model. This is a different situation than comparing different, separately trained models where training variability could obscure performance comparisons. Furthermore, results are consistent across models that were trained for experiments with different (ID) datasets.

For simplicity, the same validation data was used in our experiments for selection of the checkpoint and calibration. Empirical calibration curves in the appendix show that this did not have an effect in practice.

Thank you for your detailed rebuttal as it is answersing most of my question. But i prefer to keep my score at 2 for the following reasons :

-First, I may be not understanding something, but even for the identity flow, the expression in the eq. 10 seems to grow in the tails for an Std Gaussian and some values of aa (such as a=2a=2 ?). I'm therefore still unsure whether it defines a density measure probability and how lemma K7 follows from that.

-Second, if I'm correct, the validation data were used for both chosing the checkpoint and the calibration phase to get the FPR95 scores but you suppose that the model is fixed indepentely from the calibration set

Thank you for engaging with our rebuttal.

Does Eq (10) define a normalised density with the identity flow map and a=2a=2?

We found a couple of typos in the equation that are now fixed. In particular, there was a factor of 22 missing from the denominator in the exponential, and it should have read ϕ−1(X)\phi^{-1}(X) instead of XX inside the exponential. This will always result in a normalised PDF, including the case suggested by the reviewer. This can be seen from a change of measure argument, noting that the two Jacobian determinants will cancel out, and then simplifying the resulting product of exponentials.

Same data for early stopping checkpoint selection and calibration?

Using the same data for early stopping and calibration is not a prescribed step in our method; one is not obliged to do this. Nevertheless, we have included a new Corollary K.3 that shows a logarithmic correction factor can be incurred if one does wish to do this.

Reviewer_sxXsOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the well-known failure of likelihood-based out-of-distribution (OOD) detection, where deep generative models assign higher likelihoods to OOD samples than to in-distribution (ID) samples. Instead of correcting the likelihood, the authors propose Signal in the Noise (SITN), which exploits the diffeomorphic and mass-preserving properties of continuous normalising flows (CNFs) trained with flow matching. Specifically, a test sample is mapped back to the noise space through the inverse flow, and SITN examines at the single-sample level whether the resulting latent vector is consistent with the prior N(0,ID)\mathcal{N}(0, I_D). Since this prior factorises completely, the latent vector is treated as DD i.i.d. univariate standard normal samples. Marginal normality is then tested with the Anderson-Darling statistic, whereas dimensional independence is assessed through the coefficient of variation of the power spectrum. The two statistics are combined by a max-quantile strategy calibrated on held-out ID data, which requires no OOD data and provides false positive rate control. Empirically, SITN outperforms likelihood-based baselines such as the log-likelihood, Typicality, and DoSE in most cross-dataset settings among CIFAR-10, SVHN, and CelebA in both directions. Furthermore, SITN achieves the best average performance on the synthetic perturbations of CIFAR-10-C.

Critical strengths
  1. The paper is clearly written and easy to follow.

  2. Going beyond typicality tests that examine whether the likelihood lies within the typical set, the paper tests whether the latent vector actually exhibits the properties of the standard normal prior. As a researcher in this field, I find this idea novel and refreshing. Existing approaches have mostly relied on the value of the likelihood itself (log⁡p(x)\log p(\mathbf{x}) or log⁡p(z)\log p(\mathbf{z})). Others have combined it with additional statistics such as the Jacobian term or the PNG compression length, and some further require an auxiliary model. In contrast, assessing whether the latent vector is consistent with the prior from the perspectives of marginal normality and independence is a sound and well-motivated idea.

  3. SITN achieves higher performance than likelihood-based baselines in most cross-dataset settings, which demonstrates a practical improvement.

  4. The additional analyses are well executed. They include the visualisation of samples with the highest OOD scores and the relationship between each score and the PNG complexity. In particular, prior work has shown that the likelihood is strongly and inversely related to the PNG bit length. The proposed score appears largely agnostic to complexity (Appendix B, Figure 6), which suggests that SITN avoids the paradoxical behaviour that the likelihood exhibits in OOD detection.

Critical weaknesses
  1. (Major) The paper discusses only continuous normalising flows (CNFs). To my knowledge, the likelihood paradox has been studied mainly with discrete normalising flows such as Glow [1] and ResFlow [2]. The manuscript lacks a discussion on whether SITN extends to such models (e.g., coupling layer-based flows), and the justification for restricting the study to CNFs is missing. The diffeomorphic and mass-preserving properties that motivate the framework are shared by all normalising flows, so it remains unclear whether the choice of CNF is essential to the method or merely a matter of implementation.

  2. (Major) Reduced computational overhead is presented as one of the main contributions, yet the paper does not quantify how much time is actually saved. This is my own understanding, but I suspect that the computational advantage over the standard likelihood test would be limited if SITN were implemented on normalising flows other than CNFs. For models whose Jacobian determinant is available in closed form (e.g., RealNVP [3] and Glow [1]), the likelihood computation does not incur a large overhead. I would therefore like the authors to discuss the overhead of SITN beyond the CNF setting.

  3. (Minor) The paper does not contain a limitations section. For instance, zero autocorrelation does not imply independence, so the independence test cannot detect nonlinear dependence. The authors should include a dedicated section that discusses such limitations. This section could also cover the failure mode on constant backgrounds (Section 5.5) and the restriction to low-resolution images.

  4. (Minor) The main experiments appear rather thin. Table 1 compares SITN only against the log-likelihood, Typicality, and DoSE. The complexity correction and WAIC baselines are reported only in the appendix (Appendix E, Table 3), and the same holds for DiffPath. Since these methods are directly relevant to the likelihood paradox, I would like to see them included in the main table. With the current main table alone, the empirical evidence looks insufficient to support the claim of consistent improvement over prior work. In addition, including the GMM-based method of [6] as a baseline would make the evaluation more solid.

  5. (Minor) The related work would benefit from additional references. Works that explain the paradox from the entropic perspective [4, 5] and from the image complexity perspective [6] would make the paper more solid.

Reference [1] Kingma, Durk P., and Prafulla Dhariwal. "Glow: Generative flow with invertible 1x1 convolutions." Advances in neural information processing systems 31 (2018). [2] Chen, Ricky TQ, et al. "Residual flows for invertible generative modeling." Advances in neural information processing systems 32 (2019). [3] Dinh, Laurent, Jascha Sohl-Dickstein, and Samy Bengio. "Density estimation using real nvp." arXiv preprint arXiv:1605.08803 (2016). [4] Caterini, Anthony L., and Gabriel Loaiza-Ganem. "Entropic issues in likelihood-based ood detection." I (Still) Can't Believe It's Not Better! Workshop at NeurIPS 2021. PMLR, 2022. [5] Kim, Donghwan, and Hyunsoo Yoon. "Mitigating the Likelihood Paradox in Flow-based OOD Detection via Entropy Manipulation." arXiv preprint arXiv:2602.09581 (2026). [6] Osada, Genki, Tsubasa Takahashi, and Takashi Nishide. "Understanding likelihood of normalizing flow and image complexity through the lens of out-of-distribution detection." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 19. 2024.

Questions for authors

Please see the Weaknesses.

2 comments

Thank you for your detailed review. It is really good to hear that you found our method novel and refreshing.

Justification for the use of CNFs and applicability to DNFs

We used CNFs because of their higher architectural flexibility and widespread adoption in modern generative modelling. We agree that the theory and broader framework are also compatible with DNFs as long as certain implementation details are consistent. For example, if non-standard or learned conditional priors are used in multi-scale architectures like Glow, one would need to transform the latents to conform to the standard normal prior assumed by SITN's test statistics. Furthermore, applying SITN’s CV component to the flattened latent representation may not work as well for a method like Glow, where channels at different scales correspond to different spatial resolutions. In such cases, SITN might benefit from some adjustments to implementation details.

Computational overhead

We would like to clarify that we do not see reduced computational overhead as a focus or key contribution of our work. Our intention was to note that SITN does not introduce any significant computational overhead compared to likelihood-based baselines (other than for example WAIC which requires the training and evaluation of an entire ensemble of models or other works that require additional training or model evaluations as discussed in the related works section). We have updated the wording in the manuscript to make this clearer.

That said, we discuss SITN’s computational complexity beyond obtaining the latent representation in Section 3.3 of the manuscript. Finally, we note that there is a wealth of literature aimed at improving the computational efficiency of CNFs that could be used to improve the computational efficiency of our method.

[1] OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport (Onken et al., 2021)

[2] Multisample Flow Matching: Straightening Flows with Minibatch Couplings (Pooladian et al., 2023)

Paper structure (limitations section, organisation of results)

Thank you for your constructive recommendations to improve the structure of the paper.

We previously discussed limitations in the methods and discussion sections of the paper, but are happy to do so in a dedicated section using the extended page limit for the camera-ready version. We have added the section in the updated manuscript.

We also considered including the additional baselines directly in the main table in the results section. However, for several reasons we prefer to keep them in a dedicated appendix. First, focusing on a subset of most relevant baselines in the main text allows us to effectively communicate the main message of the paper while keeping things consistent (same set of methods shown across the different analyses and ablations in the main text) and aesthetically clean (e.g. no shrinking of tables). Practically, the additional baselines required deviations from our standard evaluation setup used for the baselines in the main paper and SITN. WAIC required the training and evaluation of an entire ensemble of models, Complexity Correction required the additional computation of PNG compression sizes, and DiffPath required additional evaluations of trajectories. Including them in every experiment would have increased the computational and logistic complexity of our evaluations substantially. That said, we agree that the additional baselines strengthen the results and we will reference them more prominently in the main text for the reader’s awareness.

Additional references for related work

Thank you for bringing these additional references to our attention. We have included them in the related works section of the updated manuscript.

The work by Osada et al. is very closely related to the complexity correction by Serra et al., which we already included as an additional baseline in Appendix E. Instead of directly correcting likelihoods through a likelihood ratio, they include the complexity term as an additional variable in a GMM. As such, their method also shares the same limitation of requiring a good complexity estimate, which can be arbitrary and severely limiting in non-standard data domains where suitable compression algorithms are unavailable.

Thank you for the detailed response. The authors have adequately addressed my main concerns and clarified the scope and limitations of the proposed method. I maintain my original rating.

Reviewer_EuNqOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method for OOD detection using continuous normalizing flows. Instead of relying on likelihoods, the authors map samples back to the Gaussian noise prior and test whether the resulting latent representations exhibit the expected statistical properties. The method combines different statistics measuring marginal normality and dimensional independence. The authors evaluate the method on several image datasets and synthetic perturbations and show tangible improvements over existing likelihood-based approaches.

Critical strengths
  • The method is simple, requires no OOD data, and is computationally cheap to apply to an already trained flow model.
  • The empirical results are strong overall. The ablations are informative too and suggest that much of the gain comes from the choice of the test statistics. The scores also show very little correlation with image complexity, addressing a known failure mode of likelihood-based methods.
  • The paper is also well written and easy to follow.
Critical weaknesses
  • Although the combination of ideas is interesting, the conceptual novelty is somewhat limited. The contribution builds on existing work on likelihood-based OOD detection, latent-space goodness-of-fit testing (Jiang et al., 2022), and combining multiple statistics (DoSE). The paper would benefit from a clearer distinction from Jiang et al.

  • I believe the positioning of this paper could be broader. Combining complementary statistics extracted from generative models has also been explored for diffusion models (e.g., Graham et al., 2023; Liu et al,2024, Heng et al., 2024; Jaziri et al., 2026). While this is not necessarily a weakness, I think the paper would benefit from discussing the distinction between testing the statistical properties of the final noise representation, as done here, and examining reconstruction behavior or denoising trajectories, as done in diffusion-based approaches. These perspectives seem complementary and could help better motivate the contribution of this work.

  • Given the computational cost of inverting continuous normalizing flows on high-dimensional inputs, I think the previous point is worth discussing as it remains unclear how efficiently the proposed approach scales to larger samples.

Jiang, D., Sun, S., & Yu, Y. (2022). Revisiting Flow Generative Models for Out-of-Distribution Detection. ICLR 2022.

Morningstar, W. R., et al. (2021). Density of States Estimation for Out-of-Distribution Detection. AISTATS 2021.

Graham, M. S., et al. (2023). Denoising Diffusion Models for Out-of-Distribution Detection. CVPR 2023.

Heng, A., Thiery, A. H., & Soh, H. (2024). Out-of-Distribution Detection with a Single Unconditional Diffusion Model. NeurIPS 2024.

Jaziri, A., et al. (2026). Beyond Binary OOD Detection: Characterizing Distributional Shifts with Multi-Statistic Diffusion Trajectories. AISTATS 2026.

Liu, Z. et.al . Unsupervised Out-of-Distribution Detection with Diffusion Inpainting. ICML, 2023

Questions for authors
  • Jiang et al. (2022) also investigate goodness-of-fit testing in the latent space of flows. Could the authors clarify how SITN differs from their approach, particularly regarding single-sample detection?
  • How sensitive is SITN to the quality of the learned flow? I am wondering whether deviations from the Gaussian prior caused by imperfect training could affect performance?
  • The independence statistic relies on the power spectrum of the flattened latent representation. How sensitive is this statistic to the ordering of latent coordinates? Could you also elaborate a bit on what forms of dependence is it actually capturing?
2 comments

Thank you for your constructive review. We appreciate that you found the empirical results convincing and the paper well written.

Extended discussion and differentiation from related work

We have extended the current discussion of the work by Jiang et al. as well as DoSE in the related works section.

As mentioned in the original version of the manuscript, the work by Jiang et al. is concerned with group OOD detection and not directly applicable to our setting. Given a batch of multiple samples, they project latent representations onto random directions and test these for normality using a Kolmogorov-Smirnov test. SITN, on the other hand, is natively a single-sample method. It leverages the complete factorisation of the noise prior to directly perform a goodness of fit test across the elements of a single latent representation. Beyond that, SITN is also different in terms of the used statistics to test both normality and dimensional independence, and in how these are combined with a max-quantile approach to achieve testing of the constituent hypotheses.

Similar to DoSE, SITN combines multiple test statistics. However, it uses different statistics and combines them in a different way – both aspects are compared to the corresponding DoSE components in the paper and consistently result in improved performance. Moreover, other than for DoSE, the statistics chosen in SITN are theoretically motivated to specifically test the constituent hypotheses of marginal normality and dimensional independence to test for consistency with the Gaussian noise prior. Finally, DoSE is based on the likelihood and its subcomponents in the instantaneous change of variables formula and hence shares the complexity bias and failure cases of other likelihood-based methods.

We also agree that our approach may provide complementary value to other approaches based on other inputs like trajectory information or reconstructions.

Computational costs

We discuss differences and advantages over related work above and in the manuscript. Additionally, regarding computational costs, we would add two things. First, computational costs will be quite low and not constitute a bottleneck for many applications – as reported in Appendix L, evaluations took about 6 minutes per 1000 samples in our setup. Second, there is a wealth of literature aimed at making CNFs more efficient that could be leveraged to improve computational efficiency [1, 2].

[1] OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport (Onken et al., 2021)

[2] Multisample Flow Matching: Straightening Flows with Minibatch Couplings (Pooladian et al., 2023)

Sensitivity to CNF quality

We discuss this in Section 3.3 of the manuscript. One should expect poorer quality CNFs to result in increased false negative (i.e., Type II error) rates, while the false positive rate is not impacted.

Sensitivity to latent coordinate ordering and captured forms of dependence

As discussed in 3.2 of the manuscript, the CV statistic specifically tests for autocorrelation.

To assess the method’s sensitivity to the latent ordering, we performed additional experiments comparing the standard SITN implementation (with simple flattening through row-wise concatenation) to six alternative latent orderings: column-wise concatenation, snake, diagonal, morton, hilber, or spiral. The results are shown in an additional appendix (Appendix M) in the updated manuscript along with illustrations of the different flattening operations. Overall, the experiments show that the latent ordering does affect performance, with simple row- or column-wise concatenation performing best among the considered options.

I thank the authors for their response, which has addressed my concerns. I have no further questions and support the acceptance of the paper. I will therefore maintain my current score.

Reviewer_qUnxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Signal in the Noise (SITN), a single-sample OOD detector operating in the Gaussian latent space of continuous normalising flows. It combines an Anderson–Darling statistic for marginal normality with a power-spectrum statistic for dependence, using ID-calibrated quantiles. Experiments cover cross-dataset detection, CIFAR-10-C corruptions, and OpenOOD benchmarks. The paper also develops calibration guarantees and analyzes failure modes of likelihood-based detectors.

Critical strengths
  • A simple and practically appealing detection mechanism. SITN exploits information beyond latent likelihood using one generative model, without class labels or OOD examples, and avoids divergence evaluations (Sections 3.2–3.3). The component ablation demonstrates complementary sensitivity to different corruptions (Appendix A, Figure 5), supporting the rationale for combining normality and dependence statistics.
  • Substantial empirical improvements with useful controls. On CIFAR-10→SVHN, SITN achieves 0.946 AUROC versus 0.802 for DoSE (Table 1), while its mean CIFAR-10-C AUROC improves from 0.78 to 0.85 (Figure 2). Additional baselines and calibration-split controls strengthen the evidence that these gains reflect useful latent information rather than an unfavorable baseline configuration (Table 3; Appendix I).
Critical weaknesses
  • The statistical guarantees require correction and qualification. Appendix K.4 contains substantive errors: Equation (10) need not define a normalized density—even for an identity flow, it is non-integrable when (a=2)—and therefore does not establish Lemma K.7. Moreover, CV scale invariance alone does not justify the conditional-probability equality in Equations (14)–(15). Separately, Corollary K.2 provides an FPR bound with finite-sample slack, rather than exact control at the requested level. These issues weaken the advertised theoretical contribution, although they do not invalidate the empirical AUROCs.
  • Detection power at the claimed operating constraint remains unestablished. Results emphasize AUROC; Figure 12 demonstrates calibration but does not report OOD recall at low, ID-calibrated false-positive rates. This matters particularly given modest performance on CIFAR-10→CelebA (0.671; Table 1) and semantic near-OOD detection (0.572 on CIFAR-100 and 0.690 on TIN; Table 7). The evidence therefore leaves uncertainty about practical effectiveness when false alarms must be tightly limited.
Questions for authors
  1. How should Equation (10) and Lemma K.7 be corrected, and what additional argument justifies Equations (14)–(15)?

  2. What finite-sample FPR guarantee holds for the implemented Equation (4) rule, accounting for empirical-quantile ties and the actual calibration-set sizes?

  3. With thresholds selected exclusively from ID calibration data, what OOD recall and realized ID false-positive rates are obtained at nominal FPRs of 1% and 5%, particularly on the weaker dataset pairings?

3 comments

Thank you for your helpful review and comments. We are glad that you liked the simplicity and practicality of our method.

Questions on statistical guarantees

Equation (10) is correct, as ∥X∥22\|X\|_2^2 is in the superscript of the exponential. We have changed the typesetting of this Equation in the updated submission to make this clearer. We have also simplified the derivation around Equations (14)-(15), removing the incorrect conditional probability step in the process. The overall conclusions remain the same. The finite-sample slack in Corollary K.2 is to be expected. It is not generally possible for finite sample estimates to be guaranteed to exactly recover a population statistic.

Performance at calibrated thresholds

Thank you for the suggestion. We have amended the manuscript with an additional appendix (Appendix L) showing actual FPR on ID test data as well as TPR across thresholds calibrated to FPRs between 0.001 and 0.1 on validation data for the different CIFAR-10 dataset pairings. SITN scores were calibrated as described in the paper, while the rest of the methods do not require validation data to be split and were calibrated directly on the full validation set through their empirical CDF. SITN consistently shows higher TPRs across thresholds, including on the particular dataset pairings you highlighted.

Thanks for clarifying the questions I was concerned about in the reviewing process. But i prefer to keep my score because of the main experimental results and conclusions are not illustrated in the main paper and the intricate equations.

Thank you for your response. We appreciate your review, but find it hard to understand your reasons for maintaining a score of 2.

In your initial review, you highlighted the practical value of our proposed method, the strong improvements over relevant baselines, and the value of the additional controls reported in the supplementary material. You also indicated that you only had minor, fixable concerns about the paper’s soundness.

You raised two points for the rebuttal: (1) questions about the theoretical derivations in the appendix, and (2) a request to include results for thresholds calibrated to specific false positive rates.

We addressed both of these points during the rebuttal, not only promising improvements, but updating the manuscript with the requested additional analysis. In your response, you indicated that these had resolved your concerns.

We would appreciate it if you would reconsider your final score.

Comments on the paper

1 comment

We thank all the reviewers for their feedback and for taking the time to engage in the discussion phase.

We are glad that the reviewers unanimously praised the practical value and effectiveness of the method, the strong empirical evaluations, and that the paper was well written and easy to follow. We are grateful for the suggested improvements and pointers to additional related work, which have helped us to further strengthen the manuscript during the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.