acceptodds
Under review as a conference paper at ICLR 2027

Out of Distribution Detection with Null Space Analysis

Abstract

Neural network development and training often relies on highly curated datasets. The use of curated data does not adequately take into account the possibility of encountering unexpected inputs or outliers. However, encountering outliers is common when deploying systems in real world applications. Since outliers are samples not drawn from the data distribution of interest, the diversity of outliers precludes the ability to train with fully representative outlier data. One mechanism to create more robust neural network systems that can gracefully handle outliers or unexpected inputs is to leverage out-of-distribution (OOD) detection. In this paper we introduce Null Space Analysis (NuSA) as a way to manipulate the null space inherent in many neural networks to detect OOD data. NuSA functions by minimizing the null space projections for inlier data, so that when OOD data is encountered a much larger null space projection will occur. Experiments show an improvement over existing state-of-the-art algorithms using a fixed architecture and several OOD datasets.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Out-of-Distribution, Null Space

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ht2gOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Null-space analysis method for OOD detection. The methods operates by minimizng null space projections for ID data such taht whenever OOD data comes during test, the corresponding projection is larger and easy to detect.

Critical strengths
  1. The method is simple as it does not rely on auxiliary or curated outlier datasets.
  2. The result shows that FPR95 gains over conventional baselines is substantial.
Critical weaknesses
  1. Null space analysis has ben already established research.
  2. Lack of rigorous evaluation on other OOD benchmarks. Also, important post hoc OOD baselines are missing such as Energy, Mahalanobis, KNN, ViM, ASH, LogitNorm, Adagrad etc.
  3. Presnetation is poor and hard to follow. Too much of related works section provided.
  4. No algorithm or codes are provided maiking the claims hard to reproduce.
  5. No statistical reporting of mean and std has been done on the results raising concerns about stability of the reported values.
  6. Lack of generality- not applied to other architectures; only tested in WRN.
Questions for authors
  1. How does NuSA differ from NECO and ViM, both conceptually and empirically?
  2. Can authors report the histograms of NuSA scores for ID and OOD data?
3 comments
AuthorsOpenReview ↗

“How does NuSA differ from NECO and ViM, both conceptually and empirically?”

NuSA is related to but distinct from both NECO and ViM. ViM and NECO are both post-hoc methods. Further, ViM and NECO rely on the PCA of intermediate feature representations of in distribution (ID) samples. ViM analyzes residual magnitudes in the subspace that is orthogonal to the principal subspace associated with the eigendecomposition of the ID feature covariance. They select the top eigenvectors, ranked by eigenvalue, to define the principal subspace. NECO measures a similar value but instead projects directly onto the principal subspace. On the other hand, NuSA is a training time regularization method based on exact computation of the null space of the final layer’s weight matrix. To make the distinction clearer, PCA identifies directions of significant variation in the feature representations, whereas row space and null space analysis identify directions visible or invisible, respectively, to the selected transformation. Our work examines the effectiveness of using the NuSA score as an outlier detector when the model has been trained using the NuSA penalty.

AuthorsOpenReview ↗

“Null space analysis has ben already established research.”

In regards to the novelty of NuSA, it is difficult to process the “established research” without the inclusion of references. We are unaware of any other work in null space analysis of neural network weights for outlier detection.

Thank you authors for the rebuttal. Several of my concerns still remains unanswered (W2, W4, W5, W6, Q1), and there has not been any update to the paper to reflect the changes as advised in the comments. Due to severe presentation issues and lack of supporting empirical evidences, I do not intend to increase my score.

Reviewer_T3C7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Null Space Analysis , a training-time approach for out-of-distribution detection that makes use of the null space of a neural network's fully connected layer. The main idea is to minimize the normalized null-space projection of in-distribution features during training, so that OOD samples can be identified through larger projection scores at test time. The proposed objective is added to the original training loss without requiring auxiliary OOD samples or changes to the network architecture. The authors evaluate NuSA using WideResNet models trained on CIFAR-10 and SVHN, with several other image datasets used for OOD testing. The experimental results show improvements in FPR95, AUROC, and detection error compared with the selected baselines. The paper also examines the relationship between NuSA scores and classification accuracy, as well as the additional computational cost.

Critical strengths
  1. An interesting approach to OOD detection using the network's null space. I find the idea of using the information discarded by a neural network layer for OOD detection interesting. Rather than relying only on the classifier's output or adding a separate detection branch, NuSA incorporates a null-space regularization term directly into training. The method does not require auxiliary OOD samples or architectural changes, which makes it relatively straightforward to integrate into existing networks.
  2. Encouraging experimental results with a simple training objective. The reported results are promising, particularly when CIFAR-10 is used as the in-distribution dataset. NuSA achieves substantial improvements in FPR95 and generally strong AUROC values compared with the selected baselines. I also appreciate the additional analysis relating NuSA scores to classification accuracy and the evaluation of computational overhead. These experiments provide useful evidence that the proposed score can serve as an indicator of unfamiliar inputs.
Critical weaknesses
  1. The implementation of the proposed null-space regularization needs further clarification. My main concern is how the null-space regularization is implemented during training. In Section 3.2, the paper describes the null space using the columns of U from the SVD. However, for a wide weight matrix, the relevant null space should be obtained from the right singular vectors. This may be a notation issue, but I would appreciate clarification.More importantly, since the proposed regularization term depends on the null space of the weight matrix, it is important to understand how gradients are computed through this operation. The paper mentions performing SVD during optimization but provides limited details about the actual implementation. Without this clarification, I find it difficult to fully assess the correctness of the central training mechanism.

  2. The experimental evidence is promising but still somewhat limited. The reported OOD detection results are encouraging, especially on CIFAR-10. However, the evaluation is restricted to one network architecture and a relatively small set of image datasets. It is therefore difficult to determine how well the approach generalizes to more challenging or larger-scale settings.I would also like to see a sensitivity analysis of the regularization weight and a more detailed comparison of in-distribution classification accuracy. Since NuSA changes the training objective, understanding this trade-off is important. Comparisons with additional relevant feature-subspace-based OOD methods would further strengthen the empirical evidence. Overall, I appreciate the idea behind NuSA and the improvements reported in the experiments. However, the uncertainty surrounding the implementation of the core regularization term, together with the limited experimental validation, makes me hesitant to recommend acceptance at this stage. I therefore lean toward weak rejection, although a convincing clarification of these points could change my assessment.

Questions for authors
  1. Could the authors clarify how the null-space projection is computed and differentiated during training? In particular, I would like to understand which singular vectors are used and how the gradients of the NuSA loss propagate back to the weight matrix. A brief explanation or pseudocode would be helpful.
  2. The experiments use a fixed regularization weight of λ = 1.0. Could the authors provide results for a few different values of λ, together with the corresponding classification accuracy? This would help clarify the trade-off between OOD detection performance and the original classification task.
  3. Could the authors discuss how NuSA compares with related feature-subspace-based methods, such as ViM? I would also be interested in whether the approach remains effective on more challenging near-OOD datasets or with a different network architecture.
3 comments
AuthorsOpenReview ↗

“Could the authors clarify how the null-space projection is computed and differentiated during training? In particular, I would like to understand which singular vectors are used and how the gradients of the NuSA loss propagate back to the weight matrix. A brief explanation or pseudocode would be helpful.”

The null space basis is obtained from the columns of V corresponding to the zero singular values. The orthonormal null space basis vectors are then used to construct a projection matrix that maps features onto the null space of the weight matrix of interest, which in our analyses is the final layer of the class head. In reference to the notation error in section 3.2, thank you for pointing this out. This was in fact a mistake. The null space is spanned by the columns of V, not U, corresponding to the zero singular values. The gradients can be tracked through the row space components of the SVD via PyTorch. Functionally, maximizing the magnitude of the row space projection and minimizing the magnitude of the null space projection have the same impact since the row space and null space are defined to be orthogonal, Figure 2 also demonstrates this effect.

AuthorsOpenReview ↗

The experiments use a fixed regularization weight of λ = 1.0. Could the authors provide results for a few different values of λ, together with the corresponding classification accuracy? This would help clarify the trade-off between OOD detection performance and the original classification task.

We thank the reviewer for this suggestion. We agree that such an investigation could provide more insight on the impact of this hyperparameter on the proposed method as well as model performance. In section 4, we acknowledge a trade off between NuSA regularization and accuracy with a drop of 1% in accuracy for the reported lambda of 1. The tradeoff weight, lambda, was selected based on a grid search over several values ranging from 0.1 to 2.0, and the best was selected based on single-run OOD performance. The results display the effectiveness of the proposed method under this setting. We do not believe the absence of this evaluation into lambda undermines the contributions or conclusions of this work.

AuthorsOpenReview ↗

Could the authors discuss how NuSA compares with related feature-subspace-based methods, such as ViM? I would also be interested in whether the approach remains effective on more challenging near-OOD datasets or with a different network architecture.

NuSA is related to but distinct from ViM. NuSA is a training time regularization method based on an exact computation of the null space of the final weight matrix, whereas ViM is a post-hoc method that derives a unique subspace based on the covariance of the inlier feature distribution. We will perform a deeper investigation into near-OOD performance and evaluate on a suite of architectures as future work.

Reviewer_5DgDOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

NuSA is a method for detecting OOD inputs. The last layer of a classifier turns a long feature vector into a few class scores, so part of every feature vector is simply ignored by that layer; this ignored part is the layer's "null space". NuSA measures what fraction of an input's features falls into this ignored part and uses that fraction as the OOD score. During training, the same fraction is added to the usual classification loss, so that in-distribution (ID) images keep almost all of their features in the part the last layer uses. At test time, an input with a large ignored fraction is flagged as OOD.

Critical strengths

Simple and cheap. NuSA is one extra loss term on the last layer. It needs no change to the architecture and no outlier data, adds about 0.5% to training time, and its score comes from the same forward pass.

Critical weaknesses

The papers do evaluate on CIFAR-10 and MNIST, and several datasets existed decades ago. The evaluation uses the 2018 ODIN protocol, which the field has since dropped.

OOD detection is now evaluated on ImageNet-1K with large pretrained models such as ViT-B/16 (OpenOOD v1.5, where MSP still reaches only 76.02 near-OOD AUROC), and with vision-language models that need no training on the ID data (MCM, NeurIPS 2022; NegLabel, ICLR 2024). The paper tests none of these. By its own statement, NuSA "can only be applied to layers that reduce dimensionality" (§3.2), so it is unclear how it would work on a ViT-B/16, whose 768 features feed 1000 ImageNet classes.

On SVHN, the only OOD set that was not resized, SSD (ICLR 2021) and LogitNorm (ICML 2022) report FPR95 of 2.0 and 8.03 against NuSA's 14.31; neither is compared. The baselines that are compared fall far below their own papers: here ODIN is worse than MSP on iSUN and LSUN, while the ODIN paper halves MSP's FPR95 on the same sets.

Questions for authors

Can NuSA be used with a large pretrained model, including one whose last layer does not reduce dimension, such as ViT-B/16 on ImageNet-1K? Please report the OpenOOD v1.5 ImageNet-1K benchmark (near- and far-OOD) against ViM, NECO, KNN, LogitNorm and a vision-language detector such as NegLabel. This is the result most likely to change my assessment.

Which versions of LSUN and TinyImageNet (crop or resize) were used, and what were the WideResNet depth and width? How were ODIN and SSD set up? Why is ODIN worse than MSP on iSUN and LSUN, when the ODIN paper reports the opposite?

2 comments
AuthorsOpenReview ↗

“Can NuSA be used with a large pretrained model, including one whose last layer does not reduce dimension, such as ViT-B/16 on ImageNet-1K? Please report the OpenOOD v1.5 ImageNet-1K benchmark (near- and far-OOD) against ViM, NECO, KNN, LogitNorm and a vision-language detector such as NegLabel. This is the result most likely to change my assessment.”

As far as limitations of NuSA, we state in the paper that NuSA is only applicable to architectures featuring a fully connected layer that reduces the dimensionality. The reason being that, assuming the weight matrix is full rank, the null space will always be trivial when the dimensionality is increasing through the weight matrix. In general, as long as NuSA can be applied to an appropriately sized fully connected layer there is no other limitation as to which architectures it can be used on. We acknowledge that the datasets used were introduced under earlier experimental settings. Nevertheless, our purpose was to evaluate the ability of the proposed method to perform outlier detection, and that was achieved through this analysis, which shows our methods competitive performance. We appreciate this feedback and will incorporate more datasets and models for a more extensive evaluation.

I would expect to have more experiments to resolve my concerns, which do not exist in the rebuttal. Thus, I will not raise my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.