acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Error Minimization for Open Set Recognition

Abstract

The success of supervised learning largely stems from explicit error minimization. In contrast, Test-Time Adaptation lacks ground-truth labels, precluding direct optimization. This work enables explicit error minimization for Test-Time Open-Set Recognition (TOSR). Specifically, we identify a previously overlooked fact: under a fixed recall, the challenging objective of Test-Time Error Minimization is equivalent to the tractable objective of Negative Prediction Maximization. Based on that, our algorithm learns a set of discrete boundaries and searches for their ensemble to maximize negative predictions. Experiments demonstrate that our method successfully closes around 50% of the performance gap between TOSR methods and supervised learning on several benchmarks.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
open set recognition, test-time adaptation, out-of-distribution detection, novelty detection, anomaly detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6FKDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Test-Time Error Minimization (TTEM) for test-time open-set recognition (TOSR), where models must distinguish known from unknown classes using an unlabeled test stream. The central insight is that, under a fixed known-class recall, minimizing the total classification error is equivalent to maximizing the number of negative predictions. Based on this observation, the authors introduce a discrete-boundary ensemble that learns candidate decision boundaries using kernel SVMs, greedily selects boundaries to increase negative predictions, and combines them through an OR rule. An adaptive detector further switches between a prior detector and the test-time detector to mitigate the cold-start problem. Experiments verify the advantages of the proposed method.

Critical strengths
  1. The idea of reformulating error minimization as negative prediction maximization under a fixed known-class recall is interesting.
  2. The method achieves strong results on several benchmarks, particularly with CLIP features, and also works with conventional backbones.
Critical weaknesses
  1. The main concern is whether the surrogate objective is valid in the actual test-time setting. The equivalence requires the same known-class recall on the test stream, but the method calibrates recall on a held-out validation set. The paper does not establish why this calibration should remain reliable on unlabeled test data, especially under distribution shift. The reported results are encouraging, but do not resolve this issue.
  2. The threshold calibration in Eq. (5) appears inconsistent with the stated goal of retaining 99.9% of known-class samples. Given the decision rule in Eq. (6), the use of the upper Gaussian quantile may lead to the opposite behavior. The authors should clarify this discrepancy and explain how the target recall is maintained by the final ensemble.
  3. TTEM relies on the prior detector to filter candidate unknown samples for boundary learning, potentially inheriting its detection biases. Unknown patterns missed by the prior detector may therefore remain underrepresented during adaptation. The authors should evaluate the robustness of TTEM to prior detector quality and clarify whether it can effectively discover unknown patterns beyond those identified by the initial detector.
  4. The computational cost is not sufficiently analyzed. The method repeatedly trains kernel SVMs and reranks boundaries using accumulated prediction records, yet provides no systematic complexity analysis or detailed runtime comparison. The memory budget alone does not establish computational efficiency, especially since the paper acknowledges that adaptation is relatively slow.
Questions for authors
  1. Under what assumptions does validation-set recall calibration transfer to the unlabeled test stream, and how is the resulting uncertainty quantified?
  2. Does the number of negative predictions reliably track actual test error under known-class distribution shifts or non-stationary test streams?
  3. How are the boundary thresholds and ensemble size (K) calibrated? Please clarify the apparent quantile-direction issue in Eq. (5) and explain how recall is maintained by the final detector.
  4. What are the runtime and computational costs of SVM training, prediction-record maintenance, and greedy boundary ranking as the test stream grows?
Reviewer_SnKJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tackles the problem of test-time open-set recognition, where models are allowed to use test-time samples to improve performance. The authors establish a theoretical relationship between error minimization and negative prediction maximization under fixed know-class recall. Based on this, they propose to learn discrete decision boundaries between known classes and unknown using SVMs, and greedily ensemble them to maximize negative predictions. Experiments on different datasets and backbones show the effectiveness of the proposed method.

Critical strengths
  1. The proposed method is intuitive and simple. Learning discrete boundaries between known and unknown classes is reasonable.
  2. The experiments on different benchmarks show the effectiveness of the proposed method, achieving impressive improvements over existing testing-time OSR methods.
Critical weaknesses
  1. While the authors establish a theoretical connection between error minimization and negative prediction maximization, this theoretical connection only applies under a fixed known-class recall. However in test time, it may not be practical to know in advance whether the detectors actually achieve the same recall as the ground truth labels are not available. The authors calibrate detector thresholds on a held-out known-class validation set, but the final detector ensembles many boundaries and the recall on test data may not stay at 95%. If the recall is not fixed, maximizing negative predictions may instead lead to rejecting more samples, including known ones. It would be helpful to clarify if this issue exists and if now, how the implementation solves this problem.

  2. The authors claim that the proposed method closes about 50% of the gap to supervised learning, but the experiment results do not fully support this. For example, on CUB and SVHN, the gap reduction is only about 2% and 28%. Moreover, the fully supervised reference is limited to a kernel SVM trained on labled test data, which may not sufficiently represent supervised performance. This comparison also may not be fair. The proposed method uses an ensemble of up to 300 kernel SVMs vs. the supervised reference uses a kernel SVM. A fairer reference would be the same ensemble trained with ground-truth test labels.

Questions for authors
  1. There are some inconsistencies between the notations. For example, below Eq. 5 it is stated that "γ is set to 99.9% to ensure that the boundary...", but in Sec. 4.1, it writes "the boundary hyperparameter γ is set to 0.001". Under Eq. 5, γ=0.001 would reject almost all known samples. The authors should clarify the definition of γ, as boundary calibration is important to the proposed method.

  2. The ablation studies can be more comprehensive to better justify the proposed method. For example, compare greedy boundary ensemble with single SVM boundary, or other ensemble baselines such as random/simple ensemble.

  3. The paper also does not report computation cost. Since the proposed method trains a kernel SVM every batch and stores them up to 300 boundaries, with re-running greedy ranking over the prediction history at each batch, the computation cost for this method, and possibly comparisons with previous methods, should be reported to give a better understanding of the performance and computation cost tradeoffs.

  4. There are some typos, e.g., negtive, potitive, conclution, etc. Please carefully correct all of them.

Reviewer_WoEjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The work proposes a test time adaptation method for open set recognition with a proxy metric of negative prediction minimization, where the algorithm learns a set of discrete boundaries and ensembles them to maximize negative predictions.

Critical strengths

The idea of adaptation for open set recognition during test time through a proxy metric for error minimization is quite interesting. Moreover, the idea of discrete boundary training with known class replay set and the test time batch seems also quite unique. The paper is generally well written.

Critical weaknesses

The work’s foundation stands on proposition 1 where they advocate to maximize the negative prediction under a fixed recall. However, later on, when they apply proposition 1, the fixed recall assumption does not appear to be in place or at least the connection with the fixed recall assumption is not clear. The work also has inadequate ablation studies with concerns about the experiment set ups. There is no report on the closed set performance. Moreover, the compute needed for the whole TT-OSR-A pipeline is not reported to analyze the applicability of the method during test time.

Questions for authors
  1. The TOSR problem should be clearly defined. The term “Test time error minimization” in Eq 2, is it only defined for the unknown recognition problem? Most often the OSR problem is coupled with a base closed-set problem, meaning that the classification problem for known classes should be unaffected during the unknown recognition process. The closed set performance is not reported.

  2. In Section 3.2, was it a soft margin SVM or a hard margin SVM train? It seems that as D- may have both knowns and unknowns, a soft margin SVM or an asymmetric margin SVM should be appropriate. If indeed a soft margin SVM was used, the kernel SVM formulation in Eq 4 should be checked again.

  3. The following design choices are not clear enough:

“K is determined according to the known-class recall.”

“We can remove the bottom-M boundaries”

  1. Does proposition 1 have any connection with precision?

  2. For Table 1 and Table 2, do the other methods also use the same CLIP/ resnet101 backbone? Otherwise, the improvements in Table 1 might be due to the CLIP features and there is no way to distinguish whether the improvements are because of the proposed method.

  3. Was the backbone for Table 2 trained/ finetuned before test time?

  4. Why is γ\gamma set to 99.9%?

  5. The ablation studies varying K, varying the model architectures for Table 3 and 4, Kernel SVM vs Linear classifiers, comparison of test-time adaptation compute are expected.

Comments on the paper

1 comment

We sincerely thank the Area Chair and all reviewers for their time and constructive feedback. Your insightful comments have been extremely helpful in identifying ways to improve the quality and clarity of our work. We would like to take this opportunity to clarify several concerns regarding the weaknesses raised during the review.

(1) How does the algorithm guarantee the recall on the test set?

One concern raised about our algorithm is how it guarantees the recall on the test set. We would like to point out that this problem has already been well addressed and that standard practices for handling it have been established in the OSR and OOD communities. Our method follows these established practices. A straightforward indication is that FPR@TPR95 is a standard evaluation metric in this field, where TPR95 explicitly refers to a recall of 95%. If controlling the recall were an unsolvable problem, the practical significance of the entire field would be questionable. Fortunately, since each algorithm produces a one-dimensional detection score, we only need to estimate the probability density in this one-dimensional space and determine an appropriate threshold. Density estimation in a one-dimensional space is a trivial problem. By simply constructing an empirical cumulative distribution function (CDF), we can readily achieve an estimation error of less than 1%, even without relying on any specific AI-based algorithms.

(2) How does the algorithm preserve the classification accuracy on known classes?

Another concern raised about our algorithm is how it guarantees the classification accuracy on known classes. We would like to point out that this problem has also been well addressed, with standard practices established in the OSR and OOD communities. Our method follows these established practices. Specifically, the standard approach is to keep the classification-related parameters fixed. This broad family of methods is commonly referred to as post-hoc methods.

(3) Computational efficiency.

Regarding computational efficiency, few algorithms can achieve breakthroughs in both performance and speed simultaneously. In most cases, techniques are progressively improved over time. We acknowledge that computational efficiency is not a strength of our current algorithm. We will provide more detailed runtime experiments in the revised version to offer readers a better understanding of the computational cost of our method.

Once again, we sincerely thank the Area Chair and all reviewers for their time and effort, and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.