acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Generalized Category Discovery

Abstract

Test-Time Adaptation (TTA) and Generalized Category Discovery (GCD) are traditionally treated as disjoint problems: the former adapts models to domain shift assuming all test classes are known, while the latter discovers novel categories assuming labeled training data for known classes. However, real-world deployment rarely fits either setting. Motivated by this gap, we introduce **Test-Time Generalized Category Discovery (TT-GCD)**, a unified and more realistic scenario where a vision-language model must adapt to distribution shifts, classify known categories using only textual supervision, and discover novel categories, all during test time and without access to labeled data. To address this challenging scenario, we propose **PACT** (**P**rototype **A**ssignment for **C**ategory discovery at **T**est time), a fully unsupervised framework that casts known-class recognition and novel-class discovery via prototype assignment. PACT first re-aligns shifted visual features with the text-derived class representations of the VLM using confident zero-shot predictions. Known and novel categories are then both represented by prototypes in the visual embedding space, estimated from the unlabeled test stream, and each test image is assigned to the category whose prototype is most similar to its visual feature. Extensive experiments across corruption and domain-shift benchmarks demonstrate that PACT outperforms adapted state-of-the-art TTA and GCD methods, effectively bridging the gap between adaptation and discovery.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Generalized category discovery, Test-time adaptation, Vision-language models, Distribution shift

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cmSWOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces the Test-Time Generalized Category Discovery (TT-GCD), which unifies test-time domain adaptation, known-class recognition, and unknown-class discovery, and proposes PACT(Prototype Assignment for Category discovery at Test time), which addresses this problem through confidence-weighted feature realignment and unified visual prototype assignment, achieving significant improvements on five benchmarks.

Critical strengths

A meaningful and practically relevant problem formulation. This paper defines a new, real-world task and introduces Test-Time Generalized Category Discovery (TT-GCD), which effectively addresses distribution shifts, known-class recognition, and new-class discovery under fully unsupervised test-time conditions. PACT combines lightweight, Layernorm-based feature realignment, unified visual prototype assignment, and online updates to provide a simple and effective solution for this challenging setting. The PACT method proposed in this paper has been extensively tested on five benchmarks, demonstrating significant performance improvements and clearly outperforming the current state-of-the-art TTA and GCD methods. With a processing time of just 128 samples per batch, it outperforms other comparison models in terms of both performance and efficiency.

Critical weaknesses

I am particularly interested in the implementation details and theoretical foundations of the paper. Equation (8) assigns samples to their nearest prototypes, whereas Equations (9)–(10) use different criteria to update known and novel prototypes. The paper does not clearly explain how samples with conflicting zero-shot predictions and prototype assignments are handled.

Questions for authors

Could the authors provide more detailed implementation information regarding the prototype assignment and update procedures in Equations (8)–(10)?

1 comment

We thank the reviewer for the positive assessment. We are encouraged that the reviewer finds TT-GCD "a meaningful and practically relevant problem formulation", and PACT "a simple and effective solution for this challenging setting" that outperforms the compared methods "in terms of both performance and efficiency". We address the requested details below.

W1 & Q1: Assignment and update procedures of Eq. (8)–(10). The two update rules read different signals by design, and we now state this explicitly in the revised manuscript. For each incoming batch: (i) every sample is assigned by Eq. (8) to the most similar prototype among the active Known prototypes μk\mu_k and the Novel prototypes νl\nu_l jointly, and all assignments are computed before any update; (ii) each Known prototype μk\mu_k is updated by Eq. (9) from the samples whose zero-shot prediction (Eq. 1, re-aligned encoder) is class kk, weighted by their confidence, with rate 0.05; (iii) each Novel prototype νl\nu_l is updated by Eq. (10) from the samples that Eq. (8) assigned to it, with rate 0.10.

A conflicting sample, whose zero-shot prediction is Known class kk and whose nearest prototype is νl\nu_l, is predicted as Novel category ll, contributes to μk\mu_k with its confidence weight, and contributes to νl\nu_l. There is no arbitration between the two rules. The Known prototypes are the only component tied to a class name, and updating them from the text prediction keeps each μk\mu_k anchored to its name. Updating them from their own nearest-prototype assignments would instead let a Known prototype drift towards a neighbouring Novel cluster, with no signal to pull it back. The Novel prototypes have no text, so they follow the feature geometry alone.

On the theoretical foundations, Eq. (8) is the MAP assignment under a mixture of von Mises–Fisher components with shared concentration and uniform priors [r1], and Eqs. (9)-(10) are the corresponding online spherical k-means updates, where each prototype is an exponential moving average of the normalized mean of its supporting samples.

[r1] Banerjee et al., Clustering on the unit hypersphere using von Mises-Fisher distributions, JMLR'05.

Reviewer_J3j5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Test-Time Generalized Category Discovery (TT-GCD), which aims to recognize known classes and discover novel ones from unlabeled test data under distribution shift. The authors propose PACT, which combines feature re-alignment with online prototype-based classification and clustering. Experiments on five benchmarks show improvements over adapted TTA and GCD baselines, with ablation studies supporting the main design choices.

Critical strengths

The paper addresses a practical problem by considering domain shifts and unseen classes together, which existing TTA and GCD methods typically handle separately. PACT is also simple and efficient, with strong results across several benchmarks, particularly DomainNet.

Critical weaknesses
  1. PACT does not clearly separate known and novel samples during adaptation. Novel samples are assigned pseudo-labels from known classes during feature re-alignment, and even samples assigned to novel prototypes can still contribute to updating known prototypes. This may introduce errors into both stages, but the paper does not examine how much this affects performance.

  2. The main experiments use the ground-truth number of novel classes, although this information is assumed to be unavailable at test time. While the authors also report results with an estimated class count, it is unclear whether the estimation follows the online setting or how sensitive the method is to estimation errors.

Questions for authors
  1. How much do incorrectly pseudo-labeled novel samples affect feature re-alignment and known-prototype updates? Have the authors tested whether filtering these samples improves performance?

  2. In Section 5.3, is the number of novel classes estimated using only the initial buffer or the entire test stream? How sensitive is PACT to errors in the estimated class count?

2 comments

We thank the reviewer for the positive assessment and the precise reading of the method. We are glad the reviewer finds that the paper "addresses a practical problem" by considering domain shifts and unseen classes together, and that PACT is "simple and efficient, with strong results across several benchmarks". We address the weaknesses and questions jointly below (W & Q).

W1 & Q1: Known and novel samples during adaptation. The reviewer describes PACT accurately: novel samples do receive pseudo-labels from known classes during feature re-alignment, and samples assigned to novel prototypes can still contribute to updating known prototypes. This design is deliberate. PACT is fully unsupervised, so no labels and no reliable way to separate known and novel samples are available at test time. A filter would have to discard samples based on an uncertain decision, and it would remove known samples together with novel ones. PACT therefore keeps every sample and scales its contribution by the confidence weight of Eq. 2. As the pseudo-labels are computed once using the pretrained parameters and kept fixed, they are never refreshed from the adapting encoder, so the error accumulation associated with confirmation bias [r1] is limited.

To measure how much the incorrectly pseudo-labeled novel samples affect both stages, we thank the reviewer for the suggestion and we have run two experiments. First, an oracle filter removes every novel sample from feature re-alignment and from the known-prototype updates. As it uses the ground-truth split, it is an upper bound on what filtering out novel samples could achieve. Second, we remove the confidence weight, so that every sample contributes equally.

PACT (%)CIFAR-100-C: All / Known / NovelDomainNet: All / Known / Novel
PACT29.35 / 33.86 / 24.8446.41 / 53.56 / 39.36
Without confidence weight28.73 / 32.65 / 24.8045.54 / 52.52 / 38.62
Oracle filter30.51 / 34.99 / 26.0349.45 / 56.67 / 42.32

Without any separation of known and novel samples, PACT is within 1.16 and 3.04 points of the oracle. Removing the confidence weight lowers All by 0.62 and 0.87 points. On CIFAR-100-C the loss is almost entirely on Known (−1.21 against −0.04), as the weight acts on the known-prototype estimates, and on DomainNet it is spread over both (−1.04 and −0.74).

[r1] Arazo et al., Pseudo-labeling and confirmation bias in deep semi-supervised learning, IJCNN'20.

W2 & Q2: Buffer or full stream, and sensitivity to the estimated count. The main experiments use the true number of novel classes, following standard GCD practice [r2, r3], and Section 5.3 studies the case where it is estimated. Section 5.3 as submitted uses the full-stream estimate. We now also report the strictly online estimate computed on the initial buffer only. To measure the sensitivity, we set L̂ to a fixed multiple of the true L.

L̂CIFAR-100-C: All / Known / NovelDomainNet: All / Known / Novel
0.50 × L27.52 / 35.17 / 19.8644.37 / 54.81 / 34.02
0.75 × L28.81 / 34.58 / 23.0445.94 / 54.17 / 37.80
True L29.35 / 33.86 / 24.8446.41 / 53.56 / 39.36
1.25 × L29.38 / 33.00 / 25.7646.89 / 53.70 / 40.17
2.00 × L29.08 / 31.63 / 26.5346.56 / 51.61 / 41.59
Buffer-only L̂27.83 / 34.74 / 20.9245.02 / 53.53 / 36.60

When the count is estimated from the initial buffer alone, PACT still outperforms every baseline of Tables 2-4, all of which are given the true count, on all five datasets, 53.27 against 39.48 on CIFAR-10-C, 27.83 against 19.79 on CIFAR-100-C, 18.23 against 16.93 on ImageNet-C, 45.02 against 22.56 on DomainNet and 17.30 against 13.74 on CUB-C.

[r1] Arazo et al., Pseudo-labeling and confirmation bias in deep semi-supervised learning, IJCNN'20. [r2] Vaze et al., Generalized category discovery, CVPR'22. [r3] Wen et al., Parametric classification for generalized category discovery: A baseline study, ICCV'23.

Reviewer_kmcEOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a test-time category discovery setting in which a model must recognize known classes and discover novel classes in an unlabeled, distribution-shifted stream. PACT adapts the visual encoder using an initial buffer, builds visual prototypes for known and novel classes, and updates those prototypes as new samples arrive.

Critical strengths

PACT achieves strong results across corruption, natural domain-shift, and fine-grained benchmarks. The ablations show that encoder re-alignment and visual prototypes both contribute to performance. The paper also evaluates an estimated novel-class count in Figure 4; PACT remains the best reported method on the four benchmarks shown there.

Critical weaknesses
  1. The main result tables provide the true number of novel classes, although this number is unknown in the proposed task. Figure 4 addresses estimated class counts, but does not clearly state whether estimation uses only samples observed so far. On ImageNet-C, the estimated count is 113 rather than 500, and PACT’s lead over the strongest baseline narrows to about 1.6 points. Estimated-count results are also absent for CUB-C.
  2. The Known metric does not fully test recognition by known class name. The paper reports Known, Novel, and All accuracy after one Hungarian matching over the full stream. This is a standard GCD evaluation, but it can remap known-class labels. Since known class names are available in this task, accuracy without such remapping would provide important additional evidence.
  3. The adapted baselines use different encoder-update rules. The TTA baselines update LayerNorm parameters, the GCD baselines keep the backbone frozen, and PACT performs an initial encoder re-alignment. These choices may be reasonable given the original methods’ different assumptions, but their effect on the comparison is not fully explained.
Questions for authors

Regarding weakness 1: Which samples are used to estimate the novel-class count in Figure 4? Is the estimate made before the rest of the stream arrives? Could the authors clarify why CUB-C is omitted and, if available, report results when the count is estimated using only the initial buffer? Regarding weakness 2: Why is Hungarian matching applied to the Known metric when known class names are available? Do the authors have known-class top-1 accuracy without label remapping, alongside clustering accuracy for novel classes? Regarding weakness 3: Why do the adapted GCD baselines keep the backbone frozen while PACT updates the encoder? Have the authors examined whether giving those baselines a comparable initial encoder update changes the performance gap? If so, the results would help clarify the source of PACT’s advantage.

5 comments

We thank the reviewer for the thorough and constructive review. We are encouraged that the reviewer finds that PACT achieves "strong results across corruption, natural domain-shift, and fine-grained benchmarks", and that the ablations show that encoder re-alignment and visual prototypes "both contribute to performance". We address the weaknesses and questions jointly below (W & Q).

W1 & Q1: Estimated novel-class count, online estimation, and CUB-C.

We agree, and we state this plainly. Our main tables use the true number of novel classes, following standard GCD practice [r1, r2]. The estimate of Figure 4 follows the search procedure of [r1] and is computed over the whole test stream, so it is not an online estimate. We thank the reviewer for this suggestion and we have run the corresponding experiment, where the same estimator is applied to the initial buffer alone (W = 1024), before the rest of the stream arrives, and kept fixed afterwards. We also include CUB-C which was missing from Figure 4.

PACTTrue LGiven L: All (%)Buffer-only: L̂Buffer-only: All (%)Full-stream (Fig. 4): L̂
CIFAR-10-C556.674.753.274.6
CIFAR-100-C5029.1129.427.8326.9
ImageNet-C50022.16144.718.23113.3
DomainNet17346.41123.345.02135.3
CUB-C10018.2059.717.3034.6

All is the accuracy under the evaluation of the paper (%), and L̂ is the estimated number of novel classes, averaged over the shifts of each dataset. On ImageNet-C, the lead of PACT narrows when the count is estimated. ImageNet-C is where the estimator under-counts most (145 with the buffer and 113 with the full stream, against 500), and halving the count already costs about 2 points on CIFAR-100-C and DomainNet (Please see Reviewer J3j5, W2 & Q2 for complete progression of estimated L̂). This comparison also gives the true count to the baselines only. Even with the buffer-only estimate, PACT remains above every baseline of Tables 2–4, all of which are given the true count, on all five datasets, including ImageNet-C (18.23 against 16.93 for HiLo++).

W2 & Q2: Known accuracy without Hungarian remapping. We thank the reviewer for this thoughtful observation. We used Hungarian matching over the full stream for two reasons. First, known-class top-1 accuracy without remapping rewards assigning samples to known class names, since a novel-class sample assigned to a known name does not reduce it. A method that assigns every sample to a known class name obtains the highest known-class accuracy while discovering no novel class, which is close to the behaviour of the TTA and open-set TTA methods that TT-GCD is designed to move beyond (Sec. 1). Second, as the reviewer notes, it is the standard GCD evaluation when known and novel classes are mixed [r1, r2], and it guarantees that each predicted category is credited for one class only. That said, we agree that accuracy without remapping is well defined in our setting and provides important additional evidence, and we report it below for all methods. Known is the known-class top-1 accuracy without label remapping, where a known-class sample is correct only if it is assigned to the prototype of its own class name. Novel is the clustering accuracy for novel classes, with Hungarian matching over the novel prototypes only.

All / Known / Novel (%)CIFAR-10-CCIFAR-100-CImageNet-CDomainNetCUB-C
TENT++35.59 / 58.47 / 12.7115.48 / 26.29 / 4.689.50 / 18.81 / 0.2020.25 / 39.56 / 1.267.85 / 14.55 / 1.15
BATCLIP++33.22 / 58.06 / 8.3816.44 / 28.07 / 4.8011.75 / 18.97 / 4.5316.68 / 32.74 / 0.899.18 / 15.40 / 2.96
SAT++34.38 / 59.21 / 9.559.50 / 13.97 / 5.025.34 / 10.47 / 0.206.46 / 11.84 / 1.185.78 / 9.26 / 2.30
OSTTA++41.70 / 68.53 / 14.8718.35 / 32.12 / 4.5810.63 / 21.06 / 0.2019.67 / 38.36 / 1.307.26 / 13.25 / 1.28
UniEnt++41.69 / 70.02 / 13.3618.16 / 31.79 / 4.549.07 / 17.94 / 0.2020.90 / 40.90 / 1.256.12 / 10.85 / 1.39
SimGCD++28.02 / 50.46 / 5.5811.99 / 22.40 / 1.588.58 / 13.72 / 3.449.24 / 15.41 / 3.068.88 / 4.69 / 13.07
HiLo++14.98 / 0.00 / 29.974.95 / 0.00 / 9.9011.25 / 12.58 / 9.933.88 / 0.14 / 7.574.44 / 0.19 / 8.70
PACT55.79 / 64.72 / 46.8527.34 / 33.04 / 21.6416.12 / 16.57 / 15.6844.38 / 54.55 / 34.3615.18 / 18.69 / 11.68

Evaluating known and novel classes separately makes the two abilities required by TT-GCD visible, and confirms the first point above. The TTA baselines, which are designed to recognise the known classes, keep high Known accuracy, and several of them are above PACT on Known on CIFAR-10-C and ImageNet-C. Their Novel accuracy, however, falls to between 0.20 and 5.02% on the four larger datasets.

The GCD baselines fail for a different reason. In their original setting, a supervised loss on labeled known-class data ties each known head to its class and sets the balance between known and novel predictions. In TT-GCD, this supervision is replaced by zero-shot pseudo-labels from the shifted stream. SimGCD++ leans towards the known classes on four datasets (50.46 Known against 5.58 Novel on CIFAR-10-C) and towards the novel ones on CUB-C (4.69 against 13.07). HiLo++ collapses entirely to the novel side. Its robustness to domain shift relies on fine-tuning the backbone with labeled source data, which TT-GCD does not include. Without this, it assigns almost every sample to a novel prototype. No known-class sample on CIFAR-10-C or CIFAR-100-C is assigned to a known class name. Its Known accuracy is therefore close to zero on these four datasets. Its Novel accuracy is nonetheless the highest among the baselines on every dataset except CUB-C.

[r1] Vaze et al., Generalized category discovery, CVPR'22. [r2] Wen et al., Parametric classification for generalized category discovery: A baseline study, ICCV'23.

W3 & Q3: Encoder-update rules of the baselines. The update rule of each baseline follows from what its original method can use at test time. The TTA methods [r3–r7] adapt normalization-layer parameters online from unlabeled batches, which on a ViT backbone are the LayerNorm affine parameters, so they keep their original rule. SimGCD [r2] and HiLo [r8] instead update their backbone with a supervised loss on labeled Known-class data, which TT-GCD does not provide. We therefore kept their backbone fixed and trained only their classifier. We thank the reviewer for the suggested experiment, which we have run. PACT's image–text re-alignment (Sec. 4.1) is applied before each GCD baseline below:

All (%)SimGCD++: paper → re-alignedHiLo++: paper → re-alignedPACT
CIFAR-10-C28.49 → 26.4016.63 → 32.1156.31
CIFAR-100-C15.20 → 8.225.69 → 15.1129.11
ImageNet-C14.07 → 16.3016.93 → 18.8122.20
DomainNet12.44 → 12.025.37 → 19.9646.89
CUB-C13.74 → 6.375.83 → 10.2918.20

The re-alignment transfers to HiLo++, which improves on every dataset (+1.88 to +15.48), whereas SimGCD++ gets worse as its projection head is trained on the frozen features. With the identical initial encoder update, the best GCD baseline still trails PACT by 24.2, 14.0, 3.4, 26.9 and 7.9 points. Conversely, PACT without re-alignment, i.e., with the same frozen encoder as the GCD baselines in the paper, already reaches 23.08 on CIFAR-100-C and 42.13 on DomainNet (Table 5), which is above both the original and the re-aligned GCD baselines. The advantage of PACT therefore comes mainly from the unified visual-prototype assignment, and re-alignment is a complementary component that can also benefit other methods.

[r1] Vaze et al., Generalized category discovery, CVPR'22. [r2] Wen et al., Parametric classification for generalized category discovery: A baseline study, ICCV'23. [r3] Wang et al., Tent: Fully test-time adaptation by entropy minimization, ICLR'21. [r4] Maharana et al., BATCLIP: Bimodal online test-time adaptation for CLIP, ICCV'25. [r5] Mishra et al., Semantic anchor transport, 2026. [r6] Lee et al., Towards open-set test-time adaptation utilizing the wisdom of crowds in entropy minimization, ICCV'23. [r7] Gao et al., Unified entropy optimization for open-set test-time adaptation, CVPR'24. [r8] Wang et al., HiLo: A learning framework for generalized category discovery robust to domain shifts, ICLR'25.

Comments on the paper

1 comment
Area_Chair_eRruOpenReview ↗

Urgent Reminder: Reviewer Discussion Deadline

Dear Reviewers,

This is a friendly but urgent reminder that the discussion period will end on November 18, 2026. Please carefully review the authors' rebuttals and actively participate in the ongoing discussions. If you have not yet responded to the authors or initiated a discussion, please do so as soon as possible. In particular, reviewers are expected to address the authors' clarifications and responses, provide constructive follow-up feedback, and update their assessments and scores when appropriate. Please ensure that your final evaluations reflect your independent judgment and take into account the information provided during the rebuttal and discussion process. Active and timely participation is essential to ensuring a fair, thorough, and constructive review process.

Thank you for your time, effort, and cooperation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.