acceptodds
Under review as a conference paper at ICLR 2027

Prior to Memory: Bayesian Assignment Projection for Continual Generalized Category Discovery

Abstract

Continual generalized category discovery (C-GCD) requires discovering novel classes from sequentially arriving unlabeled data containing both old and novel classes while retaining recognition of previously learned classes. However, parameter updates under fixed pseudo-labels do not automatically revise sample assignments, allowing initial assignment errors to potentially persist in category memories. We propose Bayesian Assignment Projection (BAP), which jointly refines old/new separation and assignments among novel classes before memory construction. Assuming that the scale of per-class sample counts transfers from base to novel classes, BAP uses base-class counts to construct Bayesian predictions of novel-class sizes, converts predictive intervals into marginal assignment constraints, and optimizes assignments by alternating regularized information projection with auxiliary Gaussian posterior estimation. Under the stated conditions, our theoretical analysis establishes the uniqueness of the fixed-cost projection solution and guarantees a non-increasing auxiliary objective under exact alternating updates within a fixed stage. With identical feature representations, data streams, and Gaussian memory models, experiments over multiple initializations show that BAP outperforms the baseline in mean final overall recognition accuracy on all four standard benchmarks, with an average gain of 2.68 percentage points across datasets.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Generalized Category Discovery, Continual Learning, Bayesian Modeling, Information Projection, Optimal Transport

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nM5EOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Bayesian Assignment Projection (BAP) for continual generalized category discovery, aiming to correct inaccurate sample assignments before class memory construction. It combines Bayesian class-size constraints derived from base-class statistics with local geometric evidence through alternating assignment projection and Gaussian estimation. Experiments on four benchmarks demonstrate improved recognition performance, achieving an average gain of 2.68 percentage points over VB-CGCD across multiple initializations.

Critical strengths
  1. The paper addresses error accumulation in C-GCD by explicitly revising sample assignments before memory construction. The integration of Bayesian cardinality constraints with geometric evidence is well-motivated and supported by theoretical analysis.
  2. BAP improves overall recognition accuracy across four standard C-GCD benchmarks under controlled comparisons. The ablation studies further demonstrate the benefits of cardinality constraints and alternating assignment refinement, supporting the effectiveness of the proposed approach.
Critical weaknesses
  1. BAP relies on comparable observation scales and shared class-count statistics between base and novel classes. These assumptions may not hold in realistic long-tailed or imbalanced scenarios, potentially limiting its generalizability. The performance degradation under class-dependent subsampling further highlights this concern.
  2. The proposed framework assumes that the number of new classes is known at each incremental stage. This restricts its applicability to realistic open-world scenarios where novel categories emerge without prior knowledge, weakening its practical significance.
Questions for authors
  1. Clarify how BAP handles violations of the shared class-count assumption, particularly under long-tailed distributions or class-dependent sampling. Explain the observed performance degradation on CUB and its implications for robustness.
  2. Clarify how strongly BAP depends on knowing the exact number of novel classes at each stage and whether inaccurate estimates would significantly affect assignment refinement and memory construction.
Reviewer_NTpMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Bayesian Assignment Projection (BAP) to address the problem of pseudo-label errors affecting category memory in Continual Generalized Category Discovery (C-GCD). BAP leverages sample counts from historical classes to predict reasonable sizes for novel classes and combines feature similarity with alternating optimization of information projection and Gaussian parameters to refine sample assignments, ultimately constructing more accurate category memory.

Critical strengths

The proposed method is interesting. The experimental analysis is comprehensive, and the method achieves promising performance across multiple benchmark datasets.

Critical weaknesses
  1. The proposed method relies on a very strong prior assumption that historical and novel classes have similar sample-count distributions. However, this assumption may not hold in real-world scenarios, particularly under imbalanced or long-tailed class distributions. Therefore, the performance improvements may heavily depend on specific dataset settings, raising concerns about the practical applicability and generalizability of the method.

  2. This strong assumption even deviates from the original C-GCD setting and introduces a substantial gap between the proposed method and realistic scenarios, further raising concerns about its practical relevance and generalizability.

  3. The method assumes that the number of novel classes is known in advance, limiting its practical applicability. Extending it to unknown class counts would require substantial methodological modifications.

  4. The method mainly combines established techniques, including Bayesian count estimation, information projection, and alternating Gaussian optimization. Its methodological novelty is therefore relatively limited.

Questions for authors

Given the strong assumptions underlying the proposed method, I would like to ask how sensitive BAP is to incorrect estimates of the number of novel classes. Extended method be to scenarios where the number of novel classes is unknown.

2 comments

Thank you for the additional experiments. However, my central concern remains unresolved. Using historical counts to constrain future novel-class counts introduces an additional structural prior not guaranteed by C-GCD; benchmark-specific count regularities should not be treated as intrinsic task properties. Furthermore, performance gains under prior mismatch do not validate this assumption. If incorrect hard constraints exclude the true assignments, improved accuracy alone does not resolve the potential inconsistency between the method’s statistical justification and its assignment behavior.

Thank you for further highlighting this issue. We acknowledge that shared-count constraints are not intrinsic to C-GCD. Their applicability depends on the data distribution and the observation conditions of historical and future classes. We will clarify this assumption in the revision and explain that incorrect hard constraints may exclude true assignments.

Even when the count prior is biased, BAP can still improve recognition through another mechanism: using historical class feature distributions and the geometry of current samples to revise which samples are used to learn novel classes and to update the novel-class models. This contribution cannot be explained solely by bringing class counts closer to their predicted values.

Specifically, initial clustering may scatter samples from a novel class across other classes or mix in samples from other classes, biasing the initial class statistics. BAP uses historical class distributions as a background reference while estimating each novel-class distribution, reassessing how well current samples match historical or novel classes. After each assignment update, we re-estimate the novel-class Gaussian parameters using the revised sample weights, then use the updated distributions to determine assignments in the next iteration. Changes in sample assignments can therefore refine class estimates, which in turn can help refine subsequent assignments.

This process has an explicit mathematical basis. In Eq. (6), novel-class parameters are determined by the weighted feature statistics of current samples; counts predicted from history do not directly determine class centers. Even if a class's assigned sample count remains unchanged, changing its constituent samples changes the matching costs in the next iteration. Equation (8) combines this modeling of sample features with assignment constraints in one objective. Proposition 2 shows that, under the stated conditions, alternating updates do not increase this auxiliary objective. This provides an optimization basis for repeatedly refining class statistics and sample assignments.

This also explains why gains may persist under class imbalance: even when count predictions are inaccurate, current features may still help recover misassigned novel-class samples, providing more useful training samples for the final class memories. In the D3 imbalance experiment, the three seeds yielded 642, 696, and 303 net assignment corrections for novel classes with fewer samples, relative to their respective initial assignments. The final average recognition accuracy of these classes increased from 54.53% with VB to 63.69% with BAP. These diagnostics are consistent with the mechanism described above. Incorrect count constraints also introduce assignment errors, however, so these gains should not be regarded as validation of the shared-count assumption.

The specific contribution of this work is to use historical information and current sample evidence to jointly refine old/novel separation, partitions among novel classes, and class statistics before class memories are constructed. When count information is reliable, it can work together with geometric evidence. When count information is biased, subsequent refinement may still preserve recognition gains by improving novel-class sample assignments and class estimates.

Reviewer_kAAEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses Continual Generalized Category Discovery (C-GCD), where models must discover novel classes from sequentially arriving unlabeled data while retaining knowledge of previously learned classes. It proposes Bayesian Assignment Projection (BAP), a framework that uses Bayesian predictions of novel-class sizes to refine sample assignments before constructing class memories, thereby reducing the propagation of pseudo-label errors. The method combines constrained information projection with Gaussian posterior estimation and provides theoretical guarantees on solution uniqueness and monotonic improvement of an auxiliary objective under specified conditions.

Critical strengths

Bayesian predictive intervals as assignment constraints is a novel aspect of the paper.

Critical weaknesses

Novelty is low, and the work seems like a combination of existing techniques. Ablations that really showcase the novel aspects are missing or incomplete.

This paper seems closely related. It is necessary to cite and compare.: Z. Li, B. Dai, C. Meinel and H. Yang, "Otem-IGCD: An Optimal Transport-based EM Framework for Imbalanced Generalized Category Discovery," 2024 International Joint Conference on Neural Networks (IJCNN), Yokohama, Japan, 2024, pp. 1-9, doi: 10.1109/IJCNN60899.2024.10651384. keywords: {Visualization;Systematics;Neural networks;Generalized Class Discovery;imbalanced distribution;optimal transportation;EM},

Questions for authors
  1. Does the count assumption hold for future classes (especially in realistic streams) ? 
BAP assumes base and future novel classes follow a shared count-rate model. But base classes may be balanced while a future class is rare, or classes may be observed at different rates. For example, base classes may have 500 images each while a new class has only 20. 
How does BAP avoid forcing such a class toward an incorrect size?

  2. Can global size constraints override the local evidence from individual samples? 
BAP combines class-size constraints with feature similarity. If the feature representation itself is wrong, the local costs (geometric evidence) may not identify the correct class. 
Could a class-mass correction (global constraint) then move a sample away from its true class just to satisfy the expected counts?

  3. Feature representation when different novel classes are overlapping
Since BAP relies on feature similarity to assign samples, how well does it work when the learned features cannot clearly separate two novel classes? Could a poor feature representation lead to incorrect assignments even when the optimization converges?


  4. What do the optimization guarantees mean for the final assignments? 
Propositions 1 and 2 establish assignment projection uniqueness and non-increasing objective values for exact within-stage updates under stated conditions. However, this decreasing objective does not necessarily mean the labels are correct. So, even if BAP finds the best solution according to its objective, how do we know it has actually grouped the samples into the correct classes?

  5. How often does candidate search over the different clustering strategies improve the actual assignments?
Since BAP tries different starting clusterings and picks the one with the lowest objective, how often does this actually lead to more correctly assigned samples compared to using just the initial clustering?

  6. How much does the Bayesian projection add beyond simpler methods? Do extra components actually really help ?
VB-CGCD already uses self-corrective relabeling, while other methods use class priors or various assignment constraints. In Table 4(c), the plug-in Poisson and row-argmax variants score 0.10 and 0.25 points higher than full BAP on C100. It would help to report results across more seeds with variation to check whether the full method consistently beats these simpler alternatives.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Additional Evidence

We thank all reviewers for their comments and suggestions. During rebuttal, we added experiments with multiple random seeds and assignment diagnostics to further examine the effectiveness and applicability of our method.

Our contribution is to turn historical class-count patterns into predictive constraints for discovering future novel classes, jointly refining old/novel separation and partitions among novel classes before memory construction. In continual category discovery, current assignments determine the class memories used in subsequent stages, so assignment errors can persist across the learning process. BAP uses base-class counts to predict individual novel-class counts and their total, combines these predictions with image evidence to optimize sample assignments, and alternates assignment refinement with class-statistics updates. Historical count information thus helps determine both which samples enter novel-class memories and how they are partitioned into different novel classes, directly connecting assignment refinement to continual memory construction.

The additional evidence shows that these gains extend to settings with biased count predictions. Under the standard, reduced-count, and class-imbalanced CIFAR-100 settings, BAP improves over VB by an average of 4.54%, 4.25%, and 2.82 %, respectively, with gains under all three random seeds in every setting. The gains also benefit novel classes: final accuracy on the 50 cumulatively introduced novel classes improves by 7.27% in the standard setting, while accuracy on novel classes with fewer samples improves by 9.16 % in the imbalanced setting. Assignment diagnostics further show 1,272–2,078 net corrections per seed across the five incremental stages in the standard setting, directly supporting the value of refinement before errors enter memory.

The available evidence supports two settings for understanding BAP's applicability: historical and future classes have reasonably comparable observation scales; or sample counts shift or classes are imbalanced, while the features still provide useful discriminative information. Gains in the first setting also occur in challenging recognition tasks: in the original manuscript's standard CUB fine-grained experiment, overall accuracy increases from 64.76% to 66.98%. The second setting is supported by the CIFAR-100 experiments above. The high retention of correct novel-class assignments made with high confidence under imbalance also shows that reliable sample evidence continues to play a role.

To address the main concerns, we distinguish the experimental protocol from the count assumption: historical counts come from base-class labels already available in C-GCD, and true sample counts of future classes are not provided to the prediction process. Component experiments further clarify the calibration role of Bayesian prediction and the constraint-preserving role of joint hard assignment. The CUB subsampling and class-number sensitivity experiments clarify the boundaries under fine-grained conditions with few samples, severe count mismatch, and underestimation of the number of classes. We will add these results and conditions to the revised manuscript to make the connection between the core contribution, empirical gains, and applicability boundaries clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.