acceptodds
Under review as a conference paper at ICLR 2027

CoMove: Evolving Supervision for Class-Incremental Learning

24 views

Abstract

Class-incremental learning must accommodate new classes while retaining knowledge supported by limited past data. As the classifier expands and replay becomes sparse, both the geometry of learning targets and the evidence supporting them change. We introduce CoMove, a method that adapts the geometry and allocation of supervision throughout learning. For progressive equiangular tight frame (ETF) classifiers, we characterize the displacement of inherited prototypes and derive a spherical transport that aligns teacher references with the expanded geometry while preserving rankings among old classes. Peer, prototype, and transported-teacher relations are integrated through a common objective with shared negative competition and source-specific responses. Class composition and replay support determine their contributions, transferring under-supported supervision to the teacher while preserving total query weight. A single allocation rule applies across datasets and replay budgets, without tuning source weights; inference uses the ETF directly. Experiments on CIFAR-10, CIFAR-100, and TinyImageNet demonstrate consistent improvements over competing methods across low-memory settings. CoMove achieves the highest mean final class-incremental accuracy in all six dataset–buffer settings, with gains over the best baseline in each setting averaging 4.69 percentage points, alongside substantially reduced forgetting. The same incremental objective also supports learning without exemplar replay. Our code is provided in the supplementary material.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
class-incremental learning, neural collapse, equiangular tight frames, knowledge distillation, memory replay, supervision allocation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_N6xXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoMove, a class-incremental learning method that jointly addresses teacher-reference mismatch caused by progressive ETF expansion and insufficient supervision support under limited replay. CoMove updates teacher references through spherical transport and integrates peer, prototype, and teacher supervision through a common objective with shared negative competition. Their contributions are allocated according to class composition and replay support. The paper establishes exact prototype alignment and old-label ranking preservation. Across three datasets and two replay budgets, CoMove achieves the highest mean final Class-IL accuracy in all six main comparisons, exceeding the best baseline in each setting by an average of 4.69 percentage points. The evaluation also includes exemplar-free experiments and component ablations.

Critical strengths
  1. CoMove improves accuracy across all six main comparisons and substantially reduces forgetting on CIFAR-100 and TinyImageNet. Using the same supervision-allocation rule across datasets and replay budgets adds practical value.

  2. The exact characterization of inherited-prototype displacement, together with teacher transport that requires no additional map fitting, provides a clear and verifiable explanation of how teacher references can be aligned with the expanded classifier geometry.

Critical weaknesses
  1. Möbius transport is theoretically, but I am not yet convinced that its additional geometric properties are important to CoMove’s empirical success. The simpler normalized affine transport performs comparably or better in several settings, and removing pooling does not consistently degrade performance. Furthermore, each core ablation uses only a single paired run, limiting the evidence supporting the necessity of these designs.

  2. The threshold in the replay coverage rule, ρi=min⁡(1,(nyi−1)/(K−1))\rho_i=\min(1,(n_{y_i}-1)/(K-1)), lacks sufficient justification, and the paper does not report a sensitivity analysis. In particular, when an old class has only one replay example, all positive prototype and peer supervision weight is transferred to the teacher. However, sparse replay support does not necessarily imply that the teacher is reliable.

  3. The introduction’s overall motivation is understandable, but the transition from general class-incremental learning to progressive ETF classifiers does not adequately explain why this framework is chosen. The transition from geometric transport to supervision allocation also needs a clearer connection to the motivating problems, making the necessity of the core designs less intuitive.

  4. The related work largely enumerates methods and describes local differences. It does not sufficiently synthesize the remaining research gaps or explain how addressing these gaps motivates the significance of the proposed method.

  5. The high-level workflow is clear, but Sections 4.2–4.3 combine several objective-design mechanisms in a dense presentation. The lack of a step-by-step explanation makes their necessity, interactions, and individual contributions difficult to understand.

  6. The manuscript contains numerous typesetting and layout issues. For example, the caption of Figure 2 exhibits abnormal interword spacing, the bold run-in heading around line 290 is awkwardly broken rather than starting as a new paragraph, and there are conspicuously large vertical gaps between paragraphs in several places. The authors should carefully proofread the manuscript line by line and correct these presentation issues, as they significantly detract from the professionalism and readability of the paper.

Questions for authors
  1. What motivates choosing K−1K-1 as the threshold in the replay coverage rule?

  2. Under matched initial-task training, initialization, and replay protocols, how much of CoMove’s improvement over ProNC remains? This would help distinguish gains due to initialization changes from those due to the proposed incremental-learning mechanisms.

2 comments
AuthorsOpenReview ↗

Response to N6xX (Part 1/2): Component evidence and initial-task training

We thank Reviewer N6xX for the feedback. We now report three-seed component ablations (Table 4) and controls from ProNC's initial-task training.

1. On teacher transport (Weakness 1)

The reviewer questions the empirical benefit of Möbius over normalized affine and requests repeated ablations.

Both continuations reduce forgetting, with comparable accuracy. Across three paired seeds in all six settings, normalized affine changes mean FAA by -0.22 to +0.25 points relative to Möbius. Both lower FF in all 18 pairs with raw teacher, which retains small FAA advantages on C100/500 and Tiny/500 (Table 5).

Both extend the same affine prototype map and coincide at inherited prototypes. Möbius adds tangent-angle preservation at comparable cost, motivating its use with similarity-based supervision. We agree that these results do not establish an accuracy benefit from conformality. General response 3/4, point 1 gives the formulas and timings.

On C10/200, transport's FAA benefit is 1.83 ± 0.39 points larger with CoMove responses than with logistic responses, positive on all three seeds (Table 16). This supports their joint use in this setting.

2. On pooling (Weakness 1)

The reviewer also notes that removing pooling does not consistently degrade performance.

The repeated results confirm a modest, dataset-dependent effect. Pooling adds 0.55 ± 0.36 / 0.28 ± 0.31 FAA points on C10 at buffers 200/500 (paired mean ± sample SD); C100/Tiny changes range from -0.21 to +0.05. The unpooled objective remains competitive. Table 20 and General response 4/4, point 2 give all pooling results; Table 10 covers every component and both evaluation protocols.

3. On initial-task training and initialization (Question 2)

The reviewer asks how much improvement over ProNC remains when initial-task training, initialization and replay are matched.

The gains persist: +3.51–8.34 FAA points over published ProNC in all six settings. The main comparison already shares the backbone, schedule, augmentation and replay policy. It differs in initial-task training: ProNC fits an ETF to learned features; CoMove trains toward a fixed ETF.

We now retain ProNC's learned encoder, fitted ETF and replay buffer, changing only the incremental objective. Across three seeds, full CoMove also lowers FF by 8.27–32.31 points (Tables 6 and 25). With raw references, FAA gains remain 3.61–8.61 points. Accuracy is comparable to or higher than with our own birth: the gains do not rely on that procedure. General response 2/4, point 2 details the control.

AuthorsOpenReview ↗

Response to N6xX (Part 2/2): Replay coverage and presentation

1. On the coverage threshold (Weakness 2; Question 1)

The reviewer asks why coverage uses K−1K-1 and requests a sensitivity analysis.

K−1K-1 scales support with competing classes and the ETF simplex rank. With ncn_c class-cc examples in the merged dataset, we use \rho_i=\min\left\\{1,\frac{n_{y_i}-1}{K-1}\right\\}. This gives zero coverage for a singleton and full coverage at nc=Kn_c=K. The rank motivates the scale, without guaranteeing statistical sufficiency or feature diversity (Section 4.3).

We varied only the denominator over three paired seeds, at buffer 200 (Table 7). Class-IL mean ± sample SD (%):

ThresholdC100 FAA ↑C100 FF ↓Tiny FAA ↑Tiny FF ↓
0.5(K−1)0.5(K-1)47.93 ± 0.6418.67 ± 0.3633.56 ± 0.627.94 ± 0.27
K−1K-148.69 ± 0.9319.58 ± 0.5034.09 ± 0.627.75 ± 0.36
2(K−1)2(K-1)49.08 ± 0.9920.44 ± 0.6233.56 ± 0.397.93 ± 0.39

All mean FAA changes are below one point. Doubling the threshold raises C100 FAA and FF; the default has the best Tiny FAA and FF. By comparison, disabling transfer costs 5.14 ± 0.53 / 1.80 ± 0.22 FAA points on C100/Tiny. This supports a shared default with a modest sensitivity trade-off.

2. On singleton classes (Weakness 2)

The reviewer notes that sparse replay does not guarantee teacher reliability.

We agree: coverage measures available support and leaves teacher reliability unmodeled. At zero coverage, prototype negatives still provide label-conditioned competition, although teacher errors can be inherited.

Retaining full prototype attraction and transferring only peer weight lowers Tiny/200 FAA by 1.08 ± 0.16 points and raises FF by 1.22 ± 0.95 over three paired seeds (Table 13). Each old class has one exemplar at the last transition. This supports the allocation in that setting; teacher reliability remains a limitation, now discussed in Appendix C.1.2.

3. On motivation (Weakness 3)

The reviewer asks why progressive ETF classifiers are chosen and how transport connects to allocation.

The revised introduction explains that progressive ETFs preserve equal-norm, equiangular targets as classes arrive. Expansion moves these targets while replay support shrinks: transport corrects the reference; allocation adjusts its influence.

4. On related work (Weakness 4)

The reviewer asks for a synthesis of research gaps in the related work.

Section 2 now synthesizes the gap: replay and distillation preserve past information, while ETF expansion moves its targets. CoMove connects these changes through analytic alignment and support-dependent allocation. We distinguish this from learned drift compensation and replay selection.

5. On method clarity (Weakness 5)

The reviewer finds the mechanisms in Sections 4.2–4.3 too densely presented.

We now explain the mechanisms in order: relations, shared negatives, pooling, source responses, then allocation. Each subsection opens with its purpose; Figure 1 follows the same transport–objective–allocation structure. The ablations separate their effects.

6. On typesetting (Weakness 6)

The reviewer identifies typesetting and spacing issues.

Corrected: Figure 2's caption spacing, the run-in heading around submitted line 290, and excess manual vertical spacing. We also adjusted and visually checked the wrapped tables and paragraph endings.

Reviewer_k2S6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission focus on the class-incremental learning with limited old training data. In brief, the proposed CoMove derives how old class prototypes move to new one based on the ProNC’s ETF framework. Based on the theoretical derivation, CoMove builds a training objective which integrates information from old samples, class prototypes, and the teacher model. As a results, CoMove achieves superior average accuracy in class-incremental learning across all six benchmarks, outperforming the current state-of-the-art baselines by an average of 4.69 percentage points.

Critical strengths
  • Clear Motivation: Based on the theory of ETFs for ProNC, this submission demonstrates that even when using idealized samples from old classes, there may be a discrepancy between the objectives pursued by existing "prototype" methods and those implied by a "frozen teacher model."
  • Rigorous Geometric Derivation: CoMove introduces mechanisms such as the inherited-vertex formula, spherical well-posedness, prototype alignment, and the preservation of rankings for old classes. These mechanisms explain how old prototypes evolve, how features from a teacher model are transferred to new prototypes, and how the ranking relationships among old classes are maintained.
  • Significant Performance Gains: Across all six benchmarks and various buffer constraint settings, CoMove achieves higher average accuracy in Class-IL tasks, outperforming the best-performing baseline model by an average of 4.69 percentage points.
  • Comprehensive Ablation Studies: The research reveals that while some components improve performance on old classes, they do so at the expense of learning new classes; other components, however, benefit both. Furthermore, the study indicates that although a "replay" mechanism helps mitigate forgetting, it can also lead to a decline in final accuracy.
Critical weaknesses
  • There is a gap between the conceptual theory and the empirical evidence: 1) The experiments do not prove that Möbius conformality clearly attributes to performance improvement; 2) on the CIFAR-100 dataset, replacing the transport mechanism with a teacher model results in a change in accuracy of only 0.04 to 0.12 percentage points; 3) on the TinyImageNet dataset (buffer 500), transport mechanism has a negative impact; 4) it looks like normalized affine map performs better across diverse settings, and 5) removing pooling operations may lead to improved results in several experiments.
  • No detailed comparison between the proposed CoMove and ProNC: the method description does not demonstrate which advances over PorNC.
Questions for authors

A relavant work[1] can also be taken into account for literature review. Other questions please refer to the weaknesses.

[1] Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraints, CVPR 2025

2 comments
AuthorsOpenReview ↗

Response to k2S6 (Part 1/2): Transport and pooling

We thank Reviewer k2S6 for recognizing the geometric analysis and empirical gains. We have repeated the component comparisons over three paired seeds in all six settings.

1. On Möbius versus normalized affine

The reviewer questions the benefit of conformality over normalized affine.

Either continuation is effective. Both extend F(vc)=αvc+b=ucF(\mathbf v_c)=\alpha\mathbf v_c+\mathbf b=\mathbf u_c. Their mean FAA differs by -0.22 to +0.25 points, with no consistent winner (Table 5). Möbius preserves tangent angles, motivating its use with similarity-based supervision; we agree that the experiments do not establish an accuracy benefit from conformality. General response 3/4, point 3 gives the full results.

2. On dataset-dependent transport benefits

The reviewer also notes the small transport gains on CIFAR-100 and the lower FAA on TinyImageNet/500.

The clearer benefit is lower forgetting. Below, Δ=Mo¨bius−raw\Delta=\text{Möbius}-\text{raw}: three-seed paired mean ± sample SD, Class-IL points. Positive ΔFAA and negative ΔFF favor transport.

Dataset / bufferΔ\Delta FAA ↑\uparrowΔ\Delta FF ↓\downarrow
C100 / 200+0.13 ± 0.09-1.44 ± 0.21
C100 / 500-0.08 ± 0.18-0.94 ± 0.58
Tiny / 500-0.31 ± 0.40-0.68 ± 0.04

Raw references retain small FAA advantages in two settings; Möbius lowers FF in all 18 pairs. FF measures decline from earlier peaks, distinct from final old-class accuracy. Prototype displacement is much larger on C10 (35.26° versus 13.26°/9.21°), consistent with its larger FAA gains; General response 3/4, point 4 explains this geometric context.

The C10/200 interaction test also supports combining transport with CoMove's responses: its FAA benefit is 1.83 ± 0.39 points larger than with logistic responses, positive on every seed (Table 16).

3. On pooling

The reviewer notes that removing pooling can improve accuracy.

The repeated results show a modest C10 benefit and mixed effects elsewhere. Pooling adds 0.55 ± 0.36 / 0.28 ± 0.31 FAA points at C10 buffers 200/500. C100/Tiny changes range from -0.21 to +0.05; removing pooling also lowers Tiny/200 FF by 0.20. The unpooled objective remains competitive. Table 20 and General response 4/4, point 2 give all FAA/FF results.

AuthorsOpenReview ↗

Response to k2S6 (Part 2/2): Comparison with ProNC and related work

1. On the advances beyond ProNC

The reviewer asks for a detailed comparison clarifying CoMove's methodological advances over ProNC.

CoMove adds teacher-reference alignment and an evolving learning objective. It retains ProNC's ETF construction, derives how inherited prototypes move, and extends that map to teacher features with exact prototype alignment and old-label ranking preservation (Section 4.1).

ProNC combines cross-entropy with separate alignment and distillation regressions. CoMove places peers, prototypes and teacher references against shared negative competition, with feature-derived responses and support-dependent weights. General response 1/4, point 2 compares the formulas and coefficients. Removing coverage transfer costs 5.14 ± 0.53 FAA points on C100/200 and 1.80 ± 0.22 on Tiny/200 over three paired seeds (Table 4).

The gains also survive replacing our initial-task procedure: from ProNC births, CoMove with raw references exceeds published ProNC by 3.61–8.61 FAA points in all six settings (Table 6). Full CoMove remains comparable to or better than with our own birth. General response 2/4, point 2 reports this control.

2. On related work

The reviewer suggests including Ferret in the related work.

Added to Section 2. Ferret (CVPR 2025) addresses runtime memory and latency through pipeline parallelism and gradient compensation. We distinguish these constraints from CoMove's limited exemplar storage and supervision design.

Reviewer_YYPMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoMove, a class-incremental learning (CIL) framework that adapts both the geometry and allocation of supervision as new classes are introduced. The authors characterize the displacement of existing class prototypes during classifier expansion and introduce a Möbius-based spherical transport to align frozen teacher representations with the updated classifier geometry. The proposed transport preserves the relative ranking of old classes. In addition, CoMove combines three supervision sources—same-class peers, ETF prototypes, and transported teacher representations—through a unified learning objective with shared negative competition and source-specific loss responses. A support-dependent allocation mechanism dynamically transfers supervision weight from peers and prototypes to the teacher when replay support becomes insufficient. Experiments on CIFAR-10, CIFAR-100, and TinyImageNet, under different replay budgets and an exemplar-free setting, demonstrate improvements in final Class-IL accuracy and forgetting compared with several existing methods.

Critical strengths
  1. The paper reports strong Class-IL performance on CIFAR-10, CIFAR-100, and TinyImageNet under replay budgets of 200 and 500 examples. CoMove achieves the highest mean final Class-IL accuracy among the compared methods in all six main settings. The improvements are particularly notable on TinyImageNet, where the method also substantially reduces forgetting. These results demonstrate the practical effectiveness of the complete framework under limited-memory conditions.

  2. The paper provides a theoretical characterization of inherited ETF prototype displacement and establishes prototype alignment and old-class ranking preservation for the proposed transport. The attempt to explicitly account for changing replay support when combining peer, prototype, and teacher supervision is also meaningful. In particular, the coverage-transfer ablations suggest that reallocating supervision toward the teacher can improve performance when old-class examples become scarce.

Critical weaknesses
  1. The proposed framework builds substantially upon the progressive ETF construction introduced by ProNC. While the characterization of prototype displacement and the associated spherical transport are mathematically interesting, the Möbius transformation itself is an established mathematical construction. The paper's methodological novelty therefore rests primarily on its application to teacher representations and its integration with the proposed supervision objective. The support-dependent supervision allocation is intuitively reasonable, but the specific class-count-based weighting and coverage rules appear largely heuristic. Similarly, the unified objective combines established supervision sources, although its particular formulation is new. The paper would benefit from a clearer identification of which elements constitute fundamental methodological advances beyond ProNC and existing contrastive or distillation-based approaches, rather than refinements or combinations of known techniques. This concern is particularly consequential because the empirical results do not consistently establish the necessity of the most distinctive geometric component.

  2. The component ablations raise questions about whether the additional complexity of the proposed objective and geometric transport is justified. a) Appendix C.1 (Tables 7 and 8) shows that normalized affine transport achieves higher final Class-IL accuracy than the proposed Möbius transport in four of the six evaluated dataset–buffer settings. Although some differences are small, these results do not establish a consistent practical advantage for the more sophisticated geometric construction. b) Removing the proposed pooling mechanism improves Class-IL accuracy in three of six settings, suggesting that its contribution is not consistently positive. The benefits of the source-dependent responses also vary across different benchmarks. c) The component ablations are based on paired single runs. While pairing improves comparability, repeated experiments are needed to determine whether the relatively small differences between several variants are statistically meaningful. Taken together, these observations make it difficult to determine which components are essential to the reported improvements and whether the additional methodological complexity is sufficiently justified. Stronger component-level evidence would be important for establishing the significance of the proposed contributions.

Questions for authors
  1. Could the authors clarify the primary methodological novelty of CoMove beyond the progressive ETF framework of ProNC and existing contrastive or knowledge-distillation approaches?

  2. The ablations in Tables 7 and 8 show that normalized affine transport outperforms Möbius transport in four of six Class-IL settings. Given that both can align inherited prototypes, what practical advantage justifies the additional complexity of the Möbius formulation?

  3. The coverage factor is defined using dataset-level class counts, rather than the actual availability of positive peers in each mini-batch. Moreover, the same coverage factor is applied to both peer and prototype supervision, although these sources depend on data availability in different ways. Could the authors explain the rationale for this choice and provide evidence that it is preferable to simpler alternatives, such as fixed weighting, batch-level positive-pair counts, or separate coverage factors for different supervision sources?

3 comments
AuthorsOpenReview ↗

Response to YYPM (Part 1/3): Methodological contribution

We thank Reviewer YYPM for recognizing the limited-memory results and geometric analysis. We clarify the two contributions here, then address repeated ablations and allocation controls in Parts 2 and 3.

1. Beyond progressive ETF construction (Weakness 1; Question 1)

The reviewer asks what CoMove adds beyond ProNC and existing objectives.

We add analytic teacher alignment and an evolving supervision objective, retaining ProNC's nested ETF construction.

ProNC expands its classifier while distilling toward previous features. Even at neural collapse, alignment targets uyi\mathbf u_{y_i} while distillation targets vyi\mathbf v_{y_i}: the minimizers differ (Proposition 3). We derive the affine map F(vc)=αvc+b=ucF(\mathbf v_c)=\alpha\mathbf v_c+\mathbf b=\mathbf u_c and extend it to unit teacher features, with exact prototype alignment and old-label ranking preservation (Theorems 1–2).

The classifier states determine the map, without feature-pair fitting. Revised Section 4.1 and General response 1/4, point 1 distinguish this derivation and use of transport from the inherited ETF construction.

2. How the objective changes (Weakness 1; Question 1)

The reviewer asks what the integration of established supervision sources adds.

Their competition, responses and allocation evolve together. The three positive relations face a common negative partition; feature-derived responses control prototype and teacher attraction. Class counts set base weights, and sparse replay transfers peer/prototype weight to the teacher at constant total query weight. The same rules apply across datasets and buffers.

This replaces ProNC's separate regressions and setting-specific coefficients with a shared objective. Sections 4.2–4.3 now explain each mechanism; General response 1/4, point 2 compares the loss formulas and coefficients.

3. Evidence beyond the transport choice (Weakness 1)

The reviewer questions whether the gains require the geometric component.

Substantial gains remain without our birth or transport. From ProNC births, retaining its learned encoder, ETF and replay buffer, CoMove with raw references exceeds published ProNC by 3.61–8.61 FAA points in all six settings over three seeds. Full CoMove gains 3.51–8.34 points (Tables 6 and 25; General response 2/4, point 2). This establishes the incremental objective's contribution independently of our initial-task procedure; Part 2 examines what transport adds.

AuthorsOpenReview ↗

Response to YYPM (Part 2/3): Component evidence and complexity

1. On Möbius versus normalized affine (Weakness 2; Question 2)

The reviewer asks what justifies Möbius when normalized affine has higher FAA in four settings.

Both continuations work, at comparable cost. In the new three-seed comparison, normalized affine remains ahead in four settings, but mean FAA differences are only -0.22 to +0.25 points. Both lower FF in all 18 pairs with raw teacher; raw retains small FAA advantages on C100/500 and Tiny/500 (Table 5).

They extend the same affine map and coincide at inherited prototypes. Möbius adds tangent-angle preservation, a geometric motivation for similarity-based supervision; these results do not establish an accuracy benefit from conformality.

Both maps require O(d)O(d) work and no learned parameters. Their isolated forward-time difference is 0.016–0.030% of a full training step on a GTX 1080 Ti, FP32, batch 32, d=512d=512 (Table 22). General response 3/4, point 2 provides formulas and timing details.

2. On pooling (Weakness 2)

The reviewer observes mixed pooling gains and requests repeated ablations.

The repeated results confirm a modest C10 benefit and small, mixed effects elsewhere. Pooling adds 0.55 ± 0.36 / 0.28 ± 0.31 FAA points on C10 at buffers 200/500 (paired mean ± sample SD). C100/Tiny changes range from -0.21 to +0.05; removing pooling lowers Tiny/200 FF by 0.20 (Table 20).

The unpooled objective remains competitive. Stopping cross-query derivatives within the pooled objective instead lowers CIFAR FAA in all 12 pairs: mean losses are 5.49/3.98 points on C10 and 0.89/1.27 on C100 at buffers 200/500. This supports full differentiation when pooling is used. General response 4/4, point 2 distinguishes the controls.

3. On component contributions and interactions (Weakness 2)

The reviewer asks which components account for the gains and how they interact.

The repeated ablations separate the effects of coverage transfer, source responses, pooling and its cross-query gradients, with three paired seeds in all six settings (Tables 4 and 10).

Coverage transfer contributes substantially on C100 and Tiny. Disabling it, while retaining total query weight, reduces mean FAA by 5.14/3.20 points on C100 and 1.80/1.10 on Tiny at buffers 200/500. The loss occurs in every paired run. C10 already has full coverage, so this intervention leaves its objective unchanged.

Source responses have their clearest benefit on C10. Replacing them by logistic responses, with transport and allocation unchanged, gives Δ=logistic−CoMove\Delta=\text{logistic}-\text{CoMove} (three-seed paired mean ± sample SD, Class-IL points):

Dataset / bufferΔ\Delta FAA ↑Δ\Delta FF ↓
C10 / 200-5.02 ± 0.22+8.84 ± 0.14
C10 / 500-3.44 ± 0.94+6.20 ± 1.87

Both metrics worsen on every seed at both budgets. C100/Tiny FAA changes stay within 0.28 points, with mixed signs. Binary tasks give δi=1\delta_i=1; larger task blocks allow entropy to attenuate the adjustment.

Transport and source responses reinforce each other on C10. On C10/200, transport improves FAA by 2.12 ± 0.43 points with CoMove responses versus 0.29 ± 0.44 with logistic responses: an interaction of +1.83 ± 0.39, positive on every seed. FF reductions are 4.61 ± 0.84 versus 0.94 ± 0.52, an interaction of -3.68 ± 0.69 (Table 16).

These results identify complementary contributions: coverage transfer and source responses account for substantial gains on different datasets; transport consistently reduces FF; pooling has a smaller effect, while its cross-query gradients matter on CIFAR. The evolving class-count budget and simpler allocation rules are evaluated in Part 3, point 4.

AuthorsOpenReview ↗

Response to YYPM (Part 3/3): Support-dependent allocation

1. On the weighting assumptions (Weakness 1)

The reviewer asks how the class-count budget and coverage rules are motivated.

The rules track class composition and replay support. With OO old and qq new classes, K=O+qK=O+q, peers receive q/Kq/K and prototype/teacher supervision split O/KO/K equally under exchangeability (Appendix A.3).

For old classes, ρi=min⁡1,(nyi−1)/(K−1)\rho_i=\min\\{1,(n_{y_i}-1)/(K-1)\\} scales additional class examples by competing classes, also the ETF simplex rank. Coverage decreases as classes accumulate at fixed support. These assumptions motivate a shared rule; the controls below test it.

2. On global counts (Question 3)

The reviewer asks why coverage uses dataset counts rather than the positive peers available in each minibatch.

Dataset counts measure available class support. The peer term separately handles batch availability, averaging actual positives and falling back to the prototype when none occur (Appendix A.2). This separates underlying support from sampling variation.

3. On shared coverage for peers and prototypes (Question 3)

The reviewer points out that prototypes exist even when positive peers are absent.

We agree. Learning alignment to a fixed prototype still relies on class examples. Class balancing can assign large weights to a few replay examples; coverage limits their peer/prototype attraction and transfers the remaining mass to the teacher. Prototype negatives remain active. We test retaining full prototype attraction below.

4. On simpler alternatives (Question 3)

The reviewer requests fixed weights, minibatch counts and separate coverage factors.

Three paired seeds, buffer 200, with the same birth, transport, responses, competition and total query weight (Table 13). Δ=alternative−CoMove\Delta=\text{alternative}-\text{CoMove}, Class-IL points, paired mean ± sample SD. Positive ΔFAA and negative ΔFF favor the alternative.

AlternativeC100 ΔFAA ↑C100 ΔFF ↓Tiny ΔFAA ↑Tiny ΔFF ↓
No transfer, ρ=1\rho=1-5.14 ± 0.53-0.16 ± 0.61-1.80 ± 0.22+1.27 ± 0.30
Keep prototype weight-4.05 ± 0.50-0.06 ± 0.77-1.08 ± 0.16+1.22 ± 0.95
Minibatch coverage+0.55 ± 0.70+2.47 ± 0.21-0.13 ± 0.36+0.17 ± 0.59
Fixed first-transition budget-2.35 ± 0.81+13.37 ± 0.63+2.61 ± 0.72+14.94 ± 0.49
Equal source fractions-20.15 ± 1.49+44.69 ± 1.21-15.64 ± 0.27+51.21 ± 0.55

Separate factors. Keeping prototype coverage at one transfers only unsupported peer weight. Like disabling transfer, this lowers FAA in every pair. On C100, no transfer reduces acquisition by 9.64 points and final old-class accuracy by 3.31, despite similar FF (Table 17).

Minibatch counts. Replacing nyi−1n_{y_i}-1 with batch peer counts improves C100 FAA with higher FF; Tiny stays close. The denominator and clipping are unchanged.

Fixed budgets. Freezing first-transition fractions at (mp,ma,mt)=(1/2,1/4,1/4)(m_{\mathrm p},m_{\mathrm a},m_{\mathrm t})=(1/2,1/4,1/4) retains coverage and total query weight. Tiny gains 2.61 FAA points but incurs 14.94 more FF points. Final old-class accuracy stays close; better acquisition creates higher earlier peaks (Table 18). Equal thirds substantially worsen both metrics on both datasets.

These controls support coverage transfer and expose the acquisition–forgetting trade-off of fixed budgets. The shared allocation rule does not maximize FAA against every alternative; teacher reliability and feature diversity also remain unmodeled.

Comments on the paper

6 comments
AuthorsOpenReview ↗

Overview of our responses

We thank the reviewers for their careful reading and constructive suggestions. We have conducted additional experiments and revised the manuscript to clarify CoMove's contributions, isolate the sources of its gains, and improve the presentation. The four replies to this comment address shared concerns; individual replies state the conclusions relevant to each reviewer and point to the supporting evidence.

Responses to shared concerns

  1. General response 1/4: What does CoMove add beyond ProNC? CoMove characterizes how inherited prototypes move and extends this map to teacher features. Its evolving objective places peer, prototype and teacher relations against shared negative competition, with weights that adapt to class composition and replay support. We clarify both contributions and their relation to ProNC's ETF construction.

  2. General response 2/4: Do the gains depend on initial-task training? In new experiments over three seeds in all six settings, we retain ProNC's initial-task training, learned encoder, ETF and replay buffer, changing only the incremental objective. Even with raw teacher references, CoMove gains 3.61–8.61 FAA points over published ProNC. These gains persist without our initial-task procedure or teacher transport.

  3. General response 3/4: Teacher alignment. New comparisons show that both normalized affine and Möbius lower forgetting in all 18 paired comparisons with raw teacher. Their mean FAA differs by at most 0.25 points, with no consistent winner and comparable measured cost. We explain their close geometric relationship. New Figure 11 illustrates how initial class count affects decision margins and temporal-target mismatch at the prototypes.

  4. General response 4/4: Repeated component ablations. New three-seed ablations in all six settings isolate coverage transfer, pooling, source responses and cross-query gradients. Coverage transfer benefits CIFAR-100 and TinyImageNet; source responses and cross-query gradients have their clearest effects on CIFAR-10. Pooling has a smaller, dataset-dependent effect.

A separate comment, "Summary of manuscript revisions", lists the changes and their locations in the revised manuscript.

AuthorsOpenReview ↗

Summary of manuscript revisions

Update (17 November 2026). We have uploaded an updated manuscript with a clearer overview in Figure 1 and further improvements to the presentation of the method and results. Appendix C.9 (Table 26) now includes complete Class-IL and Task-IL accuracy and forgetting results for the exemplar-free comparison, using the same runs and published sources. Changes relative to the original submission remain highlighted in red. The revisions summarized below are included in this version.

Changes are highlighted in red. The following revisions address the reviewers' concerns:

  1. Contributions and method clarity (YYPM, k2S6, N6xX). We explain how moving ETF targets and shrinking replay support motivate teacher alignment and evolving allocation. Sections 4.1, 4.2 and 4.3 now introduce the mechanisms step by step; Figure 1 follows the same three stages.

  2. Component-level evidence (YYPM, N6xX, k2S6). Table 4 now reports means and standard deviations over three paired runs, with full metrics in Appendix C.1.1. Figure 3 adds across-run variability to acquisition and retention trajectories. Appendix C.4 reports paired pooling effects and gradient diagnostics.

  3. Transport accuracy, forgetting and cost (all three reviewers). We bring the repeated comparison of raw teacher, normalized affine and Möbius references into the main text (Table 5). Appendix C.5 relates the two formulas and adds geometric diagnostics, GPU timings and Task-IL results. New Figure 11 plots analytic decision margins and temporal-logit deficits at the prototypes as a function of initial class count.

  4. Initial-task training control (N6xX). Table 6 adds the comparison from ProNC initial-task checkpoints, isolating the change in incremental objective. Appendix C.8 gives all Class-IL and Task-IL metrics.

  5. Coverage and simpler alternatives (N6xX, YYPM). We add threshold sensitivity (Table 7), alternative coverage rules, fixed budgets and equal source weights (Appendix C.1.2), and transport-response interactions (Appendix C.1.3). Appendix C.2 separates acquisition gains from final old-class accuracy.

  6. Related work and layout (k2S6, N6xX). Section 2 now synthesizes the research gaps and includes Ferret. We corrected Figure 2's caption spacing, broken paragraph headings and excessive vertical gaps.

We believe these revisions clarify CoMove's contributions and strengthen the experimental evidence. We welcome further questions and would be glad to discuss any remaining concerns.

AuthorsOpenReview ↗

General response 1/4: What does CoMove add beyond ProNC?

1. From classifier expansion to teacher transport

What we borrow. ProNC's nested ETF construction appends orthonormal columns for OO old and KK total classes (Appendix D.1.2, Equations (35)-(36)):

\qquad \mathbf U_K=\sqrt{\frac K{K-1}}\mathbf Q_K\mathbf H_K,$$ Here $\mathbf H_K=\mathbf I_K-K^{-1}\mathbf 1_K\mathbf 1_K^\top$. Recentring and rescaling move the inherited vertices. *What we identify.* ProNC aligns student feature $\mathbf z_i$ with the expanded prototype $\mathbf u_{y_i}$ and frozen teacher $\mathbf a_i=\operatorname{sg}[\nu(\mathbf W_{t-1}f_{t-1}(x_i))]$ through its Equations (3)-(4): $$\ell_i^{\mathrm{align}}=\frac12(\mathbf z_i^\top\mathbf u_{y_i}-1)^2, \qquad \ell_i^{\mathrm{distill}}=\frac12(\mathbf z_i^\top\mathbf a_i-1)^2.$$ At neural collapse, $\mathbf a_i=\mathbf v_{y_i}$: the two terms have different minimizers and cannot vanish simultaneously (Proposition 3). *What we add.* Chaining symmetric lifts (Definition 1) yields the exact affine map (Theorem 1): $$F:\\{\mathbf v_c\\}_{c\le O}\longrightarrow\\{\mathbf u_c\\}_{c\le O}, \qquad F(\mathbf v_c)=\alpha\mathbf v_c+\mathbf b=\mathbf u_c,$$ Here $\alpha=\sqrt{\frac{K(O-1)}{O(K-1)}}$, $\mathbf b=O^{-1}\sum_{c\le O}\mathbf u_c$, $\mathbf b\perp\operatorname{span}\\{\mathbf v_c\\}_{c\le O}$ and $\\\|\mathbf b\\\|^2=1-\alpha^2$. For an arbitrary unit feature, the affine map has norm defect (Appendix D.3) $$\\\|\alpha\mathbf x+\mathbf b\\\|^2-1=2\alpha\mathbf b^\top\mathbf x.$$ We therefore extend $F$ to teacher features on the sphere: $$T:\mathbb S^{d-1}\longrightarrow\mathbb S^{d-1}, \qquad T(\mathbf v_c)=F(\mathbf v_c)=\mathbf u_c\quad(c\le O).$$ ProNC uses $\mathbf a_i$ directly; CoMove uses $\mathbf r_i=T_{\mathbf b}(\mathbf a_i)$. The Möbius continuation aligns prototypes exactly and preserves old-class rankings, including ties (Theorem 2). Revised Section 4.1 explains this construction; General response 3/4 compares its continuations. ## 2. How the incremental objective differs *From regression targets to shared competition.* For a minibatch of $B$ queries, the objectives are $$\begin{aligned} \mathcal L_{\mathrm{ProNC}} &=\frac1B\sum_i\left( \ell_i^{\mathrm{ce}}+\eta_{\mathrm a}\ell_i^{\mathrm{align}} +\eta_{\mathrm t}\ell_i^{\mathrm{distill}}\right),\\\\ \mathcal L_{\mathrm{CoMove}} &=\frac1B\sum_i\left( \omega_i^{\mathrm p}\ell_i^{\mathrm p} +\omega_i^{\mathrm a}\ell_i^{\mathrm a} +\omega_i^{\mathrm t}\ell_i^{\mathrm t}\right). \end{aligned}$$ ProNC combines cross-entropy with weighted regressions (its Equation (5)). CoMove's three terms share negative competition (Equation (11)); feature-derived responses control prototype and teacher attraction (Equation (12)). *From tuned coefficients to support-dependent allocation.* ProNC grid-searches coefficients by dataset and buffer (its Appendix A.2): | Dataset | Buffer | Cross-entropy | ETF alignment $\eta_{\mathrm a}$ | Distillation $\eta_{\mathrm t}$ | |---|---:|---:|---:|---:| | CIFAR-10 | 200 | 1 | 13 | 90 | | CIFAR-10 | 500 | 1 | 12 | 80 | | CIFAR-100 | 200 / 500 | 1 | 18 | 170 | | TinyImageNet | 200 | 1 | 20 | 165 | | TinyImageNet | 500 | 1 | 20 | 200 | CoMove uses a fixed temperature and the *same rules* for $\omega_i^{\mathrm p},\omega_i^{\mathrm a},\omega_i^{\mathrm t}$ in all six settings. Base masses follow class counts (Equation (25)); sparse replay transfers peer and prototype weight to the teacher at constant total query weight (Equation (13)). The objective also works with raw references, $\mathbf r_i=\mathbf a_i$: General response 2/4 tests it from ProNC births; General response 4/4 isolates its components.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.