acceptodds
Under review as a conference paper at ICLR 2027

Angular LDA for Learning After Deployment

Abstract

Traditional machine learning models are static and once deployed, they no longer can acquire new knowledge. However, real-world environments are constantly evolving, and deployed AI systems will inevitably encounter classes and situations that were not present during training. To remain effective, these systems must be capable of post-deployment learning: identifying novel classes, acquiring ground-truth supervision, and learning them incrementally on the fly from the new data without relying on retraining with both past and newly collected data. To address this challenge, we propose LAFA, a novel post-deployment learning method that leverages pre-trained models and angular linear discriminant analysis with feature and test-time enhancements. Extensive experiments on eight datasets using three backbones demonstrate that LAFA consistently outperforms strong baselines without saving any past training data. Its performance can match that of offline retraining (or joint training) that has access to all past and present data. We believe the proposed approach is sufficiently mature for practical deployment.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Post-Deployment Learning, Open-World, Incremental Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TW2YOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents LAFA, a framework for post-deployment learning that integrates out-of-distribution (OOD) detection with online class-incremental learning. The proposed approach employs a frozen pre-trained model for feature extraction, followed by random projection and Angular Linear Discriminant Analysis (aLDA) to update class statistics incrementally without retaining previous training data. It also incorporates test-time magnitude augmentation, using unnormalized features for inference while constructing class statistics from normalized features. Experiments on eight datasets with three pre-trained backbones report consistent improvements over existing continual learning baselines, particularly PLDA.

Critical strengths
  1. The paper considers the integration of OOD detection and incremental classification under a streaming data setting, where new classes may emerge after the initial training stage.

  2. The experimental evaluation covers multiple datasets and pre-trained backbones. The paper also provides ablation studies examining the effects of normalization, random projection, and test-time magnitude augmentation.

Critical weaknesses
  1. Limited methodological novelty. The proposed method primarily combines existing techniques, including feature normalization, random projection, and incremental LDA. In particular, Angular LDA largely amounts to normalizing class mean directions, while test-time magnitude augmentation modifies the inference rule without introducing a new learning objective or optimization procedure. Angular representations and normalized classifiers have already been explored in continual learning. Overall, the proposed method appears closer to an empirical refinement of existing approaches than a substantial methodological contribution.

  2. Unclear distinction from existing continual learning settings and incomplete framework. The proposed learning-after-deployment setting substantially overlaps with existing open-world and online class-incremental learning frameworks. Although label acquisition is presented as an essential component, the paper does not implement it and instead assumes that ground-truth labels are available for detected OOD samples. Consequently, the proposed system essentially reduces to OOD detection combined with supervised online class-incremental learning. This also weakens the claims regarding practical deployment.

  3. Insufficient baseline comparisons. Several closely related continual learning methods, including SimpleCIL and RanPAC, are missing from the experimental comparisons, despite their relevance to normalized classifiers, random projection, and analytic incremental learning. Furthermore, first-session adaptation is applied selectively across baselines, making it difficult to assess the fairness of the comparisons. The comparison against joint fine-tuning is also insufficient to support the claimed performance advantage, as a joint analytic classifier using matched feature representations would provide a more appropriate reference.

  4. Insufficient analysis of the proposed components and their mathematical justification. Although ablation studies are provided, they do not isolate the contribution of angular class mean normalization from conventional feature normalization. Similarly, the effectiveness of test-time magnitude augmentation is demonstrated empirically without sufficient justification for using normalized features during training and unnormalized features during inference. The incremental covariance update in Equation (6) does not appear to be mathematically equivalent to the covariance definition in Equation (5). In particular, the update does not fully account for changes in class mean directions and uses the total sample count rather than the class-specific count in its correction factor. Although Appendix A discusses the angular covariance approximation, it does not adequately justify this incremental update or analyze its approximation error.

  5. Insufficient evaluation of OOD detection and practical deployment. The paper mainly reports aggregate F-scores for OOD detection without sufficiently analyzing how detection performance and annotation requirements evolve during deployment. The claim that AUROC and AUPRC cannot be applied to dynamic OOD detection is also insufficiently justified, as these metrics could be evaluated at individual deployment stages. Furthermore, the experiments do not consider annotation errors or realistic labeling costs, despite emphasizing practical deployment.

Questions for authors
  1. What is the key methodological novelty beyond combining existing normalization, random projection, and LDA-based incremental learning techniques?

  2. How does the proposed learning-after-deployment framework differ from existing open-world continual learning settings, particularly without implementing label acquisition?

  3. Can the authors provide comparisons with closely related methods such as SimpleCIL and RanPAC under consistent adaptation and evaluation settings?

  4. Can the authors isolate the contribution of Angular LDA from standard feature normalization and clarify the mathematical correctness of Equation (6)?

  5. Can the authors provide a more detailed evaluation of dynamic OOD detection, annotation costs, and robustness to imperfect labels?

7 comments

Thank you for the thorough review. We address each weakness and question below, and a revised PDF is uploaded.

Q1:

What is the key methodological novelty beyond combining existing normalization, random projection, and LDA-based incremental learning techniques?

Response:

First, one key difference is the mean direction and angular statistics used in our system rather than the arithmetic mean as used in LDA, which results in major improvements. "Random projection + incremental LDA + the post-deployment pipeline" without the angular statistics is exactly the LAFA−N variant in Table 1. Its average accuracy is 80.15%, lower than PLDA (82.90%) and 5.25 points below LAFA (85.40%), and its F-score falls from 92.51% to 65.07% on ImageNet-R and from 88.35% to 66.69% on CUB. With few samples per novel class, the mean direction is what makes the same LDA score usable for both classification and OOD detection.

Second, at test time, test-time magnitude augmentation (TMA) uses the raw, unnormalized feature, which rebalances the class-dependent bias of the LDA score in favor of the few-shot novel classes (see our response to Q4). For example, on CIFAR100, novel-class accuracy rises from 68.7% to 88.2%.

Finally, a single score is used for both classification and OOD detection, so detection improves as new classes are learned.

Also, importantly, the proposed LAFA achieves accuracy comparable to joint training (i.e., retraining from scratch using all available data). We believe this level of performance demonstrates its readiness for practical deployment.

To the best of our knowledge, no existing method combines angular LDA and TMA, both of which are instrumental in enabling effective learning from limited data—a defining characteristic of post-deployment learning. In our search, we found that SimpleCIL (Zhou et al., 2023) normalize its features, but does so during both training and at test time and it also does not detect OOD samples and does not decide which samples to learn from a stream. LAFA normalizes only when building the class statistics, uses the raw feature at test time (i.e., TMA), and uses one score for both classification and OOD detection. SimpleCIL is included as a baseline in our response to Q3, where we will see that LAFA is significantly more accurate on every dataset.

Q2:

How does the proposed learning-after-deployment framework differ from existing open-world continual learning settings, particularly without implementing label acquisition?

Response:

Our setting is described in the Introduction (Section 1), and its differences from related settings are discussed in Related Work (Section 2). In online class-incremental learning (online CIL), every incoming sample arrives with its label, and the learner never decides whether to learn it. In our setting, a label exists only if the system decides to ask for it, so the system must first detect that a sample is out-of-distribution (OOD). If it asks for labels it does not need, the labels are wasted (low precision); if it misses novel samples, it never learns them (low recall). Its detection errors therefore decide what it learns. The set of known classes also grows sample by sample, as a novel class goes from unseen, to emerging (1 to 29 labels received), to well-learned (30 labels or more).

Our setting is similar to that of open-world continual learning (OWCL)(e.g., Bendale & Boult, 2015). However, in OWCL, novel samples are usually detected against a fixed set of known classes, collected, and learned later in a batch. In our setting, detection and learning are interleaved for every single sample, with no buffer of stored samples, and the detector must also recognize emerging classes that it is currently learning.

As stated in Introduction section, the post-deployment system performs three tasks: (1) classify incoming samples and detect OOD samples, (2) obtain the label of a detected sample from a human or an AI agent, and (3) learn it incrementally. This paper focuses on tasks (1) and (3); task (2) usually involves a natural-language dialogue, and we leave it to future work. In the experiments, the human is simulated by giving the true label of each queried sample, as in active learning and in PLDA. The system still does not just reduce to OOD detection plus supervised online CIL, because it decides which samples receive labels: what it learns and how many labels it asks for both depend on its own detection decisions.

Since a real annotator can make mistakes, we also ran LAFA with a noisy oracle that returns wrong labels (see our response to Q5): with 20% wrong labels, its accuracy drops by only about one point, and it still stays more accurate than PLDA.

Q3:

Can the authors provide comparisons with closely related methods such as SimpleCIL and RanPAC under consistent adaptation and evaluation settings?

Response:

We ran SimpleCIL, RanPAC and SLDA on exactly the same adapted backbone features and the same streams as LAFA and PLDA (3 runs). Each method is evaluated in two ways. In the first, we assume a perfect OOD detector: every novel-class sample is detected and its label is given. In the second, each method runs with its own OOD detector inside the same post-deployment loop as LAFA. Since SimmpleCIL, RanPAC and SLDA does not have an OOD detector, we use their own highest classification scores (which is also what our LAFA system uses) as the OOD score, the same emerging-class rule, and the same well-learned threshold (30 labels); PLDA uses its own OOD score.

The results are given in the table below. Accuracy (%) is measured on the test set of all classes (base and novel) at the end of the stream, and the four dataset columns are with each method's own OOD detector. "Queries" is the average number of label requests per stream (the average stream has 4521 samples).

MethodCIFAR100TinyImageNetImageNet-RCUBAverage accuracy (%), own OOD detectorAverage accuracy (%), assuming a perfect OOD detectorAverage number of queries
SLDA89.0284.4580.9182.6384.2584.254517
RanPAC81.3778.5979.7982.1780.4880.662308
SimpleCIL87.9784.3181.9180.1183.5884.421809
PLDA87.3384.2681.6682.0583.8384.611931
LAFA (ours)90.4186.0483.8482.8885.7986.411846

LAFA is the most accurate method on every dataset, both with its own OOD detector and assuming a perfect OOD detector. With its own OOD detector (85.79%), it is even more accurate than every other method that is given a perfect OOD detector (at most 84.61%). Note that in our paper, we compared our method with a more recent method, LoRanPac, which improved RanPAC.

First-session adaptation (FSA) fine-tunes the pre-trained model once before deployment, with a light adapter, using only base-class data. Following LoRanPac, the paper applies FSA to the analytic methods that keep the backbone fixed afterwards (LoRanPac, PLDA and LAFA). The replay-based baselines (A-GEM, GDumb, ER, iCaRL, MIR) already adapt the backbone themselves, by training adapters during the stream on all classes, including the novel ones. All methods in the table above share exactly the same FSA backbone, and without FSA, LAFA also outperforms PLDA (Appendix I: 83.01% vs. 80.66% average accuracy with DINOv2).

As a reference with matched features, we also trained joint analytic classifiers offline, on the same projected features and with all labels of the same data given at once. This is the data LAFA would learn assuming a perfect OOD detector: the base-class training data plus the novel-class samples of the stream. Accuracy (%) is on the test set of all classes, averaged over 3 runs.

Method (same features)CIFAR100TinyImageNetImageNet-RCUBAverage accuracy (%)
Joint LDA (ordinary LDA, offline, all labels)88.1184.1376.0681.1682.36
Joint angular LDA (offline, all labels)91.6187.0883.8883.8086.59
LAFA (ours), from the stream, assuming a perfect OOD detector91.2786.4583.9184.0086.41
LAFA (ours), from the stream, own OOD detector90.4186.0483.8482.8885.79

LAFA learns one sample at a time, with no stored data and with its own OOD detector, and still stays within 0.8 points of the same classifier trained offline with all labels (85.79% vs. 86.59%). Even offline, angular LDA is 4.2 points more accurate than ordinary LDA (86.59% vs. 82.36%), so the angular statistics help independently of streaming.

Q4:

Can the authors isolate the contribution of Angular LDA from standard feature normalization and clarify the mathematical correctness of Equation (6)?

Response (part 1 of 3: angular LDA vs. standard feature normalization):

If we understand correctly, standard feature normalization normalize each feature based on the feature of all samples. This is not applicable in our setting: after deployment, samples are encountered one at a time from a stream and are not stored, so such statistics cannot be computed over the data the system learns. LAFA uses L2-normalization, which uses only the sample itself, together with angular class statistics.

The contribution of the angular statistics is isolated by the LAFA−N variant in Table 1, which is the same system with standard (non-angular) LDA on the raw projected features. Removing the angular statistics lowers average accuracy from 85.40% to 80.15%, and the F-score drops from 92.51% to 65.07% on ImageNet-R and from 88.35% to 66.69% on CUB.

Response (part 2 of 3: unnormalized features at inference, TMA):

Let zz be the projected feature of a test sample, r=∥z∥r = \|z\| its length, and z^=z/r\hat{z} = z/r its direction. For class kk, μk\mu_k is the mean direction and Λ\Lambda is the precision matrix (the inverse of the shared covariance). The LDA score is

hk(z)=r z^⊤Λμk−bk,bk=12 μk⊤Λμk,h_k(z) = r\,\hat{z}^\top \Lambda \mu_k - b_k, \qquad b_k = \tfrac{1}{2}\,\mu_k^\top \Lambda \mu_k,

where the first term measures how well the direction of the sample matches class kk, and bkb_k is a class-dependent bias (one number per class). Because the shared covariance is estimated mostly from base-class data, bkb_k is larger for the few-shot novel classes (up to 3.2× on CIFAR100), which pushes predictions away from them. Using the raw feature multiplies the first term by rr, which rebalances it against bkb_k; on CIFAR100, novel-class accuracy rises from 68.7% to 88.2%. Without TMA (normalized test features, the −M variant) average accuracy is 77.39%, and with TMA it is 85.79%.

Response (part 3 of 3: Eq. 6):

Eq. (6) defines a separate online estimator; it is not meant to reproduce Eq. (5) exactly at every step. It is a running average of the shared covariance over all NN samples: each new sample adds its residual (z~−μy)(z~−μy)⊤(\tilde{z} - \mu_y)(\tilde{z} - \mu_y)^\top, and the residuals of past samples are kept as they are. Because Σ\Sigma is shared by all classes, the correction factor uses the total count NN, not the class count nyn_y. The first sample of a new class is measured against μy=0\mu_y = 0, since the class has no mean yet.

We also implemented an update that reproduces Eq. (5) exactly at every step, including the changes of the class mean directions. If sks_k is the sum of the normalized features of class kk, the within-class scatter equals ∑i:yi=kz~iz~i⊤−βksksk⊤\sum_{i: y_i = k} \tilde{z}_i \tilde{z}_i^\top - \beta_k s_k s_k^\top with βk=(2∥sk∥−nk)/∥sk∥2\beta_k = (2\|s_k\| - n_k)/\|s_k\|^2, so it can be updated exactly from the class sums that LAFA already stores. We chose Eq. (6) because it is as good on three datasets and better on CUB.

To measure the approximation error and its effect on decisions, we kept both estimators side by side on the same stream, with only Eq. (6) driving the decisions, and compared the predicted classes and the OOD decisions on the test set at the end of the stream, see the first two rows in the table below. We also ran LAFA end to end with each estimator. Results are averaged over 3 runs, and accuracy (%) is on the test set of all classes.

CIFAR100TinyImageNetImageNet-RCUB
Same predicted class (% of test samples)99.399.498.696.7
Same OOD decision (% of test samples)99.599.699.898.2
Accuracy (%), our covariance update (Eq. 6)90.41 ± 0.5186.04 ± 0.4283.84 ± 0.2082.88 ± 0.40
Accuracy (%), exact update of Eq. (5)90.56 ± 0.2485.98 ± 0.3583.80 ± 0.1582.14 ± 0.51

On CIFAR100, TinyImageNet and ImageNet-R, which have tens of samples per novel class, the two estimators make the same decision on 98.6 to 99.8% of test samples, and their accuracies differ by less than one standard deviation. On CUB, with about 3 samples per novel class, Eq. (6) is 0.74 points more accurate. In Eq. (6), the first sample of a new class is measured against μy=0\mu_y = 0, and the first residuals of every class receive full weight. Both increase the variance along the directions of the new class, which works like a prior variance for a class with very few samples and lowers the class bias bkb_k of new classes. With the exact update, novel-class accuracy on CUB falls from 76.27% to 74.84%, while base-class accuracy is unchanged. The exact update and this comparison are now in Appendix A of the revised paper.

Reviewer_SfC8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies post-deployment learning, where a deployed classifier must detect out-of-distribution samples, obtain their labels, and learn the new classes incrementally without storing past data. The proposed method, LAFA, applies angular linear discriminant analysis to randomly projected features from a frozen pre-trained backbone. It uses an asymmetric feature normalization: features are normalized when building class statistics but left unnormalized at test time. Experiments on eight datasets with three backbones show consistent gains over the prior post-deployment method and continual learning baselines, with accuracy comparable to joint fine-tuning.

Critical strengths
  1. The pipeline is clearly described, and each claimed component is supported by extensive experiments and ablations.
Critical weaknesses
  1. My main concern is limited novelty, as the framework combines well-established techniques. The streaming learner follows Deep SLDA [1]. The random projection with nonlinear activation follows RanPAC [2]. The post-deployment pipeline, including the emerging-class set and the well-learned threshold, is inherited from PLDA [3]. Could the authors clarify what is new compared with Deep SLDA and/or RanPAC combined with PLDA?

[1] Tyler L. Hayes and Christopher Kanan. Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis. In CVPR Workshops, 2020.

[2] Mark D. McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. RanPAC: Random Projections and Pre-trained Models for Continual Learning. In NeurIPS, 2023.

[3] Derda Kaymak, Gyuhak Kim, Tomoya Kaichi, Tatsuya Konishi, and Bing Liu. Learning After Model Deployment. In ECAI, 2025.

Questions for authors
  1. Could the authors describe the training setup of the joint fine-tune baseline, especially the training data for the novel classes?
2 comments

Thank you for the positive assessment and the focused questions. A revised PDF is uploaded, and we address your concerns below.

W1:

My main concern is limited novelty, as the framework combines well-established techniques. The streaming learner follows Deep SLDA [1]. The random projection with nonlinear activation follows RanPAC [2]. The post-deployment pipeline, including the emerging-class set and the well-learned threshold, is inherited from PLDA [3]. Could the authors clarify what is new compared with Deep SLDA and/or RanPAC combined with PLDA?

Response:

First, each class in our system is represented by its mean direction, that is, the average of its L2-normalized features rescaled to unit length, and the covariance matrix is shared by all classes and estimated from normalized features. Deep SLDA [1] uses the ordinary (arithmetic) mean and covariance of raw features. RanPAC [2] uses a ridge-regression classifier on raw projected features and does not detect OOD samples. PLDA [3] uses ordinary LDA together with a separate OOD score (relative Mahalanobis distance). Second, test-time magnitude augmentation (TMA) uses the raw, unnormalized feature at test time. This rebalances the class-dependent bias term of the LDA score, which is larger for few-shot novel classes and otherwise pushes predictions away from them. This is significant. For example, on CIFAR100, novel-class accuracy rises from 68.7% to 88.2%. Third, the same LDA score decides both the predicted class and whether to ask for a label, so there is no separate detector, and detection improves as new classes are learned.

Regarding the combination you described, "RanPAC-style random projection + SLDA-style incremental LDA + the PLDA pipeline", is exactly the LAFA−N variant in Table 1, i.e., LAFA without the angular (normalized) statistics. Its average accuracy is 80.15%, lower than both PLDA (82.90%) and LAFA (85.40%), and its F-score drops to 65.07% on ImageNet-R (LAFA: 92.51%) and to 66.69% on CUB (LAFA: 88.35%).

We also ran Deep SLDA [1], RanPAC [2] and SimpleCIL on exactly the same adapted backbone features and the same streams as LAFA and PLDA [3] (3 runs). Each method is evaluated in two ways. In the first, we assume a perfect OOD detector: every novel-class sample is detected and its label is given. In the second, we use OOD detectors. For Deep SLDA and RanPAC, which don't have an OOD detector originally, we use their highest classification score (which is also used in LAFA) as the OOD score, the same emerging-class rule and the same well-learned threshold (30 labels). PLDA uses its own OOD score.

The table below shows the results. Accuracy (%) is measured on the test set of all classes (base and novel) at the end of the stream, and the four dataset columns are with each method's own OOD detector. "Queries" is the average number of label requests per stream (the average stream has 4521 samples).

MethodCIFAR100TinyImageNetImageNet-RCUBAverage accuracy (%), OOD detectorAverage accuracy (%), assuming a perfect OOD detectorAverage number of queries
Deep SLDA [1]89.0284.4580.9182.6384.2584.254517
RanPAC [2]81.3778.5979.7982.1780.4880.662308
SimpleCIL87.9784.3181.9180.1183.5884.421809
PLDA [3]87.3384.2681.6682.0583.8384.611931
LAFA (ours)90.4186.0483.8482.8885.7986.411846

LAFA is the most accurate method on every dataset, both with its own OOD detector and assuming a perfect OOD detector. With its own OOD detector (85.79%), it is even more accurate than every other method that is given a perfect OOD detector (at most 84.61%).

Deep SLDA flags almost every sample as OOD (4517 of 4521 on average), so it learns from an almost fully labeled stream, which is why its accuracy is the same in both settings. RanPAC suffers from class imbalance (This is why we compared our method in the main paper with a more recent method, LoRanPac).

[1] Hayes & Kanan, 2020. [2] McDonnell et al., 2023. [3] Kaymak et al., 2025.

Q1:

Could the authors describe the training setup of the joint fine-tune baseline, especially the training data for the novel classes?

Response:

Joint Fine-Tune is trained offline on the same data that LAFA would learn assuming a perfect OOD detector: the base-class training data used before deployment, plus the novel-class samples of the post-deployment stream (the part of each novel class's training images that LAFA can observe during deployment), with all their labels given. It is not just the full training set of the novel classes. It is the "retrain from scratch on everything the deployed system has seen" reference. The whole DINOv2 backbone (ViT-B/14) is fine-tuned together with a linear classification head, using AdamW with a cosine learning-rate schedule for 30 epochs, where only the head is trained in the first 5 epochs. Because base classes have many more training images than novel classes, the loss is a class-weighted cross-entropy (rarer classes get larger weights), and mini-batches are drawn with class-balanced sampling. The revised paper describes this setup in Section 4.1 and Appendix M.

To separate "learning from a stream vs. offline" from "analytic vs. gradient-based", we also trained analytic classifiers offline on the same projected features, with all labels of the same data given at once (also an joint or retraining setting). Accuracy (%) is on the test set of all classes, averaged over 3 runs.

Method (same features)CIFAR100TinyImageNetImageNet-RCUBAverage accuracy (%)
Joint LDA (ordinary LDA, offline, all labels)88.1184.1376.0681.1682.36
Joint angular LDA (offline, all labels)91.6187.0883.8883.8086.59
LAFA (ours), from the stream, assuming a perfect OOD detector91.2786.4583.9184.0086.41
LAFA (ours), from the stream, own OOD detector90.4186.0483.8482.8885.79

LAFA learns one sample at a time, with no stored data and with its own OOD detector, and still stays within 0.8 points of the same classifier trained offline with all labels (85.79% vs. 86.59%). Even offline, angular LDA is 4.2 points more accurate than ordinary LDA (86.59% vs. 82.36%), so the angular statistics help independently of streaming.

Reviewer_AVcKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LAFA addresses image classification after deployment, where a model must detect unfamiliar samples, obtain their labels, and learn from them without storing examples for replay. The method uses a frozen pretrained backbone followed by a random projection and ReLU. Class directions and shared dispersion statistics are estimated from normalized features. At inference, TMA restores the feature magnitude, and the resulting scores determine class predictions and label requests. Compared with PLDA, LAFA changes the feature representation, online statistics, and inference rule. The experiments cover eight datasets and multiple backbones. They evaluate both random and class-incremental sample arrival, as well as operation without first-session adaptation.

Critical strengths

1.The consistent gains over PLDA are the strongest part of the evaluation. Average accuracy improves by about 2.5–3.3 percentage points, and the advantage holds across backbones, arrival orders, and settings without first-session adaptation.

2.LAFA’s compact state and runtime close to PLDA’s are practical strengths. It maintains class directions and a shared matrix without replay examples, while handling both label requests and learning from queried samples in the tested setting.

Critical weaknesses

1.The F-score comparison relies on an unclear OOD definition. Novel classes become ID after 30 samples, yet all initially novel classes are also described as OOD. If evaluation follows each method’s queries and class transitions, the positive instances may differ across methods, including PLDA.

2.The accuracy–cost trade-off remains unclear. Increasing the well-learned threshold raises F-score but also changes how long labels are requested. Without query counts, a higher F-score cannot establish better labeling efficiency.

3.The TMA ablation shows a substantial benefit but does not isolate sample-specific magnitude. Retaining the norm also changes the balance between the linear term and class-dependent bias. Without a fixed-norm control, global score scaling remains an alternative explanation.

Questions for authors

1.For F-score, do well-learned novel-class samples remain OOD positives? If not, is their status determined before the current prediction and update using each method’s acquired labels? Please clarify the TP/FP/FN counting rule.

2.How many label requests do LAFA and PLDA make, separately for base and novel classes?

3.Does the implemented query rule include the emerging-class condition omitted from Eq. (13)?

4.Could you compare TMA and the −M variant with a constant norm chosen only from base-class data, recalibrating the OOD threshold in the same way? Accuracy and F-score on one benchmark would help distinguish sample-specific magnitude from score-scaling effects.

5.Is Eq. (6) an approximation to Eq. (5), or a separate online dispersion estimator? Updating the unit direction in Eq. (7) changes historical residuals, so Eq. (6) does not generally reproduce the batch statistic. Appendix A’s local Euclidean approximation does not establish this recurrence. If Eq. (5) is the target, could you compare the estimates and resulting decisions at a few stream checkpoints on one benchmark? The negative-forgetting result measures base-class accuracy, not agreement between the statistics.

7 comments

Thank you for the detailed review. We address your concerns below, and a revised paper is also uploaded.

W1:

My main concern is the interpretation of F-score in the dynamic setting. Section 4.3 moves a novel class into the ID set after 30 samples, but p. 7 refers to all initially novel classes as OOD. It is unclear which definition supplies the positive labels for the reported F-scores. If evaluation follows each method's acquired labels and class transitions, different methods could be evaluated on different sets of positive instances. The counting rule therefore matters for the comparison with PLDA. The missing query counts also limit what can be concluded about labeling effort. In Table 6, increasing the well-learned threshold from 5 to 50 raises CIFAR100 F-score from 37.32% to 88.60%, while changing how long labels continue to be requested. Without the actual number of requests, a higher F-score does not establish a better accuracy-cost trade-off. The paper needs an explicit evaluation rule and query statistics alongside the accuracy results.

Q1:

For the reported F-scores, does a sample from a well-learned novel class remain an OOD positive? If not, is its status determined before the current prediction and update, using each method's own acquired labels? Please specify the TP/FP/FN counting rule and report actual label-request counts for LAFA and PLDA, split between base and novel classes. Also clarify whether the implemented query rule includes the emerging-class condition omitted from Eq. (13).

Response:

Sorry about the confusion. In page 7 (in the original paper), we describe the initial experiment setup with ID being the base classes (learned before deployment) and OOD being the rest of the classes. However, as the deployment progresses, those well-learned classes after 30 samples will no longer be considered as OOD classes subsequently. This is the rule used to determine an OOD positive and for computing TP/FP/FN and F-score. We follow the same rule as in PLDA.

The implemented query rule does include the emerging-class condition. It is stated in the sentence right before Eq. (13): "A sample is flagged as OOD if it is predicted to be in CE\mathcal{C}^E or its score falls below a threshold τ\tau", where CE\mathcal{C}^E is the set of emerging classes. The equation itself displayed only the threshold case; both cases are now written inside Eq. (13) in the revised paper.

The table below gives the label requests of LAFA and PLDA, split by class type. Both methods see the same streams, and all numbers are averaged over 3 runs. Accuracy (%) is measured on the test set of all classes (base and novel) at the end of the stream.

DatasetMethodAccuracy (%)Total queries / stream lengthUseful queries (novel, not yet well-learned)Wasted queries (base classes)Wasted queries (well-learned novel)
CIFAR100LAFA (ours)90.411855 / 5000145533367
CIFAR100PLDA87.331593 / 5000140013953
TinyImageNetLAFA (ours)86.043836 / 100002959627250
TinyImageNetPLDA84.264310 / 100002992839480
ImageNet-RLAFA (ours)83.841363 / 248511781841
ImageNet-RPLDA81.661452 / 248511912593
CUBLAFA (ours)82.88331 / 600277540
CUBPLDA82.05370 / 600290800

LAFA is more accurate on all four datasets, and it asks for 11%, 6% and 11% fewer labels on TinyImageNet, ImageNet-R and CUB. On CIFAR100 it asks for more labels in total, but it passes PLDA's final accuracy early: after 40% of the stream, LAFA already reaches 88.4% with 1004 queries, while PLDA ends the stream at 87.5% with 1593 queries (Table 11 in the revised paper shows more details on the accuracies of the methods at certain stages of the stream). This also shows that the classification system LDA in PLDA is poorer than angular LDA in LAFA.

Table 6 (Table 7 in the revised paper) is a sensitivity analysis. With a larger well-learned (WL) threshold, a novel class stays "not yet learned" for longer, so more of its samples count as positives, and most of the F-score increase comes from this. This is why we did not choose the threshold by F-score. We chose 30 because accuracy saturates around this value while the number of queries keeps growing: on CIFAR100, raising the threshold from 30 to 50 improves accuracy only 0.5%, but needs 2627 instead of 1855 queries (42% more), the detailed results of LAFA on CIFAR100 are given in the table below:

WL threshold51020304050
Accuracy84.3088.1889.9590.4190.6890.90
F36.9257.9474.9182.4386.4588.70
Queries90110081418185522922627
used246491976145519192274
wasted: base324317325333345353
wasted: WL33220111867280

W2:

The TMA ablation does not isolate the claimed role of sample-specific magnitude. Removing TMA lowers average accuracy from 85.40% to 77.10% in Table 1, so the inference rule clearly matters. However, Eq. (11) shows that retaining the norm also changes the linear term relative to the class-dependent bias. The raw-versus-unit-norm comparison changes both sample-specific information and the overall balance of the score. I am not fully convinced that the large gain can be attributed to the former. A fixed-norm control would distinguish this explanation from a global score-scaling effect.

Q2:

Could you compare TMA with a constant norm chosen using only base-class data, recalibrating the OOD threshold by the same procedure? Accuracy and F-score on one representative benchmark, alongside the existing -M variant, would help determine whether sample-specific norms add value beyond changing score scale.

Response:

Let zz be the projected feature of a test sample, r=∥z∥r = \|z\| its length (norm), and z^=z/r\hat{z} = z/r its direction (unit length). For class kk, μk\mu_k is the class mean direction (unit length) and Λ\Lambda is the precision matrix, that is, the inverse of the shared covariance matrix. The class score of Eq. (11) can then be written as

hk(z)=r z^⊤Λμk−bk,bk=12 μk⊤Λμk,h_k(z) = r\,\hat{z}^\top \Lambda \mu_k - b_k, \qquad b_k = \tfrac{1}{2}\,\mu_k^\top \Lambda \mu_k,

where the first term measures how well the direction of the sample matches class kk, and bkb_k is a class-dependent bias (one number per class, the same for every sample).

Test-time magnitude augmentation (TMA) uses the raw feature at test time, which multiplies the first term by rr and so sets the balance between the direction match and the class bias bkb_k. This balance matters in our setting. The shared covariance is estimated mostly from base-class data, which makes bkb_k larger for the few-shot novel classes: at the end of the stream it is on average 3.2× (CIFAR100), 2.8× (TinyImageNet), 2.2× (ImageNet-R) and 1.1× (CUB) larger for novel than for base classes. Without rebalancing, this bias pushes predictions away from the novel classes. On CIFAR100, novel-class accuracy is 68.7% without scaling and 88.2% when the first term is scaled by the norm rr.

Before choosing TMA, we tried the control you suggested: replacing rr by a constant cc, so that hk=c z^⊤Λμk−bkh_k = c\,\hat{z}^\top \Lambda \mu_k - b_k. We use rˉ\bar{r} as the average norm of the projected features computed on base-class training data only.

Inference ruleCIFAR100TinyImageNetImageNet-RCUBAverage accuracy (%)Average F-score (%)
c=1c = 1 (no magnitude; the −M variant)81.8879.6173.6174.4877.3984.32
c=12rˉc = \frac{1}{2}\bar{r}88.6884.7082.0683.4984.7386.10
c=rˉc = \bar{r}90.6986.2483.9282.9285.9486.73
c=2rˉc = 2\bar{r}90.7186.2183.3481.7085.4986.98
c=4rˉc = 4\bar{r}90.1285.9782.6281.0584.9487.02
No bias term (cc very large)89.1585.3881.5980.2784.1087.12
TMA, raw norm rr (LAFA, ours)90.4186.0483.8482.8885.7986.52

It is correct that a constant can reach TMA's accuracy, but only if it is tuned, and the best value differs between datasets: 12rˉ\frac{1}{2}\bar{r} on CUB, rˉ\bar{r} on TinyImageNet and ImageNet-R, and 2rˉ2\bar{r} on CIFAR100. A constant that is too small fails (c=1c = 1 gives 77.39% on average). A constant is therefore a new hyperparameter, and after deployment it cannot be tuned on the novel classes, because they are not known in advance. We did not want to add such a parameter. TMA takes the scale from each test feature itself and matches the best constant within one standard deviation on every dataset (85.79% vs. 85.94% on average). So the raw norm automatically gives the right balance between the direction term and the class bias. This automatic rebalancing is what TMA contributes, and it raises average accuracy from 77.39% (c=1c = 1) to 85.79%. Sec. 3.4 now describes TMA's bias shrinkage role in the revised paper.

Q3:

Is Eq. (6) intended to approximate Eq. (5), or to define a separate online dispersion estimator? Updating the unit direction in Eq. (7) changes previous samples' residuals, so Eq. (6) does not generally reproduce the batch statistic. Appendix A motivates a local Euclidean approximation, but does not establish this recurrence. If Eq. (5) is the target, could you compare the estimates and resulting decisions at a few stream checkpoints on one benchmark? The negative-forgetting result measures base-class accuracy, not agreement between the two statistics.

Response:

Eq. (6) defines a separate online estimator; it is not meant to reproduce Eq. (5) exactly at every step. It is a running average of the shared covariance over all NN samples: each new sample adds its residual (z~−μy)(z~−μy)⊤(\tilde{z} - \mu_y)(\tilde{z} - \mu_y)^\top, and the residuals of past samples are kept as they are. Because Σ\Sigma is shared by all classes, the correction factor uses the total count NN. The first sample of a new class is measured against μy=0\mu_y = 0, since the class has no mean yet.

During development we also implemented an update that reproduces Eq. (5) exactly at every step, at the same cost of O(D2)O(D^2) per sample. If sks_k is the sum of the normalized features of class kk, the within-class scatter equals ∑i:yi=kz~iz~i⊤−βksksk⊤\sum_{i: y_i = k} \tilde{z}_i \tilde{z}_i^\top - \beta_k s_k s_k^\top with βk=(2∥sk∥−nk)/∥sk∥2\beta_k = (2\|s_k\| - n_k)/\|s_k\|^2, so it can be updated exactly from the class sums that LAFA already stores. We chose Eq. (6) because it is as good on three datasets and better on CUB.

For comparison, we ran LAFA end to end with each estimator. Results are averaged over 3 runs, and accuracy (%) is on the test set of all classes.

CIFAR100TinyImageNetImageNet-RCUB
Accuracy (%), our covariance update (Eq. 6)90.41 ± 0.5186.04 ± 0.4283.84 ± 0.2082.88 ± 0.40
Accuracy (%), exact update of Eq. (5)90.56 ± 0.2485.98 ± 0.3583.80 ± 0.1582.14 ± 0.51

On CIFAR100, TinyImageNet and ImageNet-R, which have tens of samples per novel class, their accuracies differ by less than one standard deviation. On CUB, where the data is more scarce, Eq. (6) is 0.74 points more accurate. In Eq. (6), the first sample of a new class is measured against μy=0\mu_y = 0, and the first residuals of every class receive full weight. Both increase the variance along the directions of the new class, which works like a prior variance for a class with very few samples and lowers the class bias bk=12μk⊤Λμkb_k = \frac{1}{2}\mu_k^\top \Lambda \mu_k of new classes. With the exact update, the average bias of novel classes on CUB nearly doubles (836 vs. 480), and novel-class accuracy falls from 76.27% to 74.84%, while base-class accuracy is unchanged.

Unlike the negative-forgetting result, this comparison looks directly at the two statistics and the decisions they produce. The exact update and the comparison are now in Appendix A of the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.