acceptodds
Withdrawn from ICLR 2027

Reliable Deployment-Time Learning: From Label Acquisition to Error Recovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

This paper studies the reliability of deployment-time learning. As AI systems increasingly need to acquire new knowledge after deployment, a learning agent must detect novel objects or classes unseen during training, acquire their labels from human users or other external knowledge sources, and incrementally incorporate this new knowledge into the deployed model. However, externally acquired labels may not always be directly suitable for learning: they may use synonyms or hypernyms of the intended class names, contain errors, or even be maliciously provided. Moreover, learning new classes should not compromise previously acquired knowledge, and errors that have already been learned should be recoverable. We address these challenges based on the learning method of incremental Linear Discriminant Analysis (iLDA) over representations produced by strong pretrained foundation models. iLDA is particularly suitable for deployment-time learning because it supports efficient online incremental updates while preserving previously acquired knowledge with high accuracy. Building on this foundation, we develop methods for reliable label acquisition that resolve semantically related labels and detect potentially incorrect labels before learning, as well as an efficient recovery mechanism for removing the influence of incorrect labels that have already been incorporated into the model. Experimental results on multiple benchmark datasets demonstrate the effectiveness of the proposed methods across label acquisition, incremental learning, and error recovery, providing a practical framework for reliable learning after deployment.

Primary area:
learning on time series and dynamical systems
Keywords:
deployment-time learning, open-world learning, machine unlearning, exact unlearning, label noise, mislabeling, incremental linear discriminant analysis, vision-language models, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_G699OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2(−1)
Soundness
2
Read review
Paper summary

The paper studies deployment-time classification with unreliable external labels and varying class names. It combines frozen visual representations and incremental LDA with language-based name resolution, cross-source label verification, and reversible statistical updates for correcting previously learned labels. Experiments on four image benchmarks with simulated users demonstrate reduced label fragmentation and exact recovery to the corresponding clean-label classifier when corrected labels are supplied.

Critical strengths

The vocabulary ladder provides useful evidence that valid alternative names can fragment a classifier’s label space even when the underlying labeling error rate is unchanged (Fig. 2). The reversible sufficient-statistic formulation also gives a clear mechanism for deletion, relabeling, and class merging, supported by numerical agreement with batch reconstruction (Sec. 5 and Table 4).

Critical weaknesses
  1. The paper presents a complex system involving a visual DNN, a VLM, and an LLM, with dedicated technical designs at each stage. This complexity makes it particularly important to establish the practical value of the complete workflow. However, the discussion of related workflows is insufficient: PLDA [1] is cited, but its relationship to the evaluated baselines is not clarified, while complementary memory systems such as [2] and retrieval-augmented MLLM recognition such as [3] are not discussed. Although these methods do not share all of the paper's assumptions, a detailed comparison of their settings, capabilities, and deployment constraints is needed to clarify what the proposed workflow adds and when its additional stages are justified.

  2. The baselines in Table 1 mainly vary label-handling rules within the same classifier. These comparisons support the proposed label-processing components, but do not test whether an advanced closed-source MLLM, paired with a simple editable example memory, could provide comparable classification, label normalization, and recovery from corrected feedback. A representative comparison with matched access to discovered class names, user feedback, and corrections, together with reporting inference costs and memory requirements, would help establish the practical benefit of the proposed workflow over this alternative. Without such evidence, the system-level advantage remains insufficiently established.

[1] Learning After Model Deployment. ECAI 2025.

[2] Continual Learning in Open-vocabulary Classification with Complementary Memory Systems. TMLR 2024.

[3] RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition. TIP 2026.

Questions for authors

Please refer to the Weaknesses above.

4 comments

Thank you for the review. We address your concerns below, and a revised paper is also uploaded, with the revised parts shown in blue.

W1:

The paper presents a complex system involving a visual DNN, a VLM, and an LLM, with dedicated technical designs at each stage. This complexity makes it particularly important to establish the practical value of the complete workflow. However, the discussion of related workflows is insufficient: PLDA [1] is cited, but its relationship to the evaluated baselines is not clarified, while complementary memory systems such as [2] and retrieval-augmented MLLM recognition such as [3] are not discussed. Although these methods do not share all of the paper's assumptions, a detailed comparison of their settings, capabilities, and deployment constraints is needed to clarify what the proposed workflow adds and when its additional stages are justified.

Response:

PLDA [1] learns new classes after deployment with incremental LDA (iLDA) on a frozen pre-trained model, and assumes every label it receives is correct and uses the standard class name. The "No curation" row of Table 1 is exactly this setting. Our checks sit between the users and this learner, and our edit rule (Proposition 1) adds exact removal, relabeling and merging, which [1] does not have. The complementary memory system [2] pairs a CLIP zero-shot classifier with a growing memory of labeled examples, and RAR [3] retrieves candidate names from a memory and lets a multimodal LLM (MLLM) pick one. Both assume that the names and labels in their memory are correct and consistent; neither checks whether a user's label can be trusted, or whether "aubergine" and "eggplant" are the same class.

[1] PLDA[2] Complementary memory[3] RAROurs
Learns new classes from user labelsyesyesyesyes
Decides whether two names are one classnononoyes
Detects wrong labels before learningnononoyes
Removes or relabels a learned label exactlynoyes (delete example)yes (delete entry)yes (Proposition 1)
Merges two classesnoyes (rename examples)yes (rename entries)yes, from class counts and sums alone
Model size as the stream growsfixedgrowsgrowsfixed
Cost of one predictionone matrix-vector productCLIP + nearest-neighbour searchretrieval + one MLLM call (about 2 s)one matrix-vector product (20 to 50 µs)

Each extra stage fixes one specific failure, and it pays off exactly when that failure is present. The LLM synonym check adds +5.4, +4.9, +2.4 and +0.3 points on CIFAR-100, TinyImageNet, ImageNet-R and CUB-200, following how many different names users type per class (about 10, 10, 5 and 2). The CLIP check is what separates wrong labels from correct ones, and removing it costs 11.5 and 11.2 points on CIFAR-100 and TinyImageNet. The exact edits matter whenever a learned label later turns out wrong: after an audit they bring CIFAR-100 from 59.3% to 88.2%, identical to retraining. Finally, the Object-First check is the better choice when there are many samples per class (+2.7 and +5.9 points over Term-First on CIFAR-100 and TinyImageNet), and the Term-First check when there are few. The revised paper adds this comparison to Section 2 and Appendix N.

[1] Learning After Model Deployment. ECAI 2025.

[2] Continual Learning in Open-vocabulary Classification with Complementary Memory Systems. TMLR 2024.

[3] RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition. TIP 2026.

W2:

The baselines in Table 1 mainly vary label-handling rules within the same classifier. These comparisons support the proposed label-processing components, but do not test whether an advanced closed-source MLLM, paired with a simple editable example memory, could provide comparable classification, label normalization, and recovery from corrected feedback. A representative comparison with matched access to discovered class names, user feedback, and corrections, together with reporting inference costs and memory requirements, would help establish the practical benefit of the proposed workflow over this alternative. Without such evidence, the system-level advantage remains insufficiently established.

Response:

We built this alternative following RAR [3], with the same access as our method. The memory holds the base-class training images with their correct names plus every stream sample with the label it was learned under. For each test image, the 20 most similar memory examples are retrieved with CLIP (as in RAR), and up to 5 of their names are given to a closed-source MLLM (GPT-5.6-Terra, the same model as our synonym check) together with the image, which picks one. We tested three memories: raw (all user labels as given), curated (only labels accepted by our Term-First check, with its synonym merges) and repaired (after an audit corrected every label by editing the memory). Both systems are scored on the same 2,000 test images of CIFAR-100 and CUB-200 (matched accuracy %):

SystemMemoryCIFAR-100CUB-200
MLLM + editable memoryraw59.352.8
curated (our Term-First labels)64.355.0
repaired67.058.5
iLDA (ours)raw59.472.9
curated (our Term-First labels)79.277.5
repaired88.884.1
MLLM + editable memoryiLDA (ours)
Time per prediction1.6 to 2.2 s0.00002 to 0.00005 s
Language-model tokens per prediction (input / output)121 / 53 (CIFAR-100), 302 / 80 (CUB-200)none
What is stored27,500 or 3,300 memory examples, growing with the streamone 5,000 × 5,000 matrix and one sum per class, fixed
Images sent to an external serviceevery test imagenone

iLDA is more accurate with every memory, by 14.9 to 25.6 points once the memory is curated or repaired. The MLLM also benefits from our label processing (+5.0 and +2.2 points from raw to curated), which shows that it does not replace the checks: with raw labels it still has to choose among the users' split names. Both systems apply corrections exactly, but after repair iLDA reaches 88.8% and 84.1% against 67.0% and 58.5%, while predicting more than 30,000 times faster, using no tokens and sending no images outside. This comparison is now in Appendix N.

Dear Reviewer G699,

Thank you again for your time and constructive feedback. As the discussion period ends in less than 2 days, we wanted to kindly check whether our responses have addressed your concerns. If any questions remain, we would be glad to clarify them before the deadline. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.

Best regards, The Authors

Thank you for the detailed response. The results of directly using the MLLM are helpful. However, my concern remains. Given the complex system illustrated in the paper, it is hard to grasp the most crucial and real incremental technical novelty of the proposed system. Therefore, I think this paper still requires further refinement.

Reviewer_ut7cOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies learning after deployment when labels may be wrong or inconsistently named. It uses incremental LDA on frozen pretrained features. Term-First resolves synonyms with a language model and checks agreement across labeling sources, while Object-First groups observations visually before naming them. Sufficient-statistic updates allow sample removal, relabeling, and class merging with results equivalent to rebuilding the classifier. Experiments on four image benchmarks with simulated annotators assess matched accuracy, label fragmentation, numerical recovery, and self-correction. Further analyses examine vocabulary variation, component contributions, and editing costs.

Critical strengths
  1. The exact updates support sample removal, relabeling, and class merging without retraining on all retained samples. With frozen representations, numerical checks confirm that the updated classifier matches one rebuilt on the edited data, allowing earlier labeling decisions to be corrected.

  2. Batch updates substantially reduce relabeling time: correcting 1,000 samples takes 87.8 ms, compared with 61.5 s sequentially. The low-rank precision-matrix refresh also reduces the cost of updating the classifier, taking 24 ms versus 2551 ms for full factorization.

Critical weaknesses
  1. Both method families tune covariance shrinkage, but Appendix A does not clarify whether the images used for “validation Seed 0” are separate from those in the three evaluation runs. This matters because changing shrinkage shifts Object-First accuracy on CIFAR-100 by 7.5 points.

  2. Redundant Majority requests two extra annotations, while the proposed method allows follow-up verification and audits of quarantined samples. Table 1 reports neither query counts nor the proportion of samples receiving extra annotations, leaving the human effort behind each method’s accuracy unclear for each dataset.

Questions for authors
  1. How is user reliability p^u\hat{p}_u in Eq. (18) initialized for new users and updated during deployment? Which labels or feedback are used to judge whether a user is correct?

  2. What sample-level information is retained for removal, relabeling, and re-voting? Are individual feature vectors and annotation histories stored alongside class statistics, and how does storage grow as more samples arrive?

4 comments

Thank you for the detailed review. We address your concerns below, and a revised paper is also uploaded, with the revised parts shown in blue.

W1:

The validation protocol needs clarification. Appendix A, Table 3 states that covariance shrinkage is selected on "validation Seed 0," but Section 6 does not clearly identify the image-level validation/test partition or explain its relationship to the three evaluation seeds in Table 1. This matters because Appendix A reports a 7.5-point change in Object-First accuracy on CIFAR-100 when shrinkage is changed. Both method families receive tuning, and different selected values are not inherently unfair. However, the separation between model selection and final evaluation remains difficult to verify from the description.

Response:

The shrinkage ϵ\epsilon (epsilon) is the only value we tune. It controls how much the shared covariance matrix Σ\Sigma is pulled toward the identity matrix II before inversion: the model uses Λ=((1−ϵ)Σ+ϵI)−1\Lambda = ((1-\epsilon)\Sigma + \epsilon I)^{-1}, so a larger ϵ\epsilon means stronger regularization. In the submission, ϵ\epsilon was selected on seed 0, scored on the test images, with one value per method family.

To make the separation clear and easy to verify, the revised paper now holds out 10% of the training images of each class as a validation split. These images are never in the stream or the test set. We tune with a separate seed, and every method, baselines included, gets the value that works best for it on the validation split. We only accept values that keep at least 97% of the true classes, because merging two classes can raise matched accuracy while destroying a class. The test images are used only for the final numbers. The selected values are:

MethodCIFAR-100TinyImageNetImageNet-RCUB-200
No curation10−610^{-6}10−710^{-7}10−510^{-5}10−410^{-4}
Redundant majority10−610^{-6}10−710^{-7}10−510^{-5}10−510^{-5}
VLM veto10−410^{-4}10−510^{-5}10−510^{-5}10−510^{-5}
Agreement gate10−710^{-7}10−610^{-6}10−510^{-5}10−410^{-4}
Term-First check (ours)10−610^{-6}10−810^{-8}10−510^{-5}10−410^{-4}
Object-First check (ours)10−410^{-4}10−410^{-4}10−410^{-4}10−510^{-5}
VLM veto, given all terms10−410^{-4}10−510^{-5}10−410^{-4}10−410^{-4}
Clean oracle10−510^{-5}10−810^{-8}10−510^{-5}10−510^{-5}

With these values, the revised Table 1 (matched accuracy %, mean ± standard deviation over 3 seeds) is:

MethodCIFAR-100TinyImageNetImageNet-RCUB-200
No curation59.3 ± 1.257.9 ± 1.058.3 ± 0.473.6 ± 1.2
Redundant majority59.3 ± 1.258.2 ± 1.058.5 ± 0.573.7 ± 1.2
VLM veto74.8 ± 1.173.2 ± 0.567.9 ± 0.670.2 ± 0.5
Agreement gate71.0 ± 0.468.6 ± 0.566.5 ± 0.777.0 ± 0.3
Term-First check (ours)76.4 ± 0.273.5 ± 1.068.9 ± 0.277.3 ± 0.4
Object-First check (ours)79.1 ± 0.879.4 ± 0.665.6 ± 0.571.3 ± 0.9
VLM veto, given all terms (reference)85.4 ± 0.182.8 ± 0.275.7 ± 0.171.4 ± 0.4
Clean oracle (reference)88.1 ± 0.085.1 ± 0.178.0 ± 0.384.0 ± 0.2

The conclusions stay the same: the Object-First check is best on CIFAR-100 and TinyImageNet, where each class has about 50 stream samples, and the Term-First check is best on ImageNet-R and CUB-200, where there are only 11 and 3.

The 7.5-point change you mention comes from running the Object-First check with the value chosen for the other method family. Unlike the other methods, the Object-First check also uses Λ\Lambda to test whether two groups of images belong to the same class (Hotelling's T2T^2, Eq. 6). With too little regularization, two groups of the same class look further apart than they are, so the check keeps opening new classes. On the CIFAR-100 validation split it reaches 77.7% at ϵ=10−4\epsilon = 10^{-4} and 72.2% at 10−510^{-5}, while the Term-First check prefers smaller values (76.3% at 10−610^{-6}, 74.8% at 10−410^{-4}). That is why each method now gets its own value. Section 6 and Appendix A describe the full protocol.

W2:

Annotation costs are missing from the comparison. The methods in Table 1 use different forms of human feedback. Redundant Majority requests two additional annotations (Section 6; Appendix E, Table 7), and the proposed method can request follow-up verification or audit quarantined samples (Algorithm 1). Table 1 does not report the actual query counts or the proportion of samples receiving additional annotations. It is therefore unclear how much human input each method requires to achieve the reported accuracy. These annotation costs should be reported alongside accuracy for each dataset.

Response:

All methods share the same human budget of 20 cost units per 100 stream samples, so no method wins by asking more. A yes/no verification question ("Is this class yy?") costs 1 unit, a same/different question about two groups of images costs 1.3, a naming question ("What is this?") costs 3, and each extra annotation bought by Redundant Majority costs 1. The table below shows what each method actually spent in Table 1 (mean over 3 seeds):

DatasetMethodAccuracy (%)Cost units spentVerification / naming / same-different questionsStream samples that got a question (%)
CIFAR-100 (5,000 stream samples)No curation59.300 / 0 / 00
Redundant majority59.3998998 extra annotations10.0
VLM veto74.800 / 0 / 00
Agreement gate71.0998134 / 288 / 07.1
Term-First check (ours)76.4999128 / 290 / 07.1
Object-First check (ours)79.11,020856 / 53 / 50.4
TinyImageNet (10,000)No curation57.900 / 0 / 00
Redundant majority58.21,9981,998 extra annotations10.0
VLM veto73.200 / 0 / 00
Agreement gate68.61,998361 / 546 / 07.7
Term-First check (ours)73.51,999343 / 552 / 07.6
Object-First check (ours)79.42,0191,367 / 215 / 50.6
ImageNet-R (2,485)No curation58.300 / 0 / 00
Redundant majority58.5496496 extra annotations10.0
VLM veto67.900 / 0 / 00
Agreement gate66.5496228 / 89 / 010.8
Term-First check (ours)68.9496204 / 97 / 010.2
Object-First check (ours)65.6517334 / 59 / 40.9
CUB-200 (600)No curation73.600 / 0 / 00
Redundant majority73.7118118 extra annotations9.8
VLM veto70.200 / 0 / 00
Agreement gate77.011623 / 31 / 06.7
Term-First check (ours)77.311423 / 30 / 06.6
Object-First check (ours)71.300 / 0 / 00

For the same human cost as Redundant Majority, the Term-First check is 3.6 to 17.1 points more accurate. A majority vote can fix a random mistake, but it cannot tell that two valid names mean the same class. The Object-First check asks mostly about whole groups of images, so one question covers many samples. On CUB-200 it asks nothing, because it only asks about a group once the group has 4 samples and CUB-200 has 3 per class. Outside the human budget, the Term-First check asks the language model once per new name (answers are cached), and all methods except No curation and Redundant Majority run one CLIP check per sample. This table is now Appendix J and is referenced in the caption of Table 1.

Q1:

Estimation of user reliability. How is the user accuracy estimate (p^u\hat p_u) in Appendix C.3, Eq. (18) computed and updated during deployment? Which labels or feedback serve as the reference for judging whether a user is correct, and how is reliability initialized for a new user?

Response:

User reliability is only used by the Object-First check; the Term-First check judges each label on its own. Here p^u\hat p_u is the estimated chance that user uu gives a correct name. For each user we count correct answers aua_u and wrong answers bub_u. A new user starts at au=3a_u = 3 and bu=1b_u = 1 (a Beta(3, 1) prior), so p^u=au/(au+bu)=0.75\hat p_u = a_u/(a_u+b_u) = 0.75.

There is no ground truth. When the agent settles the name of a group of images, using the users' votes, CLIP and any same/different answers, every user who voted on that group is scored: the settled name or one of its synonyms counts as correct, anything else as wrong, and aua_u or bub_u goes up by 1. A user only gets credit when the settled name is backed by evidence other than their own label, so an adversarial user cannot raise their own score just because nobody contradicted them. Mistakes are always counted.

The vote weight uses a cautious estimate pu−p^-_u, the mean p^u\hat p_u minus one standard deviation, so a user with few answers is not over-trusted: wu=log⁡(pu−(K−1)/(1−pu−))w_u = \log\big(p^-_u (K-1)/(1-p^-_u)\big), where KK is the number of plausible names (at most 10). We scale it so a new user counts as one vote and clip it to [−2,2][-2, 2]. A user who is right less often than chance (1/K1/K) gets a negative weight, so their vote counts against the name they gave. Appendix C.3 now gives this rule in full.

Q2:

Storage required for later corrections. What sample-level information is retained for the removal and relabeling operations in Section 5 and the re-voting procedure in Appendix F? Please clarify whether individual feature vectors and annotation histories must be stored in addition to class statistics, and how this storage grows with the deployment stream.

Response:

The model itself does not grow: it is one D×DD \times D matrix S\mathbf{S} plus a count nkn_k and a sum vector sk\mathbf{s}_k for each class kk (DD = 5,000 is the feature size). What else is kept depends on which corrections the operator wants to keep possible:

CorrectionWhat has to be kept
Merging two classes (Eq. 13)Nothing beyond the class counts and sums
Answering a held-back sample later (Algorithm 1)The sample's feature, until it is answered
Periodic re-vote (Appendix F)Feature, CLIP embedding, typed name and label, only for decisions that rested on the agent's own vote
Removing or relabeling any learned sample, or all labels of one user (Table 2)Each sample's feature, label, user id and typed name

This storage grows linearly with the number of samples kept correctable. Since the backbone is frozen and the projection is fixed by a seed, it is enough to store the 768-dimensional backbone feature, or just a pointer to the image. On CIFAR-100 this is 26 MB for all 5,000 stream samples. An operator with a memory limit can keep samples correctable for a fixed window, after which class-level edits such as merges still work. Section 5 and Appendix K now explain this, and Section 3 clarifies that learning keeps only the class statistics, while keeping a decision correctable needs the stored feature.

Dear Reviewer ut7c,

Thank you again for your time and constructive feedback. As the discussion period ends in less than 2 days, we wanted to kindly check whether our responses have addressed your concerns. If any questions remain, we would be glad to clarify them before the deadline. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.

Best regards, The Authors

Reviewer_WfMSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on a new pipeline of deployment learning which consists of new item/category discovery, learning with unreliable labels, and recovering from error. Defining such a pipeline is interesting and new. To achieve deployment learning, this paper focuses on learning with unreliable labels and recovery with true labels based on the incremental linear discriminate analysis (iLDA). On top of the iLDA, a pipeline fixing the unreliable labels including Tri-Source Consensus and LLM Synonym Disambiguation is proposed. Also, objective-based verification strategy based on the visual features is also discussed. To validate the performance of proposed strategy, datasets are built by similation. On the curated dataset, the performance of proposed strategy is validated.

Critical strengths
  1. This paper is well written and easy to follow.
  2. Focusing on deployment learning and proposing the framework is new and interesting.
  3. The proposed method is simple but effective on the dataset provided.
Critical weaknesses
  1. The iLDA only focuses on the decision boundaries and no representation is involved. Although the iLDA can achieve non-forgetting property with a frozen PTM, the learning process does not involve the representation learning, which ignores the potential of representation evolution.
  2. Baselines in this paper are narrow since they are only builded upon iLDA. Although these strategies are effective upon iLDA, iLDA is only one format of learning from new data or unlearning. To better demonstrates the effectiveness of iLDA, other traditional baselines in continual learning or unlearning should be included. It is also good to see if these strategies can also work with other types of continual learners.
  3. Proposed datasets and evaluation metrics need further illustrations. For the evaluation dataset, including both the statistics and several examples can better demonstrate the details, difficulty of the data. Also, it is hard to understand the evaluation metrics if the readers are not familiar.
  4. Lack of discussion between test-time adapation (TTA). TTA also focues on adapting the model during the test time, the relation and difference with the deployment learning and TTA should be provided. Also, several TTA algorithms should be included in the experiments to validate the superiority of iLDA (if so).
Questions for authors

Please refer to the weaknesses.

4 comments

Thank you for the review. We address your concerns below, and a revised paper is also uploaded.

W1:

The iLDA only focuses on the decision boundaries and no representation is involved. Although the iLDA can achieve non-forgetting property with a frozen PTM, the learning process does not involve the representation learning, which ignores the potential of representation evolution.

Response:

We believe that representation learning is neither necessary nor necessarily beneficial in post-deployment continual learning, for the following three reasons.

(1) Strong performance without representation learning. Recent state-of-the-art continual learning methods increasingly rely on frozen pretrained models (PTMs), without updating their feature representations. These methods have achieved accuracy comparable to joint fine-tuning of PTMs on all available data, which is commonly regarded as an upper-bound baseline. This demonstrates that competitive continual learning performance can be achieved without additional representation learning.

(2) Rapid improvements in pretrained models. The quality of PTMs has improved dramatically in recent years, and we expect this trend to continue. For most applications, it is unlikely that the limited data available in a specific application, particularly in a post-deployment setting, can be used to learn better representations than those provided by increasingly powerful PTMs. Exceptions may exist in specialized domains involving images that differ substantially from those used to train general-purpose PTMs. Consequently, relying on strong frozen PTMs is often a more effective and practical strategy than continually updating feature representations using limited application-specific data.

(3) Empirical evidence that stronger PTMs lead to better performance. We conducted additional experiments to investigate the effect of PTM quality on the performance of our method. Specifically, we independently evaluated the complete pipeline using three different sizes of DINOv2. For each backbone, we recomputed the features of all images and replayed the entire learning stream using the same simulated users to ensure a fair comparison. The results (accuracy in %; Clean labels indicates that the learner receives the correct label for every sample) are presented below:

Frozen backboneDatasetNo curationTerm-First check (ours)GainClean labels
DINOv2 ViT-S/14 (small)CIFAR-10054.170.1+16.083.1
DINOv2 ViT-B/14 (base, used in the paper)CIFAR-10059.576.7+17.288.1
DINOv2 ViT-L/14 (large)CIFAR-10062.779.3+16.691.2
DINOv2 ViT-S/14 (small)ImageNet-R50.859.6+8.868.2
DINOv2 ViT-B/14 (base, used in the paper)ImageNet-R57.768.6+10.977.8
DINOv2 ViT-L/14 (large)ImageNet-R64.276.2+12.085.9

A better backbone lifts every method, and our check keeps a large gain with each one, so the method benefits from better representations. This table is now Appendix O, and the Limitations (Section 7) discuss the trade-off.

W2:

Baselines in this paper are narrow since they are only builded upon iLDA. Although these strategies are effective upon iLDA, iLDA is only one format of learning from new data or unlearning. To better demonstrates the effectiveness of iLDA, other traditional baselines in continual learning or unlearning should be included. It is also good to see if these strategies can also work with other types of continual learners.

Response:

Other continual learners. We could not compare with many continual learning methods because they do not have corresponding unlearning methods. The Term-First check only looks at the learner's predicted class, so it can sit in front of any classifier. We tried it with two standard continual learners on the same features: NCM (nearest class mean, which is iLDA without the shared covariance matrix) and an online linear head trained by SGD with a replay buffer of 2,000 samples. Results are averaged over 3 seeds:

LearnerCIFAR-100TinyImageNetImageNet-RCUB-200
Gain of the Term-First check over no curation (points)iLDA (ours)+17.0+16.0+11.0+3.8
NCM+13.2+9.6+4.7+0.9
Linear head + replay+6.8+8.8+4.3+0.7
Accuracy with correct labels (%)iLDA (ours)88.185.178.084.0
NCM85.282.171.781.9
Linear head + replay85.881.473.273.5

The check helps every learner on every dataset. The gain is largest where users type many different names for the same class (about 10 per class on CIFAR-100 and TinyImageNet) and smallest on CUB-200 (about 2). We chose iLDA because it is the most accurate learner once labels are correct, which is where the checks and later corrections take it, because its remaining errors can be removed exactly (below), and because the Object-First check needs its shared covariance matrix to decide whether two groups of images are the same class (Eq. 6).

Unlearning baselines. We could not compare with most existing unlearning methods as they are not designed for continual learning and cause catastrophic forgetting.

We compare our exact unlearning with exact unlearning for NCM, full retraining of a linear head, and three standard unlearning methods on the linear head: fine-tuning on the corrected data, NegGrad+ [Kurmanji et al., 2023] (fine-tuning plus gradient ascent on the old wrong labels; best of 6 weights) and SISA [Bourtoule et al., 2021] (5 data shards, only affected shards retrained). Accuracy (%) after repair, mean over 3 seeds.

Learner and repair methodSame model as retraining?CIFAR-100TinyImageNetImageNet-RCUB-200
iLDA, exact unlearning (ours)yes88.285.178.083.7
NCM, exact unlearningyes85.282.171.781.9
Linear head, retraining from scratchyes85.881.473.273.5
Linear head, fine-tuningno85.282.774.674.0
Linear head, NegGrad+no85.683.175.073.5
Linear head, SISAyes, for its own 5-part model85.783.568.358.2

Our exact unlearning is the most accurate on every dataset and gives exactly the retrained model (the difference is about 10−1510^{-15}, i.e. floating-point rounding). Fine-tuning and NegGrad+ are approximate, so a removed label can leave a trace in the model, which matters when a user's data must be deleted. SISA struggles when each shard sees only a few samples per class (ImageNet-R and CUB-200). Both tables are now in Appendices L and M.

W3:

Proposed datasets and evaluation metrics need further illustrations. For the evaluation dataset, including both the statistics and several examples can better demonstrate the details, difficulty of the data. Also, it is hard to understand the evaluation metrics if the readers are not familiar.

Response:

The images are the real benchmark images; only the users' answers are simulated. The main statistics (averaged over 3 seeds) are:

CIFAR-100TinyImageNetImageNet-RCUB-200
Classes (base + novel)50 + 50100 + 100100 + 100100 + 100
Training images learned before deployment22,50045,00011,0512,700
Stream samples labeled by users after deployment5,00010,0002,485600
Stream samples per class (median)5050113
Test images10,00010,0006,0825,794
Wrong labels (%)21.220.722.022.3
Synonyms / broader terms / foreign tokens (%)8.3 / 7.9 / 8.37.5 / 8.3 / 8.38.5 / 8.0 / 8.64.7 / 1.9 / 7.8
Different names typed for one class (mean)10.610.55.02.0

As an example, this is what different user types provided for two TinyImageNet classes in one run:

User typeClass "organ"Class "thatch"
Reliable"organ" ×12"thatch" ×11
Uses a synonym"pipe organ" ×9"thatched roof" ×11
Uses a broader term"keyboard instrument" ×6"roof" ×6
Adversarial"police van" ×6"vestment" ×4

Without curation, one class like "organ" ends up as three or more separate classes. Appendix I now gives both tables with examples from all four datasets.

For the metrics, take a true class eggplant with 10 test images, where the agent predicts 6 as its group "eggplant" and 4 as a second group "aubergine". Matched accuracy pairs each learned group with at most one true class (Hungarian matching), so only "eggplant" counts and the accuracy is 6/10 = 60%; splitting a class is penalized even when both names are valid. Label-space inflation is the number of learned groups divided by the number of true classes, here 2/1 = 2.0, where 1.0 is ideal. Relative covariance gap, which computes the Relative Frobenius Gap between the covariance matrices, checks whether a repaired model equals the model retrained from scratch. A value of 0 (about 10−1510^{-15} in practice, float64 precision) means the two models are identical. Section 6 now explains the metrics with this example.

W4:

Lack of discussion between test-time adapation (TTA). TTA also focues on adapting the model during the test time, the relation and difference with the deployment learning and TTA should be provided. Also, several TTA algorithms should be included in the experiments to validate the superiority of iLDA (if so).

Response:

Test-time adaptation (TTA) is a completely different paradigm in which a model adapts itself to the distribution of test data during inference, typically without access to the ground-truth labels of the test samples. The main goal is to improve prediction accuracy when the test-data distribution differs from the training-data distribution, a problem commonly known as distribution shift.

However, our problem is deployment-time continual learning, which needs to detect and learn completely new classes incrementally, which is not done in TTA, i.e., TTA does not learn or predict completely new classes.

Dear Reviewer WfMS,

Thank you again for your time and constructive feedback. As the discussion period ends in less than 2 days, we wanted to kindly check whether our responses have addressed your concerns. If any questions remain, we would be glad to clarify them before the deadline. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.