- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(−1)
- Soundness
- 2
Read review
Paper summary
The paper studies deployment-time classification with unreliable external labels and varying class names. It combines frozen visual representations and incremental LDA with language-based name resolution, cross-source label verification, and reversible statistical updates for correcting previously learned labels. Experiments on four image benchmarks with simulated users demonstrate reduced label fragmentation and exact recovery to the corresponding clean-label classifier when corrected labels are supplied.
Critical strengths
The vocabulary ladder provides useful evidence that valid alternative names can fragment a classifier’s label space even when the underlying labeling error rate is unchanged (Fig. 2). The reversible sufficient-statistic formulation also gives a clear mechanism for deletion, relabeling, and class merging, supported by numerical agreement with batch reconstruction (Sec. 5 and Table 4).
Critical weaknesses
-
The paper presents a complex system involving a visual DNN, a VLM, and an LLM, with dedicated technical designs at each stage. This complexity makes it particularly important to establish the practical value of the complete workflow. However, the discussion of related workflows is insufficient: PLDA [1] is cited, but its relationship to the evaluated baselines is not clarified, while complementary memory systems such as [2] and retrieval-augmented MLLM recognition such as [3] are not discussed. Although these methods do not share all of the paper's assumptions, a detailed comparison of their settings, capabilities, and deployment constraints is needed to clarify what the proposed workflow adds and when its additional stages are justified.
-
The baselines in Table 1 mainly vary label-handling rules within the same classifier. These comparisons support the proposed label-processing components, but do not test whether an advanced closed-source MLLM, paired with a simple editable example memory, could provide comparable classification, label normalization, and recovery from corrected feedback. A representative comparison with matched access to discovered class names, user feedback, and corrections, together with reporting inference costs and memory requirements, would help establish the practical benefit of the proposed workflow over this alternative. Without such evidence, the system-level advantage remains insufficiently established.
[1] Learning After Model Deployment. ECAI 2025.
[2] Continual Learning in Open-vocabulary Classification with Complementary Memory Systems. TMLR 2024.
[3] RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition. TIP 2026.
Questions for authors
Please refer to the Weaknesses above.
4 comments
W2:
The baselines in Table 1 mainly vary label-handling rules within the same classifier. These comparisons support the proposed label-processing components, but do not test whether an advanced closed-source MLLM, paired with a simple editable example memory, could provide comparable classification, label normalization, and recovery from corrected feedback. A representative comparison with matched access to discovered class names, user feedback, and corrections, together with reporting inference costs and memory requirements, would help establish the practical benefit of the proposed workflow over this alternative. Without such evidence, the system-level advantage remains insufficiently established.
Response:
We built this alternative following RAR [3], with the same access as our method. The memory holds the base-class training images with their correct names plus every stream sample with the label it was learned under. For each test image, the 20 most similar memory examples are retrieved with CLIP (as in RAR), and up to 5 of their names are given to a closed-source MLLM (GPT-5.6-Terra, the same model as our synonym check) together with the image, which picks one. We tested three memories: raw (all user labels as given), curated (only labels accepted by our Term-First check, with its synonym merges) and repaired (after an audit corrected every label by editing the memory). Both systems are scored on the same 2,000 test images of CIFAR-100 and CUB-200 (matched accuracy %):
| System | Memory | CIFAR-100 | CUB-200 |
|---|---|---|---|
| MLLM + editable memory | raw | 59.3 | 52.8 |
| curated (our Term-First labels) | 64.3 | 55.0 | |
| repaired | 67.0 | 58.5 | |
| iLDA (ours) | raw | 59.4 | 72.9 |
| curated (our Term-First labels) | 79.2 | 77.5 | |
| repaired | 88.8 | 84.1 |
| MLLM + editable memory | iLDA (ours) | |
|---|---|---|
| Time per prediction | 1.6 to 2.2 s | 0.00002 to 0.00005 s |
| Language-model tokens per prediction (input / output) | 121 / 53 (CIFAR-100), 302 / 80 (CUB-200) | none |
| What is stored | 27,500 or 3,300 memory examples, growing with the stream | one 5,000 × 5,000 matrix and one sum per class, fixed |
| Images sent to an external service | every test image | none |
iLDA is more accurate with every memory, by 14.9 to 25.6 points once the memory is curated or repaired. The MLLM also benefits from our label processing (+5.0 and +2.2 points from raw to curated), which shows that it does not replace the checks: with raw labels it still has to choose among the users' split names. Both systems apply corrections exactly, but after repair iLDA reaches 88.8% and 84.1% against 67.0% and 58.5%, while predicting more than 30,000 times faster, using no tokens and sending no images outside. This comparison is now in Appendix N.
Dear Reviewer G699,
Thank you again for your time and constructive feedback. As the discussion period ends in less than 2 days, we wanted to kindly check whether our responses have addressed your concerns. If any questions remain, we would be glad to clarify them before the deadline. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.
Best regards, The Authors
Thank you for the detailed response. The results of directly using the MLLM are helpful. However, my concern remains. Given the complex system illustrated in the paper, it is hard to grasp the most crucial and real incremental technical novelty of the proposed system. Therefore, I think this paper still requires further refinement.
Thank you for the review. We address your concerns below, and a revised paper is also uploaded, with the revised parts shown in blue.
W1:
Response:
PLDA [1] learns new classes after deployment with incremental LDA (iLDA) on a frozen pre-trained model, and assumes every label it receives is correct and uses the standard class name. The "No curation" row of Table 1 is exactly this setting. Our checks sit between the users and this learner, and our edit rule (Proposition 1) adds exact removal, relabeling and merging, which [1] does not have. The complementary memory system [2] pairs a CLIP zero-shot classifier with a growing memory of labeled examples, and RAR [3] retrieves candidate names from a memory and lets a multimodal LLM (MLLM) pick one. Both assume that the names and labels in their memory are correct and consistent; neither checks whether a user's label can be trusted, or whether "aubergine" and "eggplant" are the same class.
Each extra stage fixes one specific failure, and it pays off exactly when that failure is present. The LLM synonym check adds +5.4, +4.9, +2.4 and +0.3 points on CIFAR-100, TinyImageNet, ImageNet-R and CUB-200, following how many different names users type per class (about 10, 10, 5 and 2). The CLIP check is what separates wrong labels from correct ones, and removing it costs 11.5 and 11.2 points on CIFAR-100 and TinyImageNet. The exact edits matter whenever a learned label later turns out wrong: after an audit they bring CIFAR-100 from 59.3% to 88.2%, identical to retraining. Finally, the Object-First check is the better choice when there are many samples per class (+2.7 and +5.9 points over Term-First on CIFAR-100 and TinyImageNet), and the Term-First check when there are few. The revised paper adds this comparison to Section 2 and Appendix N.
[1] Learning After Model Deployment. ECAI 2025.
[2] Continual Learning in Open-vocabulary Classification with Complementary Memory Systems. TMLR 2024.
[3] RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition. TIP 2026.