- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose DURA, a dual-stage unknown-aware representation adaptation framework for On-the-Fly Category Discovery (OCD). The method operates in two stages: first, it generates pseudo-unknown samples during offline training and uses maximum-similarity repulsion to separate them from known-category prototypes; second, it adapts both the encoder and prototype memory online as unlabeled query samples arrive, enabling the model to recognize known categories and discover new ones. To support systematic evaluation in underwater scenarios, the authors introduce AquaOCD343 and construct UOCDBench by combining it with four existing underwater datasets. Experiments on UOCDBench and standard OCD benchmarks demonstrate consistent improvements over recent state-of-the-art methods, particularly in novel-category discovery.
Critical strengths
- The paper addresses online category discovery in underwater environments, where known and previously unseen categories appear in a query stream. It introduces DURA, a two-stage framework that separates pseudo-unknown samples from known prototypes during offline training and updates the encoder and prototype memory during online inference.
- The DURA framework is clearly structured, with separate offline and online objectives. The offline stage combines angular-margin classification with pseudo-unknown separation, while the online stage uses entropy minimization, prototype alignment, inter-class separation, and confidence- and support-aware prototype refinement. The method is also evaluated with CLIP, DINO, and DINOv2 backbones.
- The paper evaluates DURA on UOCDBench, which includes AquaOCD343 and four existing underwater datasets, as well as five standard OCD datasets. It reports Old, New, and All accuracy under Greedy-Hungarian and Strict-Hungarian protocols. Ablation studies examine the main components, and category-count analysis provides additional information about the number of discovered categories.
Critical weaknesses
- As the number of categories discovered online increases, each query must be matched against more prototypes, which may increase the computational and memory costs of the online adaptation stage. The paper currently lacks an analysis of inference latency, throughput, and memory usage under different numbers of prototypes. Relevant experiments are needed to evaluate the feasibility of the method in long-term real-time operation.
- DURA decides whether to create a new prototype based on the similarity between a sample and the existing prototypes. When samples from the same class have large visual differences, some of them may be incorrectly treated as new categories, causing the same real class to be split into multiple prototypes. Why are strategies such as prototype merging not considered to mitigate this issue?
- During the Online Inference stage, the model uses its own predictions for representation adaptation and prototype updates. Therefore, incorrect sample assignments may affect the prototypes and further influence subsequent predictions. Reporting the proportion of incorrect assignments and comparing the similarity between the prototypes and the samples of the corresponding classes before and after the updates would provide a clearer view of the impact of online updating.
Questions for authors
Please see the weaknesses.
5 comments
Response to Weakness 2
We appreciate the reviewer’s suggestion to investigate prototype merging. Our additional experiments show that merging reduces prototype counts but decreases All and New ACC on both datasets under both evaluation protocols. Since clustering ACC is the primary evaluation metric in OCD, these results support retaining DURA’s original online discovery and refinement procedure over the evaluated merging strategy. Specifically, using DURA-DINOv2, merging reduces final prototype counts by 36.14% on AquaOV255 and 45.06% on AquaOCD343, while lowering Strict–Hungarian New ACC by 3.30 and 6.44 percentage points, respectively.
Merging strategy. Only newly discovered prototypes can merge; all initial known-category prototypes are protected. We set tau_merge = tau − 0.05, where tau is the original discovery threshold. Every 16 query batches, we merge the pair of new prototypes with the highest cosine similarity above this threshold, update the affected similarities, and repeat until no qualifying pair remains.
The merged prototype is calculated as:
p_merge = normalize(n_a p_a + n_b p_b),
where n_a and n_b are cumulative query-assignment counts. We sum these counts, retain the earlier stable prototype ID, and record an alias for the removed ID. The threshold offset and merging interval were fixed before the experiment and shared across both datasets. Neither query ground-truth labels nor target category counts are used to determine merges.
Table Response2a. Discovery accuracy with and without prototype merging using DURA-DINOv2. ACC values are percentages, reported as mean ± sample standard deviation across five query permutations.
| Dataset | Variant | Protocol | All ACC | Old ACC | New ACC |
|---|---|---|---|---|---|
| AquaOV255 | Original DURA | Greedy | 75.70 ± 1.38 | 96.64 ± 0.07 | 63.5 ± 2.15 |
| AquaOV255 | With merging | Greedy | 74.07 ± 1.77 | 96.71 ± 0.04 | 60.94 ± 2.78 |
| AquaOV255 | Original DURA | Strict | 71.42 ± 1.29 | 89.19 ± 0.71 | 61.11 ± 2.01 |
| AquaOV255 | With merging | Strict | 69.32 ± 1.73 | 89.14 ± 1.20 | 57.81 ± 2.65 |
| AquaOCD343 | Original DURA | Greedy | 79.41 ± 0.23 | 95.31 ± 0.03 | 55.54 ± 0.60 |
| AquaOCD343 | With merging | Greedy | 78.42 ± 0.38 | 95.80 ± 0.02 | 52.32 ± 0.94 |
| AquaOCD343 | Original DURA | Strict | 70.08 ± 0.33 | 85.23 ± 0.07 | 47.35 ± 0.79 |
| AquaOCD343 | With merging | Strict | 68.88 ± 0.46 | 87.50 ± 3.86 | 40.91 ± 5.82 |
Table Response2b. Final prototype counts with and without merging. Results are mean ± sample standard deviation; reductions are calculated from the mean counts.
| Dataset | Original DURA | With merging | Reduction |
|---|---|---|---|
| AquaOV255 | 768.60 ± 18.02 | 490.80 ± 16.68 | 36.14% |
| AquaOCD343 | 710.20 ± 4.38 | 390.20 ± 5.81 | 45.06% |
These comparisons clarify the trade-off between reducing prototype counts and preserving category discrimination. On AquaOCD343, merging also increases the standard deviation of Strict–Hungarian New ACC from 0.79 to 5.82 percentage points, indicating greater variability across query orders under the evaluated merging strategy.
DURA addresses fragmentation through offline unknown separation (Section 3.3, Eq. (5)), online category alignment, and confidence- and support-weighted prototype refinement (Section 3.4, Eqs. (8) and (12)). These mechanisms are designed to improve representation consistency and limit unreliable prototype drift, helping visually diverse samples from the same category match existing prototypes.
Response to Weakness 3
We sincerely appreciate the reviewer’s valuable suggestion. Our additional experiments show that online updates improve alignment for correctly assigned queries, supported prototypes become more class-consistent, and early feedback errors reduce All/New ACC by at most 0.02/0.03 percentage points under the tested settings. Tables Response3a–3c use DURA-DINOv2. Results are mean ± sample standard deviation over five query permutations; ground truth is used only for offline auditing.
Assignment errors. We examine queries matched to existing prototypes. Initial prototypes retain known-class identities; created prototypes receive their creating query’s true class for auditing. An assignment is wrong when query and prototype identities differ. These identities stay fixed for this audit; Table Response3c separately tracks changing dominant classes. These rates differ from Hungarian ACC.
Table Response3a. Wrong-class assignment rates among matched queries (%).
| Dataset | All matched | Known-class matched | Novel-class matched |
|---|---|---|---|
| AquaOV255 | 12.15 ± 0.43 | 1.03 ± 0.04 | 18.91 ± 0.71 |
| AquaOCD343 | 8.93 ± 0.36 | 1.37 ± 0.02 | 20.51 ± 0.93 |
The All rate is sample-weighted within each run. Known-class error rates are low; novel-class queries are more challenging.
Update effects. Before updating, we select the most similar available true-class prototype and keep its ID fixed for comparison after encoder adaptation and prototype refinement. Queries without a true-class prototype are excluded.
Table Response3b. Query–true-class-prototype similarity before/after updates.
| Dataset | Assignment | Before | After |
|---|---|---|---|
| AquaOV255 | Correct | 0.72353 ± 0.00226 | 0.72414 ± 0.00226 |
| AquaOV255 | Wrong | 0.41569 ± 0.00768 | 0.41571 ± 0.00769 |
| AquaOCD343 | Correct | 0.78789 ± 0.00097 | 0.78803 ± 0.00097 |
| AquaOCD343 | Wrong | 0.32420 ± 0.00637 | 0.32420 ± 0.00638 |
Correctly assigned queries become more aligned with their true-class prototypes, while incorrectly assigned queries show little change in true-class similarity; their similarity to the assigned wrong prototype increases by only 0.000510 ± 0.000028 and 0.000242 ± 0.000014, respectively, indicating small immediate effects whose downstream consequences we examine below.
Prototype consistency. For created prototypes receiving ≥20 assignments, consistency is the dominant true-class proportion within the first or last 10 assignments. The dominant class may change between windows.
Table Response3c. Prototype consistency over time.
| Dataset | Prototypes/run | First 10 (%) | Last 10 (%) | Dominant class changed and consistency improved (%) |
|---|---|---|---|---|
| AquaOV255 | 212.4 ± 12.7 | 57.82 ± 1.35 | 82.16 ± 1.08 | 31.74 ± 2.12 |
| AquaOCD343 | 178.6 ± 8.3 | 53.67 ± 1.09 | 79.03 ± 1.24 | 27.89 ± 1.85 |
Consistency rises by 24.34/25.36 percentage points, including prototypes that become more consistent while changing their dominant class.
Early-error perturbations. Using DURA-DINO (DINOv1), we randomly replace feedback IDs with different existing prototype IDs for 5%, 10%, or 20% of queries in the first 10% of each stream. Only adaptation/refinement IDs are changed; recorded predictions remain intact, while subsequent predictions use the updated model and memory. Five paired permutations share checkpoints, initial states, and query orders with the 0% baseline. ACC covers the unperturbed last 90%; final counts cover the full stream.
Table Response3d. Early-feedback perturbation results. ACC (%) is shown as Greedy / Strict; prototype counts are shared across protocols.
| Dataset | Rate | All ACC (G/S) | Old ACC (G/S) | New ACC (G/S) | Final prototypes |
|---|---|---|---|---|---|
| AquaOV255 | 0% | 60.90±0.67 / 56.20±0.68 | 93.11±0.19 / 87.01±1.33 | 42.15±1.03 / 38.27±0.51 | 1230.20±8.93 |
| AquaOV255 | 5% | 60.89±0.68 / 56.19±0.69 | 93.12±0.18 / 87.02±1.34 | 42.13±1.04 / 38.25±0.51 | 1230.40±8.96 |
| AquaOV255 | 10% | 60.88±0.65 / 56.18±0.66 | 93.11±0.19 / 87.01±1.33 | 42.12±0.99 / 38.24±0.47 | 1230.20±9.07 |
| AquaOV255 | 20% | 60.96±0.59 / 56.26±0.65 | 93.11±0.19 / 87.01±1.33 | 42.25±0.92 / 38.36±0.33 | 1230.80±9.31 |
| AquaOCD343 | 0% | 75.03±0.32 / 65.12±0.26 | 94.64±0.05 / 84.77±0.16 | 45.55±0.80 / 35.58±0.79 | 705.20±7.26 |
| AquaOCD343 | 5% | 75.03±0.32 / 65.12±0.26 | 94.64±0.05 / 84.77±0.16 | 45.55±0.80 / 35.57±0.79 | 705.20±6.87 |
| AquaOCD343 | 10% | 75.07±0.35 / 65.16±0.30 | 94.64±0.05 / 84.77±0.16 | 45.65±0.91 / 35.68±0.91 | 704.80±7.50 |
| AquaOCD343 | 20% | 75.07±0.35 / 65.16±0.29 | 94.64±0.04 / 84.77±0.15 | 45.65±0.89 / 35.67±0.90 | 704.80±7.50 |
Maximum mean All/New ACC decreases are 0.02/0.03 points on AquaOV255 and 0.00/0.01 points on AquaOCD343; mean final counts change by at most 0.60/0.40, respectively.
Response to Weakness 1
We sincerely appreciate the reviewer’s constructive comments on computational scalability, category fragmentation, and erroneous online feedback. We have completed the experiments reported below. After completing and analyzing the remaining additional experiments requested by every reviewer, we will incorporate the results into the revised manuscript’s appendix and clearly highlight all revisions in blue.
Response to Weakness 1
We appreciate the reviewer’s suggestion to analyze computational costs as the prototype memory grows. Our additional measurements show that DURA maintains efficient online processing with limited memory growth across the evaluated complete query streams. At the larger observed prototype counts, processing time remains approximately 1.1 ms per image, with throughput around 900 images/s.
We evaluate DURA-DINOv2 on AquaOV255 and AquaOCD343 using an NVIDIA RTX 5090 and a query batch size of 64, with five random query permutations per dataset. Each run starts from the same offline checkpoint and initial state. Prototype counts grow naturally under the original discovery and adaptation procedures.
The measured processing time includes input transfer, feature extraction, sequential prototype matching, new-prototype creation, online encoder adaptation, and prototype refinement. CUDA synchronization is performed at the timing boundaries. Data loading and final Hungarian evaluation are excluded.
Table Response1. Online processing costs at different naturally occurring prototype counts. Results are mean ± sample standard deviation across runs.
The amortized processing time is calculated by dividing the total batch processing time by the number of processed queries. Peak memory is the maximum PyTorch allocated GPU memory within each interval, calculated per run and then summarized across runs. Four AquaOV255 runs reach the highest interval.
From the lowest to the highest prototype-count intervals, peak GPU memory increases by 3.58 MiB on AquaOV255 and 2.73 MiB on AquaOCD343. Average persistent prototype storage in the highest intervals is 2.25 MiB and 1.62 MiB, respectively. Together with the processing-time measurements, these results strengthen the evidence for DURA’s computational feasibility within the evaluated prototype ranges.