- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes Energy Ring for test-time discovery, using a lightweight adapter to organize known and simulated unknown features. Building on DAA, it combines norm-weighted prediction with perturbation-based stability measures to guide online adaptation and memory renewal, aiming to improve novel-class discovery while preserving known-class recognition.
Critical strengths
The overall idea is appealing, particularly the combination of radial feature structure and stability-guided adaptation. The figures are well designed, and the paper is clearly written and easy to follow. Experiments across four datasets, component ablations, sensitivity analyses, and paired comparisons with DAA provide useful evidence about the method’s benefits and limitations.
Critical weaknesses
1.The contribution appears incremental. Appendix J acknowledges the DAA foundation, while radial constraints and consistency-based selection have precedents in Ring Loss, Objectosphere, and SENTRY. The existing ablations do not establish an advantage over simpler alternatives under matched budgets.
2.The simulated-unknown mechanism lacks targeted validation. Figure 3 shows norm separation without isolating the benefits of mixing and noise. Section 3.2 specifies Gaussian noise, but the submitted train_ERA generates mixed features without adding it.
3.Performance depends on insufficiently documented parameter selection. In Table 14, real-time TA+CA drops from 81.18 with manually selected radii to 71.93 with median calibration. Appendix K.5 mentions grid search without clearly specifying validation data or label access.
4.The submitted defaults diverge from the described method. Prediction weighting uses alpha=0 rather than Table 3’s 0.05; test perturbations imply approximately 84° changes despite the local-stability interpretation; and the default CIFAR warm-up includes held-out-class training images. These findings concern the submitted defaults, not necessarily the reported experiments. The authors need to provide the exact experimental configurations and code revision.
5.TA+CA alone does not establish better discovery quality. Table 11’s 70+30→70+∞ change raises TA+CA from 81.18 to 119.90, driven by CA increasing from 47.66 to 86.55, while TA slightly decreases and post-evaluation TA falls from 37.74 to 23.65. DAA shows a similar trend. Table 10’s 0.74–1.09-point real-time HCA gains provide positive counterevidence, but cluster counts, fragmentation statistics, and unknown-only matching accuracy are still needed to distinguish improved discovery from changes in clustering granularity.
Questions for authors
-
Which exact code revision and configurations produced the reported results? The submitted defaults disable prediction weighting (
alpha=0), apply approximately 84° test perturbations, and include held-out-class training images in CIFAR warm-up. Were these settings overridden, and was Gaussian noise actually added to simulated features? -
What validation data, label access, and selection criteria were used to choose the radii, weighting coefficient, and discovery threshold? Were DAA and Energy Ring given comparable tuning budgets, and does the improvement persist under selection using only known-class data?
-
How do the authors distinguish better discovery from increased fragmentation? In Table 11, unlimited capacity raises TA+CA from 81.18 to 119.90 while TA barely changes and post-evaluation TA declines. Beyond Table 10’s modest overall HCA gains, can the authors report actual cluster counts, per-class fragmentation, and unknown-only one-to-one matching accuracy?
8 comments
Direct evaluation of reassignment quality. We compare selection policies with the same rechecking budget and a fixed offline cluster-to-label mapping. A correct assignment places a sample in its correct novel-class queue, including moving to another queue or returning to its original correct queue after updating. An incorrect assignment places it in an incorrect queue, including correct-to-incorrect and incorrect-to-incorrect changes. Samples predicted as known are removed from novel queues and counted separately; this includes known samples previously misassigned to novel queues and subsequently recognized as known. All percentages use the total number of rechecking events as their denominator, so the three outcomes sum to 100%. Labels are used only for evaluation. DAA + STMR provides the random-rechecking control.
| Model | Selection | Correct assignment (%) | Incorrect assignment (%) | Removed as known (%) |
|---|---|---|---|---|
| DAA + STMR | Random | 35.08 | 30.20 | 34.72 |
| Ours w/o energy | Lowest stability | 43.59 | 25.51 | 30.90 |
| Ours | Random | 38.62 | 28.74 | 32.64 |
| Ours | Lowest stability | 56.34 | 21.39 | 22.27 |
Our full method achieves the highest correct-assignment rate and lowest incorrect-assignment rate. Within our method, stability-guided selection improves these rates over random selection by 17.72 and 7.35 percentage points, respectively. Even without energy, our method produces more correct and fewer incorrect assignments than DAA + STMR (43.59% vs. 35.08%; 25.51% vs. 30.20%). These results support the contribution of perturbation robustness and stability-guided correction beyond norm preservation.
We will report warm-up/online update counts, memory/replay limits, and tuning-trial counts for the compared variants with their exact configurations. The common implementation controls implementation differences; the tuning-budget and selection protocol are addressed separately in W3/Q2 below.
The protocol distinction is already stated in Appendix A/Table 5, reproduced below (K: known classes; U: unknown classes). TTD requires both discovery and evaluation in real time, making DAA the direct baseline for the comparisons above.
| Task | Train | Test | Discovery | Evaluation |
|---|---|---|---|---|
| NCD | K | U | Post-hoc | Post-hoc |
| GCD | K | K + U | Post-hoc | Post-hoc |
| CGCD | K | K + U₁ + U₂ + … | Post-hoc | Post-hoc |
| OCD | K | K + U | Real-time | Post-hoc |
| TTA | K | Shifted K | N/A | Real-time |
| TTD | K | K + U | Real-time | Real-time |
W2/W4/Q1. Exact implementation, synthetic noise, and perturbations.
Thank you for distinguishing the submitted defaults from the configurations used for the reported experiments. We will provide the complete, cleaned code with a pinned experiment-matched revision and per-table configuration files, covering the effective α, known-class filtering, Gaussian-noise scale, and perturbation settings. In particular, the α=0 value is only a placeholder in the code constructor and is definitely not the experimental default; it should not be interpreted as the setting used for the reported results. We confirm that the experimental project uses additive Gaussian noise after MixUp and known-class images only for warm-up, consistent with Sec. 3.2 and Appendix B. The submitted package was a simplified, draft version as a part of Supplementary Material and the complete release will make each reported result traceable to its executed configuration.
For the simulation mechanism, MixUp places features between known representations, while Gaussian noise broadens these neighborhoods. Our simulated-unknown construction follows DAA’s feature-space simulation strategy, using MixUp with Gaussian noise (Sec. 3.2; Appendix I/Fig. 12). Its purpose is to expose the adapter to unknown-like feature patterns and prepare it for subsequent discovery. We additionally evaluated alternative simulation strategies, which also provided effective initialization, indicating that our framework is not restricted to this particular construction. Our methodological focus is the energy structure and stability-guided adaptation. Actual novel-class prototypes are initialized from arriving test samples and refined during discovery.
To clarify the role of the simulated-unknown construction, we add an ablation that isolates the effects of MixUp and Gaussian noise.
| Simulate strategy | Synthetic feature | Simulated norm before warm-up | True K/U norm gap after warm-up | Real Time TA+CA | Post TA+CA | KF |
|---|---|---|---|---|---|---|
| None | Not used | N/A | 5.83 | 62.59 | 55.83 | 5.37 |
| Noise only | r + n | 64.07 | 6.30 | 70.35 | 69.39 | 4.81 |
| MixUp only | λr + (1−λ)r′ | 56.51 | 8.55 | 78.67 | 72.75 | 4.10 |
| MixUp + noise | λr + (1−λ)r′ + n | 52.64 | 10.18 | 81.18 | 76.66 | 3.65 |
These results show that the DAA-style MixUp + Gaussian-noise construction is effective because MixUp places synthetic features between known representations while Gaussian noise broadens these neighborhoods, exposing the adapter to unknown-like patterns and preparing it for subsequent discovery. At the same time, alternative simulation strategies also provide effective initialization, indicating that our framework is not restricted to this particular construction. Our methodological focus remains the energy structure and stability-guided adaptation and actual novel-class prototypes are initialized from arriving test samples and refined during discovery, rather than being fixed by the simulation strategy.The known-only training protocol is already specified in Appendix B/Table 6. The full split table is reproduced below; test labels are withheld from the online learner and used for evaluation.
| Dataset | Split | Labelled | Known classes | Unknown classes | Samples |
|---|---|---|---|---|---|
| CIFAR100D | Train | Yes | 70 | 0 | 35,000 |
| Test | No | 70 | 30 | 10,000 | |
| CUB200D | Train | Yes | 140 | 0 | 4,195 |
| Test | No | 140 | 60 | 5,794 | |
| Tiny-ImageNetD | Train | Yes | 140 | 0 | 70,000 |
| Test | No | 140 | 60 | 10,000 | |
| AircraftD | Train | Yes | 70 | 0 | 2,331 |
| Test | No | 70 | 30 | 3,333 |
For example, CIFAR100D warm-up uses 35,000 images from 70 known classes, the 30 held-out classes are excluded from warm-up and first encountered by the model at test time. We will supply the filtering configuration and sample counts. Table 3 reports α=0.05 as the best listed weighting setting; the release will explicitly distinguish the package's α=0 default from the effective value used for each result, rather than treating a default as the experimental configuration.
W1. Contribution beyond DAA and simpler alternatives.
Relationship to prior work. As acknowledged in Appendix J, DAA provides the foundation for our TTD framework and a directly relevant baseline. We build on its implementation and compare in a common environment to reduce implementation differences. We agree that radial constraints and consistency-based selection have precedents: Ring Loss (Zheng et al., CVPR 2018) softly constrains feature norms toward a common radius; Objectosphere (Dhamija et al., NeurIPS 2018) separates known/unknown magnitudes for open-set rejection; SENTRY (Prabhu et al., ICCV 2021) uses prediction consistency to selectively minimize or maximize entropy during domain adaptation. Our contribution concerns preserving and using radial structure during online novel-class discovery, together with perturbation-based warm-up and stability-guided reassignment of earlier samples (Sec. 3.2–3.4).
Direct comparisons with simpler alternatives. We additionally evaluate DAA with the following norm-margin regularizer during both warm-up and test-time training:
=\mathbb{E}_{(z_k,z_u)} \left[\max\left(0,\;m-\bigl(\lVert z_k\rVert_2-\lVert z_u\rVert_2\bigr)\right)^2\right],\qquad m>0.$$ Norms are computed before feature normalization. During warm-up, $z_k$ and $z_u$ are known and simulated-unknown features; during testing, they follow DAA's known/novel pseudo-label and replay rules. We use $m=20$ in both stages. The loss penalizes gaps below $m$ without fixing absolute radii, and requires no unknown ground-truth labels for training. We also evaluate our method with energy removed: radial training losses and energy-weighted prediction (α=0) are disabled, while perturbation warm-up and stability-guided rectification are retained. | Method | Real Time TA+CA | Post TA+CA | KF | |---|---|---|---| | DAA | 78.42 | 73.52 | 3.54 | | DAA + norm-margin regularization | 76.23 | 73.58 | 3.97 | | Ours w/o energy | 79.82 | 75.63 | 3.50 | | Energy Ring | 81.18 | 76.66 | 3.65 | Even without energy, our method improves real-time and post TA+CA over DAA by **1.40 and 2.11 points**, respectively, supporting the contribution of perturbation warm-up and stability-guided rectification beyond norm preservation. Energy further adds **1.36 and 1.03 points**. Adding a generic norm margin to DAA does not reproduce this benefit: it imposes a relative gap on an angular framework, whereas Energy Ring anchors radial levels according to the observed norm ordering and uses them in both prediction and adaptation. **Why perturbation robustness helps without energy.** During warm-up, perturbed views of the same simulated unknown form positives, while views of different features form negatives (**Sec. 3.2, Eq. (4)**), encouraging consistency and separation. During testing, assignment agreement under perturbations defines stability (**Eq. (9)**). Reliable unknown samples support adaptation, while low-stability queued samples are prioritized for reclassification. They are reinserted according to their revised labels, and the affected prototypes are recomputed as queue means. Thus, earlier assignments can be corrected even when radial information is disabled. The existing **Table 4/Appendix D.4** also shows that adding rectification to T+W raises real-time TA+CA from **76.59 to 81.18** and post TA+CA from **71.04 to 76.66**.