- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TLM-SVDD, an extension of DLM-SVDD that replaces its primal surrogate encoder loss with a dual-based objective. Using Danskin's theorem, the authors show that the gradient of this objective matches that of the optimal primal value function without differentiating through the Frank-Wolfe solver. They further provide a support-vector contrast interpretation, establish boundedness, and analyze convergence under idealized conditions. The method also replaces the CNN backbone with a pretrained ViT-B/16. Experiments are conducted on Fashion-MNIST, CIFAR-10, CIFAR-100, CIFAR-10-LT, CIFAR-100-LT, and ImageNet-LT, using one-vs-rest anomaly detection protocols. TLM-SVDD reports strong performance compared with existing methods, and ablation studies examine the effects of backbone architecture, encoder optimization, anomaly exposure, and fine-tuning.
Critical strengths
-
Principled coupling of representation learning and boundary fitting. The proposed encoder update provides a mathematically motivated alternative to the primal surrogate used in DLM-SVDD. Under the stated assumptions, Proposition 1 derives the encoder gradient of the optimal primal value function without differentiating through the Frank-Wolfe solver. Proposition 2 provides an interpretable support-vector-weighted contrast formulation, and the objective is shown to be bounded. This formulation connects boundary fitting and representation learning through a common primal-dual optimal-value relationship, addressing an important limitation of the predecessor.
-
Strong reported performance and efforts to disentangle architectural and optimization effects. The proposed framework reports strong performance across standard anomaly detection and long-tailed benchmarks. The authors also make efforts to distinguish the effects of backbone architecture and encoder optimization. Table 3 compares fixed and fine-tuned representations across ResNet-50 and ViT-B/16 backbones, while Table 4 provides a matched ViT-based primal-versus-dual comparison with multiple random seeds and statistical testing. Furthermore, Table 5 reports cases where fine-tuning decreases performance, providing useful information about the limitations of the proposed method.
Critical weaknesses
-
Insufficient isolation and validation of the proposed dual encoder update. The main contribution is replacing DLM-SVDD’s primal surrogate with a dual-based encoder objective, but evidence isolating its effect from the backbone change remains limited. Table 3 compares ResNet-50 + primal against ViT-B/16 + dual, so it does not support the claimed consistent superiority of the dual update across three datasets independently of architecture. Table 4 provides a matched ViT-based comparison, but only on CIFAR-10 with 10% anomaly exposure. Moreover, at 10% exposure, the fixed ViT already achieves AUROCs of 0.995 and 0.989 on CIFAR-10 and CIFAR-100, respectively, exceeding all baseline results in Table 1 on both datasets. Dual fine-tuning adds 0.002 and 0.003, respectively, although the gain is larger on Fashion-MNIST (0.944 to 0.989). The primal ViT result in Table 4 (0.982 on CIFAR-10) is below the fixed ViT result in Table 3, suggesting that part of the dual update’s advantage may reflect avoiding degradation of pretrained features. This interpretation requires verification under identical settings. Table 5 also shows that dual fine-tuning reduces performance on CIFAR-100-LT at all three imbalance ratios, despite improvements on CIFAR-10-LT and ImageNet-LT. Broader comparisons of fixed, primal-updated, and dual-updated models using the same backbone are therefore needed to establish the update rule’s contribution across datasets and long-tailed settings.
-
Unclear comparability of the long-tailed evaluation and limited validation of the motivating setting. Table 2 compares conventional multi-class long-tailed classifiers with one-vs-rest SVDD-based detectors. However, the paper does not clearly explain how balanced accuracy is computed and aggregated for the SVDD methods or whether all methods are evaluated under equivalent prediction protocols. If the SVDD results represent averages of binary one-vs-rest balanced accuracies, they would not be directly comparable to standard multi-class balanced accuracy. This ambiguity makes it difficult to assess the reported state-of-the-art improvements in long-tailed recognition. Additionally, the introduction motivates anomaly detection under test-time anomalies that may differ from those observed during training. However, the experiments sample labeled training anomalies from the same set of remaining classes used to define test anomalies. Therefore, generalization to previously unseen anomaly classes is not explicitly evaluated.
Questions for authors
-
[Additional experiment] Controlled evaluation of the dual update. Could the authors extend the matched ViT-B/16 comparison in Table 4 to other datasets, particularly CIFAR-100 and representative long-tailed settings, including fixed, primal-updated, and dual-updated ViT models under identical settings? Results with ResNet-50 using the dual update would further separate the contribution of the update rule from that of the backbone. This would help determine whether the proposed dual update consistently improves representation learning rather than primarily avoiding degradation caused by the primal surrogate.
-
[Clarification] Long-tailed evaluation protocol. How exactly is balanced accuracy computed for the SVDD-based methods in Table 2? Are the results obtained by averaging binary balanced accuracy across one-vs-rest tasks, or are the scores combined into multi-class predictions? How are these results made directly comparable to conventional multi-class long-tailed classification baselines?
-
[Additional experiment] Unseen anomaly classes. The introduction states that labeled anomalies are typically not fully representative of those encountered at test time, and that a detector should avoid becoming a closed-set classifier that only separates the anomalies observed during training. Could the authors evaluate a setting where labeled training anomalies come from a subset of classes and test anomalies from held-out classes? This would directly test the motivation of the paper, and positive results in this setting would provide stronger evidence for generalization to unseen anomaly classes.
3 comments
Thank you for the careful review, and sorry that our response comes this late. The experiments you asked for took most of the discussion period. The revised paper with these changes will be uploaded shortly.
Q1. Isolating the dual update. We now compare three versions of the same ViT-B/16: fixed (pretrained encoder, only the boundary is fitted), primal (the DLM-SVDD update), and dual (our update). We do this on Fashion-MNIST, CIFAR-10, and CIFAR-100 at 10%, 50%, and 75% anomaly exposure. All three use the same initialization, data splits, anomaly samples, training budget, hyperparameters, and seeds. Test AUROC at 10% exposure:
| Fixed | Primal | Dual | |
|---|---|---|---|
| Fashion-MNIST | 0.98268 | 0.98528 | 0.98854 |
| CIFAR-10 | 0.99534 | 0.99656 | 0.99693 |
| CIFAR-100 | 0.98869 | 0.99072 | 0.99162 |
- Does the dual update only avoid damage done by the primal one? No. You noted that the primal ViT in the submitted Table 4 (0.982) was below the fixed ViT. We have updated these values based on the full matched experiments, and with them the primal update is above the fixed encoder in all nine settings, and the dual update is above the primal one in all nine. The table has been updated with these values and more details.
- Effect size. We agree that the fixed ViT is already strong. Because AUROC is close to 1 here, we also report metrics that separate the methods better. FPR95 is the fraction of anomalies still accepted when 95% of normal samples are accepted (lower is better). Balanced accuracy is the average of the true-positive rate (TPR, normal samples accepted) and the true-negative rate (TNR, anomalies rejected). AUPR-In is the area under the precision–recall curve with normal samples as the positive class. The dual update is best in all nine settings for AUROC, FPR95, and balanced accuracy, and in 7 of 9 for AUPR-In. On CIFAR-10 it adds only 0.04 AUROC points over the primal update, but it is better in all five seeds (paired t-test over seeds: , for AUROC; , for FPR95), and FPR95 drops by 19.2% compared with the primal update and by 43.6% compared with the fixed encoder. On CIFAR-100 the dual update has a higher AUROC than the primal one in 69 of the 100 one-vs-rest tasks (Wilcoxon signed-rank test, ).
- Fashion-MNIST. Based on the full experiments, with the single configuration used for all results, the fixed ViT obtains 0.983 at 10% and 0.984 at 50% and 75% (submitted: 0.944). So training the encoder with the dual update adds 0.59 AUROC points there, and 0.33 of them come from the dual update itself.
- Long-tailed settings and ResNet-50. We could not finish the matched long-tailed runs or a ResNet-50 run with the dual update in time. The drop on CIFAR-100-LT after fine-tuning is stated in the paper, and we do not claim that training the encoder always helps. A dual update on ResNet-50 would need frozen batch-norm statistics (the features that fit the boundary must be the ones that are differentiated), which changes two things at once, so we compare the update rules on the same ViT instead.
Q2. Long-tailed protocol. For the SVDD-based methods, we compute balanced accuracy separately for each one-vs-rest task, at the learned radius, and then average over the classes. The scores are not combined into one multi-class prediction. The revised paper defines both protocols in Sec. 4.1, separates the two groups in Table 2, and limits the claim to the one-vs-rest SVDD methods. Among those, TLM-SVDD improves over DLM-SVDD by 4.9 to 8.9 points on the long-tailed CIFAR sets and by 6.1 points on ImageNet-LT.
Q3. Unseen anomaly classes. For each CIFAR-10 normal class, we split the other nine classes into four seen and five held-out classes. Training, validation, and model selection use only the seen classes; the held-out classes appear only at test time. Five seeds (mean ± SD):
| AUROC | FPR95 | Balanced acc. | |
|---|---|---|---|
| Fixed | 0.984 ± 0.004 | 0.073 ± 0.017 | 0.828 ± 0.023 |
| Primal | 0.985 ± 0.005 | 0.061 ± 0.019 | 0.858 ± 0.031 |
| Dual | 0.987 ± 0.004 | 0.046 ± 0.015 | 0.862 ± 0.029 |
AUROC is about one point lower than in the standard setting, but the unseen classes are still rejected well (FPR95 of 4.6%). The order fixed < primal < dual holds, and FPR95 is lower for the dual than for the primal update in every one of the five seeds (24.8% lower on average). The AUROC differences are within one standard deviation, so we do not claim an AUROC gain here. This evaluation is added to the paper as Appendix A.4, next to published CIFAR-10 results of the methods we cite.
Thank you again for these helpful questions.
Thank you for following up, and apologies for the silence. We understood that the discussion period ends on November 18 (AoE), so we thought we still had time, and we were using it to run the experiments you requested to further strengthen the paper: the matched fixed / primal / dual comparison on the same ViT-B/16 across all three datasets and exposure ratios, and the evaluation on held-out anomaly classes. Our full response is now posted under your review, and the revised paper will be uploaded as soon as possible. We would be grateful if you could take them into account in your final assessment.
As there was no further author discussion, I will maintain my original rating based on the submitted response and review.