acceptodds
Under review as a conference paper at ICLR 2027

Understanding Deep One-Class Classification: A Training-Dynamics Viewpoint

Abstract

Deep one-class classification (DOCC) maps normal samples toward a fixed center, yet at test time it must distinguish unseen normal and anomalous inputs. How DOCC training shapes representations of inputs that never appear in the training objective remains poorly understood. We study the training dynamics of DOCC under gradient descent and derive a finite-time first-order description of the representation change of an unseen input. We show that its movement is determined by cumulative normal-sample updates transferred through gradient coupling, while the representations of normal training samples are progressively driven toward the center. This analysis identifies two key factors governing the behavior of unseen inputs: their initial separation from the center and the training-induced displacement. These factors explain representative success and failure cases for both unseen normal and anomalous inputs and yield a sufficient condition for an anomaly to remain detectable after training. We further extend the training-dynamics view to anomaly supervision and explain how labeled anomalies induce additional displacements that can correct anomalies missed by one-class training. Experiments support the predicted representation dynamics, anomaly scores, failure modes, and coupling geometry under Gaussian initialization.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Anomaly detection, Deep one-class classification, training dynamics, gradient coupling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4TggOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how deep one-class classification (DOCC) training affects unseen normal and anomalous inputs. It decomposes representation changes into initialization, accumulated updates transferred through an initial gradient-coupling kernel, and a bounded remainder. Under stability and kernel nondegeneracy assumptions, it establishes contraction of normal training representations and a sufficient condition for anomaly detectability. It further characterizes the coupling under Gaussian initialization and extends the displacement analysis to Deep SAD. MNIST experiments examine these approximations and their implications for detection.

Critical strengths

Clear analytical framework. The trajectory decomposition explicitly separates errors from local linearization, Jacobian drift, and weight decay, providing a unified description of training effects on unseen inputs.

Transparent evaluation of approximation limits. The experiments examine how approximation quality varies with training time, width, and depth, including cases where it deteriorates. The limitations acknowledge unresolved displacement directions and differences between normal and anomalous responses.

Critical weaknesses

Limited experimental coverage. Experiments use only MNIST, with the main detection and anomaly-supervision results focusing on digit 3. Please report these results across all ten tasks and include CIFAR-10 or another data domain to assess whether the proposed explanations extend beyond this setting.

Initialization and training effects are insufficiently separated. Report initial detection performance and distance distributions, then quantify displacement and final miss rates across initial-distance groups. Also report the fraction of test anomalies satisfying the detection sufficient condition to demonstrate its practical informativeness.

Theoretical novelty remains unclear. DOCC can be viewed as squared-loss regression with a constant target (c), making its kernel-driven dynamics and training-residual contraction closely related to existing NTK analyses. Please identify which results require substantive new arguments rather than direct specialization, and clarify the technical advances in the finite-time error bounds.

Anomaly-specific insights remain limited. The same dynamics apply to normal and anomalous inputs, but the analysis does not establish when their coupling, displacement directions, or final scores differ systematically. A complete detection theory is unnecessary, but the paper should clarify what its framework adds to understanding normal–anomaly separation beyond generic training dynamics, ideally through controlled experiments.

Questions for authors

Please address the concerns raised in the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4Tgg (1/2)

W1. Limited experimental coverage. Experiments use only MNIST, with the main detection and anomaly-supervision results focusing on digit 3. Please report these results across all ten tasks and include CIFAR-10 or another data domain to assess whether the proposed explanations extend beyond this setting.

A: Thank you for the suggestion. As requested, we now report both the detection and the anomaly-supervision results on all ten MNIST tasks, and we repeat the same experiments on a second dataset, Fashion-MNIST (see Table 1 in Appendix G.3). Each of the ten one-vs-rest tasks is run separately, taking each class in turn as the normal class. The results are consistent with those reported in the original submission. The table below reports the averages over the ten tasks of each dataset at m=8192m=8192, where the DOCC columns correspond to detection and the Deep SAD columns to anomaly supervision. The results for the other widths are given in Table 1.

DatasetInit.DOCCFirst-orderDeep SADMissedRecovered
MNIST0.720.900.890.9843%78%
Fashion-MNIST0.810.870.860.9646%72%

The first four columns report the AUROC at initialization, of DOCC, of its first-order prediction and of Deep SAD. All quantities except the initialization AUROC are evaluated at the corresponding Figure 5 step. "Missed" is the fraction of test anomalies not detected by DOCC at a 5% false-alarm rate, and "recovered" is the fraction of those missed anomalies that are detected by Deep SAD.

(1) All ten MNIST tasks. DOCC improves on initialization for every digit and every width, and the first-order prediction closely tracks the DOCC result. Deep SAD recovers 78% of the anomalies missed by DOCC (51–97% across digits) while losing about 1% of the anomalies detected by DOCC.

(2) A second dataset. The same qualitative picture holds on Fashion-MNIST, where Deep SAD recovers 72% of the anomalies missed by DOCC.

W2. Initialization and training effects are insufficiently separated. Report initial detection performance and distance distributions, then quantify displacement and final miss rates across initial-distance groups. Also report the fraction of test anomalies satisfying the detection sufficient condition to demonstrate its practical informativeness.

A: Thank you for the suggestion. We have added such experiments to the revised manuscript (see Figure 8 and Tables 2 and 3 in Appendix G.3). They separate the effect of initialization from that of training. The numbers below are for m=8192m=8192 at the final training step TT.

(1) Initial detection performance and distance distributions. The distributions of the initial distance of held-out normals and anomalies are shown in Figure 8(a). At initialization they overlap, with an AUROC of 0.72 on MNIST and 0.81 on Fashion-MNIST. Training raises the AUROC to 0.90 and 0.87, respectively.

(2) Displacement and final miss rates across initial-distance groups. We group the test inputs into ten bins by initial distance. For compactness, the table below reports four representative bins on MNIST. The full ten-bin results are given in Table 2 of Appendix G.3.

BinContraction, normalsContraction, anomaliesFalse alarmsMissed
1 (nearest)47%27%0%93%
455%34%2%66%
756%36%7%39%
10 (farthest)55%38%36%7%

Contraction is the fraction of the initial distance lost during training. The miss rate decreases and the false-alarm rate increases with the initial distance, and in every bin normals contract more than anomalies (47–56% versus 27–38% over the ten bins, and 48–57% versus 35–41% on Fashion-MNIST).

(3) Fraction of test anomalies satisfying the sufficient condition. Since ETE_T contains unspecified absolute constants and is a worst-case bound, we cannot meaningfully instantiate it as a numerical certificate at the widths considered. We therefore report an empirical diagnostic in which ETE_T is replaced by the measured remainder ∥ρT(x)∥2\lVert\rho_T(x)\rVert_2, the quantity that ETE_T bounds. In this diagnostic, the condition of Theorem 3 is satisfied by 3% of the test anomalies on MNIST, and the direction-aware condition of the new Corollary 1 by 31%, which is about half of the detected anomalies (7% and 33% on Fashion-MNIST, see Table 3). The measured remainder also decreases with the width, faster than the rate in our bound (Table 5).

AuthorsOpenReview ↗

Response to Reviewer 4Tgg (2/2)

W3. Theoretical novelty remains unclear. DOCC can be viewed as squared-loss regression with a constant target c, making its kernel-driven dynamics and training-residual contraction closely related to existing NTK analyses. Please identify which results require substantive new arguments rather than direct specialization, and clarify the technical advances in the finite-time error bounds.

A: Thank you for your comment. First, we would like to emphasize that our main contribution is to explain the behavior of unseen inputs in DOCC: Theorem 1 shows how normal-training updates are transferred to an unseen input through gradient coupling, which explains the success and failure cases of detection. The same viewpoint is then extended to explain the effect of anomaly supervision.

Second, the finite-time result under the independent Gaussian initialization used in the paper, now established in Appendix E of the revision, is not a direct special case of existing analyses:

(1) Zhou et al. (2026a): their local derivative and linearization bounds rely on a cancellation that holds only for symmetric initialization, so they do not apply here.

(2) Zou et al. (2020): they work under independent initialization but provide perturbation ingredients rather than the local derivative and linearization bounds required here.

(3) Combining the two: this still does not directly yield our result, because the argument of Zhou et al. breaks exactly at the cancellation step. Our Lemma 9 replaces the role of this cancellation with a new backward-product stability bound, which yields the required bounds with the same R1/3m−1/6R^{1/3}m^{-1/6} dependence up to logarithmic factors, and hence the finite-time error bound under independent initialization.

Given these bounds, Theorem 2 follows standard NTK-type arguments and Theorem 3 is a direct consequence of Theorem 1.

W4. Anomaly-specific insights remain limited. The same dynamics apply to normal and anomalous inputs, but the analysis does not establish when their coupling, displacement directions, or final scores differ systematically. A complete detection theory is unnecessary, but the paper should clarify what its framework adds to understanding normal–anomaly separation beyond generic training dynamics, ideally through controlled experiments.

A: Thank you for this comment. Indeed, our framework provides a concrete mechanism for normal–anomaly separation: unseen inputs follow the same training dynamics, but they can respond differently because their coupling to the normal training samples differs. By Theorem 4, under Gaussian initialization, inputs that are more similar to the training normals receive stronger transfer of normal-training updates.

To examine this mechanism directly, we added controlled experiments on MNIST and Fashion-MNIST comparing held-out normals and anomalies at the same initial distance from the center (see Figure 9 and Table 4 in Appendix G.3). We find that anomalies are more weakly coupled to the training normals, undergo less center-directed displacement, and remain farther from the center after training. After controlling for initial distance, further conditioning on coupling strength substantially reduces the remaining separation in final distance (AUROC from 0.89 to 0.78 on MNIST and from 0.76 to 0.62 on Fashion-MNIST).

We have also clarified this point in the new Discussion section (Section 6): the contraction of unseen inputs is selective rather than uniform, and coupling to the training normals is a key source of this selectivity.

Thank you for the careful responses and additional experiments. I appreciate the authors' efforts. However, I still have some reservations about the practical explanatory value and broader applicability of the theoretical framework. Therefore, I will maintain my original score of 2 .

Thank you for reading our responses and for acknowledging the additional experiments. We would like to briefly address the two remaining reservations you mention.

Practical explanatory value. The framework makes statements about individual unseen inputs, and we tested their implications directly across all ten tasks of both datasets:

  1. It closely tracks the trained detector. At m=8192m=8192, the first-order description stays within 0.01 AUROC of DOCC up to a step between 78k and 200k (out of 200k) on every task (Appendix G.3.1).
  2. It identifies a mechanism behind normal–anomaly separation. At the same initial distance, held-out normals contract by 47–56% and anomalies by 27–38% on MNIST (48–57% versus 35–41% on Fashion-MNIST), consistent with their different coupling to the training normals (Table 2 and Figure 9).
  3. It gives a testable prediction under intervention. When all labeled anomalies are drawn from MNIST digit 7, Deep SAD recovers 98% of the missed 7s but only 47% of the other anomalies, compared with 70% and 79%, respectively, under a uniform labeled set (Table 7).

These findings also point to concrete directions for method design: they identify gradient coupling as a quantity that can be shaped, motivating selective contraction through representation design and coupling-aware selection or weighting of labeled anomalies, as discussed in Section 6.

Broader applicability. Theorems 1–3 are established under general local-regularity and coupling conditions (Theorems 5–7 in the appendix), rather than assumptions specific to ReLU networks or MNIST. The ReLU/Gaussian setting is used to verify these conditions and obtain an explicit kernel. Section 6 explicitly states the scope of the current guarantees and which extensions, including adaptive optimization and substantial feature learning, require separate analysis.

If a specific concern remains with these results or with the stated scope of the framework, we would be glad to clarify it. We would be grateful if you could take these points into account in your final assessment.

Reviewer_cdUdOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies deep one-class classification (DOCC) from a training-dynamics perspective. It derives a finite-time, first-order decomposition of an unseen input’s representation into its initial representation, a cumulative training-induced displacement, and an approximation remainder. The displacement is determined by normal-sample updates transferred through the gradient coupling between training samples and the unseen input. This framework yields a contraction result for normal training representations. Under Gaussian initialization, the authors further characterize the coupling through a deterministic deep ReLU kernel that depends on input similarity.The analysis is further extended to Deep SAD, where labeled anomalies introduce additional repulsive contributions that can correct some missed anomalies. Experiments examine the trajectory approximation, representation contraction, detection failure modes, and the effects of network width and depth on the coupling structure.

Critical strengths
  1. Understanding how normal-only training affects unseen anomalies is central to DOCC. The gradient-coupling formulation provides an interpretable mechanism linking training updates to changes in test representations. In particular, it explains how normal-data training can inadvertently pull anomalies toward the center and cause missed detections.
  2. The experimental results provide strong support for the theories, and the observed phenomena are consistent with the theoretical analysis, thus demonstrating the rationality of the theories. The evaluation investigates trajectory approximation, contraction, and kernel concentration instead of relying solely on final AUROC. It also reports cases where the approximation deteriorates, particularly later in training and for narrower networks.
Critical weaknesses

1.The paper merely explains why normal data contracts during the DOCC training process and why abnormal data is pushed away when anomalies are introduced during training. However, what insights can this theoretical explanation provide for subsequent research? Based on the theory presented in the paper, how can we design new methods to address these issues? For instance, in Figure 1, there are always some normal and abnormal data points outside the hypersphere initially. How can we ensure that normal data can contract inside while abnormal data remains outside? Additionally, for abnormal data points that are initially inside, is there no way to detect them? As that the theoretical explanation shows that the training process is a continuous contraction process. 2.The analysis considers bias-free, fully connected ReLU networks with a fixed random output layer and full-batch gradient descent, together with local linearization and conservative width requirements. The experiments likewise exclude autoencoder pretraining and mini-batch Adam. Applicability to other architectures, optimizers, and substantial feature-learning regimes remains unclear. 3.Theorem 3 follows from a triangle inequality and depends on displacement magnitude while discarding direction. It can therefore be uninformative when training moves an anomaly away from the center. 4.All experiments use MNIST, and most detection visualizations focus on digit 3 as the normal class. Evidence from more diverse datasets would strengthen the generality of the conclusions.

Questions for authors

In Section 4, you extended your theory to the case of anomaly supervision: the gradient coupling brought by labeled anomalies pushes the anomaly representations away from c. Can your theory be used to explain other anomaly-supervised AD methods beyond DOCC, such as BGAD[1]. [1] Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detection. In CVPR, 2023.

4 comments
AuthorsOpenReview ↗

Response to Reviewer cdUd (1/3)

W1. What insights can this theoretical explanation provide for subsequent research? Based on the theory presented in the paper, how can we design new methods to address these issues? For instance, in Figure 1, there are always some normal and abnormal data points outside the hypersphere initially. How can we ensure that normal data can contract inside while abnormal data remains outside? Additionally, for abnormal data points that are initially inside, is there no way to detect them? As that the theoretical explanation shows that the training process is a continuous contraction process.

A: Thank you for this insightful question. We have added a new Discussion section (Section 6) to the revised manuscript on the implications of our analysis for method design. In short, our analysis suggests a concrete principle, which we call selective contraction: training should pull normal-like inputs toward the center while preventing anomaly-like inputs from being unnecessarily pulled into the normal region. When anomaly information is available, this can be complemented by an explicit outward correction. Below we discuss the two cases you raise.

(1) How can outside normals move inside while outside anomalies remain outside? Our analysis shows that normal-training updates are transferred to an unseen input through its coupling to the training normals, which under Gaussian initialization depends on input-space similarity (Theorem 4). Hence, inputs that are more similar to the training normals receive stronger first-order transfer of these updates. Our new controlled experiments (see Table 2 and Figure 9 in Appendix G.3) confirm this: at the same initial distance from the center, held-out normals are more strongly coupled to the training normals and contract substantially more than anomalies (47–56% versus 27–38% of the initial distance on MNIST, and 48–57% versus 35–41% on Fashion-MNIST). In other words, both groups contract, and their separation arises from the difference in how strongly they contract. The relevant quantity to control is therefore not the overall amount of contraction, but its selectivity across unseen inputs. Since gradient coupling is a key source of this selectivity, and is determined at initialization in our first-order description, one possible direction is to shape the representation before DOCC training, for example through pretraining, with the goal of making unseen normals couple more strongly to the normal training data while anomalous inputs couple more weakly.

(2) Can anomalies that start inside the normal region still be detected? Under normal-only training, there is no explicit mechanism for pushing such anomalies outward. A possible solution is anomaly supervision. As shown in Section 4, labeled anomalies introduce an additional outward displacement: the anomaly term in Deep SAD pushes labeled anomalies away from the center, and this effect is transferred to unseen anomalies through the same coupling mechanism. In our new experiments (see Tables 6 and 7 in Appendix G.3), at the end of training Deep SAD with 50 labeled anomalies recovers 81% of the anomalies missed by DOCC on MNIST and 72% on Fashion-MNIST (m=8192m=8192). At a comparable coupling to the training normals, the recovered anomalies are more strongly coupled to the labeled ones (AUROC 0.85 on MNIST and 0.79 on Fashion-MNIST), consistent with our analysis showing that weak coupling limits the outward correction. As a direct test on MNIST, when all 50 labeled anomalies are drawn from digit 7, Deep SAD recovers 98% of the missed 7s but only 47% of the other anomalies, compared with 70% and 79%, respectively, under a uniform labeled set. The analysis therefore suggests making anomaly supervision coupling-aware, for example by selecting or weighting labeled anomalies to provide broader coverage in the relevant coupling geometry, so that the outward correction can be transferred more effectively to unseen anomalies.

We view developing and evaluating such coupling-aware strategies as an interesting direction for future work enabled by the present analysis.

AuthorsOpenReview ↗

Response to Reviewer cdUd (2/3)

W2. The analysis considers bias-free, fully connected ReLU networks with a fixed random output layer and full-batch gradient descent, together with local linearization and conservative width requirements. The experiments likewise exclude autoencoder pretraining and mini-batch Adam. Applicability to other architectures, optimizers, and substantial feature-learning regimes remains unclear.

A: Thank you for this suggestion. We have clarified the scope of our analysis in the revised manuscript (Section 6). We view this work as a first step toward a theoretical understanding of the training dynamics of deep anomaly detection, and leave extensions to more general settings, such as adaptive or mini-batch optimization and substantial feature-learning regimes, for future work. We would like to make three remarks.

(1) Theorems 1–3 hold in a more general setting. Although the paper presents these results for deep ReLU networks, their formal statements (Theorems 5–7 in Appendices B–D) do not use the bias-free ReLU architecture, the fixed random output layer, or any width requirement. They only require general local regularity and coupling conditions under gradient descent. The fully connected ReLU network with Gaussian initialization is used to verify the local-regularity condition (Proposition 1) and to obtain the explicit kernel in Theorem 4. Extending the results to another architecture therefore amounts to verifying the corresponding conditions for that architecture. Related local-stability and linearization properties have been established for other wide-network architectures, including convolutional networks and ResNets (e.g., Du et al., 2019; Allen-Zhu et al., 2019).

(2) Pretraining fits the same viewpoint. Pretraining changes the initialization, and hence the two quantities that our analysis identifies as governing the behavior of unseen inputs: the initial representation and the coupling. The first-order mechanism of Theorem 1 remains the same, provided that the local-regularity conditions hold at the pretrained initialization, while the explicit kernel of Theorem 4 no longer applies.

(3) The setting is chosen deliberately. Our goal is a provable finite-time description of how training moves unseen inputs, and the experiments are designed to closely match the analyzed setting, which is why autoencoder pretraining and mini-batch Adam are excluded. The bias-free architecture follows the recommendation of Ruff et al. (2018) for DOCC, which avoids trivial constant representations. Adaptive or mini-batch optimization would require controlling stochastic or adaptive update dynamics beyond the deterministic full-batch trajectory analyzed here.

W3. Theorem 3 follows from a triangle inequality and depends on displacement magnitude while discarding direction. It can therefore be uninformative when training moves an anomaly away from the center.

A: Thank you for this observation. We agree that Theorem 3 is a direction-free sufficient condition and can therefore be conservative. A direction-aware version follows directly from Theorem 1, and we have added it to the revised manuscript (Corollary 1, Section 3.3):

∥ϕW(0)(x)+uT(x)−c∥2−ET>τ⟹sT(x)>τ.\lVert\phi_{\mathbf{W}(0)}(x)+u_T(x)-\mathbf{c}\rVert_2-E_T>\sqrt{\tau}\quad\Longrightarrow\quad s_T(x)>\tau .

This condition retains the direction of the training-induced displacement uT(x)u_T(x). Theorem 3 is recovered from Corollary 1 by further lower-bounding the left-hand side using the triangle inequality. In particular, when training moves an anomaly away from the center, the direction-aware condition captures this effect, whereas the direction-free condition does not.

Our new experiments (see Table 3 in Appendix G.3) confirm that this distinction is substantial. As an empirical diagnostic, using the measured remainder in place of ETE_T, at m=8192m=8192 the direction-free condition is satisfied by only 3% of the test anomalies on MNIST, whereas the direction-aware condition is satisfied by 31%, which is about half of the detected anomalies (7% versus 33% on Fashion-MNIST). This shows that retaining the direction substantially increases the empirical coverage of the condition.

AuthorsOpenReview ↗

Response to Reviewer cdUd (3/3)

W4. All experiments use MNIST, and most detection visualizations focus on digit 3 as the normal class. Evidence from more diverse datasets would strengthen the generality of the conclusions.

A: Thank you for the suggestion. We have extended the experiments in both directions (see Table 1 in Appendix G.3). We ran the experiments separately for all ten one-vs-rest tasks on both MNIST and Fashion-MNIST, taking each class in turn as the normal class. The results are consistent with those reported in the original submission. The table below reports the averages over the ten tasks of each dataset at m=8192m=8192. The results for the other widths are given in Table 1.

DatasetInit.DOCCFirst-orderDeep SADMissedRecovered
MNIST0.720.900.890.9843%78%
Fashion-MNIST0.810.870.860.9646%72%

The first four columns report the AUROC at initialization, of DOCC, of its first-order prediction and of Deep SAD. All quantities except the initialization AUROC are evaluated at the corresponding Figure 5 step. "Missed" is the fraction of test anomalies not detected by DOCC at a 5% false-alarm rate, and "recovered" is the fraction of those missed anomalies that are detected by Deep SAD.

(1) All ten MNIST tasks. DOCC improves on initialization for every digit and every width, and the first-order prediction closely tracks the DOCC result. Deep SAD recovers 78% of the anomalies missed by DOCC (51–97% across digits) while losing about 1% of the anomalies detected by DOCC.

(2) A second dataset. The same qualitative picture holds on Fashion-MNIST, where Deep SAD recovers 72% of the anomalies missed by DOCC.

Q1. In Section 4, you extended your theory to the case of anomaly supervision: the gradient coupling brought by labeled anomalies pushes the anomaly representations away from c. Can your theory be used to explain other anomaly-supervised AD methods beyond DOCC, such as BGAD [1]? [1] Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detection. In CVPR, 2023.

A: Thank you for this interesting question. We believe that the gradient-coupling viewpoint provides a natural way to interpret anomaly-supervised methods beyond DOCC/Deep SAD, although the specific center-based guarantees in Section 4 do not directly apply to BGAD. We have added a discussion of possible extensions of this viewpoint to BGAD and related boundary- or margin-based methods in the revised manuscript (Section 6).

Specifically, under a local first-order approximation, the update generated by any training sample is transferred to an unseen input through the corresponding gradient coupling. In Deep SAD, the anomaly loss makes this contribution explicitly repulsive from the center, which leads to the geometric condition derived in Section 4. BGAD uses a different geometry based on the log-likelihood under a normalizing flow. Its boundary-guided semi-push-pull loss pulls normal features whose log-likelihoods fall below the normal boundary toward the normal region, while pushing labeled anomalies whose log-likelihoods lie above the abnormal boundary to the anomalous side of the margin. Under the same viewpoint, the normal and labeled anomalous samples in BGAD generate updates directed relative to the boundary rather than to a center, and the effect of these updates on an unseen input is again determined by its coupling to the corresponding training samples.

This viewpoint also provides a natural interpretation of the "semi" push-pull design: only samples that violate the prescribed boundary or margin conditions generate the corresponding corrective updates, rather than continuously pulling or pushing all samples. This can reduce bias toward the few labeled anomalies while retaining corrective pressure on ambiguous samples, which is consistent with the motivation of BGAD. Extending our finite-time analysis from the center-based DOCC objective to such boundary- or margin-based supervised AD objectives is an interesting direction for future work.

Reviewer_cdUdOpenReview ↗

maintain my positive score

Thank you for the careful responses and I appreciate the authors' efforts. I will maintain my positive score of 3 .

Reviewer_KLAHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how normal-only training changes unseen representations in deep one-class classification. It describes this movement using the kernel at initialization and residuals from the training run, with conditional finite-time error, training-contraction and detection results. It also analyzes Gaussian initialization and adds a first-order interpretation of anomaly supervision. Controlled MNIST experiments test the approximation across network widths and training time, including its breakdown and differences between training and held-out inputs.

Critical strengths
  1. The paper gives an account of how normal-only training moves unseen representations. It separates the initial representation, movement caused by normal-training updates, and approximation error. The training-contraction result is kept separate from conditions for detecting an unseen anomaly. This helps explain the behavior of the existing detector, including why fitting training normals need not give good detection on unseen inputs (Sections 3).

  2. The experiments provide a good check of the approximation. It uses recorded training residuals without inserting the unseen final output into the approximation. Figure 3 shows early AUROC agreement and later divergence. Figure 7 fixes the training time, removes weight decay, and reports errors averaged over all ten tasks, including median and larger errors. Figure 4 also shows the training/held-out difference. These observations help establish where the explanation is useful.

Critical weaknesses

My understanding of the trajectory argument is:

  1. Use the coupling at initialization to approximate how training moves an unseen input’s representation.
  2. Under the stated assumptions, bound the approximation error as the weights change during training.
  3. Show that their Gaussian-initialized network satisfies the conditions needed for this error bound.

My main concern is with step 3. Proposition 1 takes the local derivative and linearization bounds from Zhou et al. (2026a). The cited proof uses cancellation from specially paired weights, whereas this paper initializes independently. I do not see the justification for applying those bounds to the initialization used here.

These bounds are used to obtain the stated Gaussian trajectory-error bound and sufficient conditions for contraction. The conditional analysis and the empirical approximation checks remain useful, but the supplied argument does not establish these bounds for the independently initialized network studied.

Questions for authors

What justifies the local bounds in Proposition 1 under independent initialization, and does that justification support the stated trajectory-error bound and conditions for contraction? (More in weakness)

1 comment
AuthorsOpenReview ↗

Response to Reviewer KLAH

W1 & Q1. Can the local derivative and linearization bounds in Proposition 1 be established under the independent Gaussian initialization used in the paper, and, if so, do they still imply the stated Gaussian trajectory-error bound and sufficient conditions for contraction?

A: Thank you for your thoughtful comment. We now provide the proofs of the local derivative and linearization bounds directly under independent Gaussian initialization (see Appendix E in the revision). The resulting bounds preserve the key asymptotic orders of the original analysis: both local errors remain of order R1/3m−1/6R^{1/3}m^{-1/6}, up to logarithmic factors. Consequently, the Gaussian trajectory-error bound and the sufficient conditions for contraction retain the same dependence on the network width mm and trajectory radius RTR_T.

In more detail, the revised Proposition 1 is, informally, as follows. Here, logarithmic factors, including the corresponding dependence on nn, ss and δ\delta, are suppressed, and the full proof is given in the revised manuscript.

Revised Proposition 1 (informal). Assume m≥poly(L,d,R)m \ge \mathrm{poly}(L,d,R). Then, with high probability under independent Gaussian initialization, Assumption 1 holds on BR(W(0))\mathcal{B}_R(\mathbf{W}(0)) with

M0≲L m,G0≲s L3/2,εgrad(R)≲s poly(L) R1/3m−1/6,εlin(R)≲s poly(L) R1/3m−1/6.M_0\lesssim \sqrt{L}\ m, \quad G_0\lesssim \sqrt{s}\ L^{3/2}, \quad \varepsilon_{\mathrm{grad}}(R) \lesssim \sqrt{s}\ \mathrm{poly}(L)\ R^{1/3}m^{-1/6}, \quad \varepsilon_{\mathrm{lin}}(R) \lesssim \sqrt{s}\ \mathrm{poly}(L)\ R^{1/3}m^{-1/6}.

Regarding the second part of the question, substituting these revised local bounds into the subsequent analysis leaves both the Gaussian trajectory-error result and the contraction conditions valid. Only their polynomial dependence on LL and logarithmic factors change (see the revised Proposition 2).

Finally, Theorem 4 (restated as Theorem 8 in the appendix) is unaffected by this revision. Importantly, its proof does not use the symmetric local perturbation estimates at issue above. Instead, it relies on the initialization analysis in Section 5 of Zhou et al. (2026a), in particular Lemmas 31 and 32, which analyze independently initialized Gaussian hidden layers. Therefore, the deterministic kernel κL\kappa_L and the stated concentration rate remain unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and the AC

We sincerely thank all reviewers for their careful reading and constructive feedback, based on which we were able to further improve the manuscript. We have uploaded a revised PDF, in which the changes in the main text are marked in blue. The conclusions of our original main theorems remain unchanged.

We summarize the main improvements here:

  1. Proof under independent Gaussian initialization (KLAH, 4Tgg). The local bounds are now established directly under independent Gaussian initialization (Lemma 9 and Proposition 1 in Appendix E), with the same R1/3m−1/6R^{1/3}m^{-1/6} dependence up to logarithmic factors.

  2. Extended experiments (4Tgg, cdUd). We report results for all ten MNIST tasks and on Fashion-MNIST (Appendix G.3).

  3. Initialization vs. training, and normal vs. anomalous inputs (4Tgg, cdUd). New experiments separate initialization from training effects and compare normals and anomalies at the same initial distance (Appendix G.3). We also add a direction-aware detection condition (Corollary 1).

  4. Implications for method design (cdUd). A new Discussion section (Section 6) discusses selective contraction, the scope of our guarantees, and possible extensions to methods such as BGAD.

We thank all reviewers and the AC again for their time and valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.