- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors investigate a family of diffusion models for the use as anomaly detectors. The mechanism used is the reconstruction quality of non-normal samples if they are reconstructed with a model that is only trained on a normal distribution. This is a commonly used mechanism for anomaly detection with generative models. In the family of the considered models, the reconstruction is an expensive operation as it usually needs hundreds of network evaluation per sample. The authors challenge this idea with the proposed one-step anomaly detection. They theoretically show that under mild assumptions the proposed one step method is optimal and corroborate the same with experiments across a diverse set of datasets and comparing methods.
Critical strengths
The paper doesn’t not propose a new architecture per se, but challenges the existing use of multiple timesteps of denoising for reconstruction in anomaly detection tasks. The solution of the one-step denoising anomaly detection is fundamentally a novel idea that the authors support with the theoretical as well as empirical results. This method is not only applicable to one specific model, but to basically all diffusion based models in the context of anomaly detection.
Critical weaknesses
Not critical weaknesses but things I’d like to bring to attention:
The git repository is empty/seems to be missing.
I was unable to follow all the proofs in detail, but the assumptions and general ideas seem reasonable. The proofs however are rather technical.
Questions for authors
I’d like to add a few minor things that I’d ask the authors to consider adressing:
-
The softmax notation is a little bit an abuse - I understand that it is more cumbersome to write out the full input/output vectors, but I think it might make it slightly clearer.
-
Many parts of the proofs and chains of equations are quite densely written. I’d like to encourage the authors to spend a little bit more “prose” to explain the intermediate step. Two examples I stumbled across: In proof of lemma A.5 the way z is defined comes a little bit out of the blue. What was the insight in defining it like this? Similarly e.g. on line 1128 the reasoning looks a bit weird, is it maybe an LLM artifact? Finally I’d also prefer the limits of the sums to be made explicit in (at least once per chain of equations), to make it easier to follow the reasoning.
5 comments
Response to Reviewer HQ84 (part2)
In particular, we address the following questions:
Why was defined that way in Lemma A.5?
Here, we briefly introduce its motivation (see Proof sketch of A.5 in detail):
To prove , we analyze the anomaly leakage factor
Define . Then, each factor can be expressed as
= \frac{\lambda(1-t_k)(1-t_{k+1})+t_k t_{k+1}}{\lambda(1-t_k)^2+t_k^2} = \frac{\langle \boldsymbol{z}_\lambda(t_{k+1}),\boldsymbol{z}_\lambda(t_k)\rangle}{\lVert\boldsymbol{z}_\lambda(t_k)\rVert_2^2}.$$ Thus, each factor has a geometric interpretation as the inner product of consecutive vectors ($\boldsymbol{z}_\lambda(t_k), \boldsymbol{z}_\lambda(t_{k+1})$) normalized by the squared norm at $t_k$. > What justifies the reasoning at the original line 1128? Thank you for pointing this out. We clarify that the reasoning at L1128 follows from the strict monotonicity of the argument of $\boldsymbol{z}_\lambda(t)$ and the ordered denoising schedule. Specifically, for $\lambda > 0$ and $0 = t_K < t_{K-1} < \cdots < t_0 = \tau < 1$, define $$\phi(t):=\arg\boldsymbol{z}_\lambda(t) =\arctan\left(\frac{t}{\sqrt{\lambda}(1-t)}\right).$$ Since $$\phi'(t)=\frac{\sqrt{\lambda}}{\lambda(1-t)^2+t^2} > 0,$$ the argument is strictly increasing on $[0,1)$, implying $0 < \theta = \phi(\tau)-\phi(0) < \pi/2$. Consequently, each consecutive angle satisfies $$\theta_k =\angle\bigl(\boldsymbol{z}_\lambda(t_k),\boldsymbol{z}_\lambda(t_{k+1})\bigr) =\phi(t_k)-\phi(t_{k+1}) > 0.$$ Summing these angular differences yields the telescoping identity $$\sum_{k=0}^{K-1}\theta_k =\sum_{k=0}^{K-1}\bigl[\phi(t_k)-\phi(t_{k+1})\bigr] =\phi(\tau)-\phi(0)=\theta.$$ Thus, the monotonicity of the argument ensures that all consecutive angular differences are positive and additive. We have explicitly clarified this reasoning in the revised proof. We need to clarify that, in all theoretical derivations, **we did not use an LLM to write proofs or claims.** As we specified in the AI use disclosure, we only use an LLM for feedback and to review the correctness of the formulation. Finally, we hope that these revisions have improved the clarity and transparency of the mathematical arguments, making them more accessible and easier for readers to follow.Gentle Reminder
Dear Reviewer HQ84,
Thank you again for your careful review, constructive feedback, and positive assessment of our work.
In our rebuttal, we addressed your comments on code accessibility and the clarity of the mathematical derivations. In the revised manuscript, we revised the notation for the softmax and summation and added a detailed proof sketch for each theorem.
As the discussion period closes on November 18, we would like to check whether our response has adequately addressed your concerns. If you have any questions, we would be happy to provide further clarification.
Best regards, The Authors of Submission 6795
Response to Reviewer HQ84 (part1)
Thank you for the detailed review and constructive feedback. Here, we address each point below.
1. Code repository accessibility
Thank you for pointing out that the code repository was inaccessible during review. We have updated the repository’s visibility settings, and the code is now accessible at the following link: https://anonymous.4open.science/r/odd. We will publicly release the code upon acceptance.
2. Softmax notation
We appreciate the reviewer’s suggestion to clarify the vector-to-vector mapping of the softmax function.
In the revised manuscript, we have modified the notation to explicitly specify both the input and output vectors. Specifically, we have revised the notation as follows:
Before revision (Lines 126–127):
:= \operatorname{softmax}\left( -\frac{\lVert\boldsymbol{x}-(1-t)\boldsymbol{x}^{(i)}\rVert_2^2}{2t^2} \right).$$ **After revision (Lines 126–127):** $$\left[\gamma_i(\boldsymbol{x},t)\right]_{i=1}^{n} := \operatorname{softmax}\left( \left[ -\frac{\lVert\boldsymbol{x}-(1-t)\boldsymbol{x}^{(j)}\rVert_2^2}{2t^2} \right]_{j=1}^{n} \right).$$ We have also added an accompanying definition of the indexed-vector notation, $$[a_i]_{i=1}^{m} := (a_1,\ldots,a_m)^\top,$$ to explicitly clarify this convention. ### 3. Clarity of proofs and derivations > Many parts of the proofs and chains of equations are quite densely written. I’d like to encourage the authors to spend a little bit more “prose” to explain the intermediate step. Two examples I stumbled across: In proof of lemma A.5 the way z is defined comes a little bit out of the blue. What was the insight in defining it like this? Similarly e.g. on line 1128 the reasoning looks a bit weird, is it maybe an LLM artifact? Finally I’d also prefer the limits of the sums to be made explicit in (at least once per chain of equations), to make it easier to follow the reasoning. Thanks for the valuable suggestion! To improve the clarity and readability of the derivations, we have revised the manuscript while preserving the original theoretical claims and their proof strategies. Specifically, we have made the following revisions: - **Added proof sketches:** We have included a proof sketch before each detailed proof to provide an overview of the main arguments and facilitate understanding of the subsequent derivations. - **Clarified summation indices:** We have explicitly specified the ranges of the summation indices to eliminate potential ambiguity. - **Clarified the proof of A.5:** We have expanded the corresponding proof sketch to provide additional context regarding the definition and role of $z$ in the derivation. - **Added intermediate derivation steps:** For the argument originally presented in L1128 (now L1177–1197 in the revised manuscript), we have included additional intermediate steps to improve clarity.