- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The paper tackles the problem of anomaly detection on categorical, mixed-type, and discrete-sequence data, where continuous and density methods tend to struggle. MaskDiff-AD trains a masked diffusion model (MDM) on normal data and scores each point by the difficulty of reconstructing a random mask over the feature space, averaged across masks, so that scoring acts directly on the discrete state space. The authors offer a non-parametric (training-free) variant alongside a parametric one. Over fourteen tabular and four text datasets and roughly twelve baselines, the parametric variant achieves the best overall average rank.
Critical strengths
-
The paper targets a practically useful and under-explored setting. Scoring directly on discrete data through masked reconstruction is a clean idea, and having both parametric and non-parametric variants is practical.
-
The tabular evaluation is broad, with relevant recent baselines, representation controls, and variance across seeds. I believe the empirical results provide a good case.
Critical weaknesses
-
The theoretical guarantee assumes separation between normal and anomalous scores for Type-II error control. It helps to explain the effect of averaging, but it does not tell when masked reconstruction will actually distinguish anomalies in practice.
-
The advantage over existing methods is more modest than the overall rank suggests. DTE-Gaussian remains stronger on mean PR-AUC, and the text results are mixed. The experiments do show that the method is competitive for categorical anomaly detection, but give less evidence that modeling the discrete space is generally better than using continuous representations.
Questions for authors
- Can you provide the code for the remaining experiments, particularly the baselines, representation controls, and text evaluation? It only includes a U2R demo as of now.
- The code normalizes the training loss across the minibatch, while the paper describes normalization per masked view. Which objective was used for the reported experiments, and does this difference affect the results?
- I tried a simple frequency baseline using only normal training data, and it got better average precision than the model in the code on U2R. Could you try this baseline on the other low-dimensional categorical datasets, too?
6 comments
Author Response (2/2)
Q3. I tried a simple frequency baseline using only normal training data, and it got better average precision than the model in the code on U2R. Could you try this baseline on the other low-dimensional categorical datasets, too?
R3. Thank you for suggesting this baseline and sharing your U2R observation.
First of all, our best understanding of the "frequency baseline" you mentioned is that it estimates feature-wise empirical frequencies from normal training data and uses these frequencies to construct an anomaly score. Under this interpretation, the underlying probability estimator is closely related to our nonparametric approach. Specifically, assigning equal kernel weights to all training samples in Eq. (7), corresponding to the limit , recovers the empirical marginal frequency of each category.
Our nonparametric estimator provides more flexibility by further allowing these frequencies to depend on the visible context: training samples that agree more closely with the visible coordinates receive greater weight when estimating the distribution of a masked coordinate. This allows the estimator to account for relationships among features, including unusual combinations of individually common values that feature-wise marginal frequencies alone would not capture.
We agree that simple frequency-based scoring may be effective on low-dimensional categorical datasets when the relevant frequencies can be estimated reliably from normal training data. However, our goal is to provide a general masked-conditional scoring framework for a range of categorical tabular and discrete-sequence datasets, including settings where complicated feature dependencies are informative for detection. In this regard, our existing experiments support our main methodological conclusion: masked-conditional modeling provides a practical framework for anomaly detection across diverse discrete-data settings, with both parametric and nonparametric implementations. We would also welcome any clarification if the frequency baseline you tested differs from our feature-wise interpretation above.
We thank the reviewer again for the thoughtful comments and constructive suggestions, and would be happy to provide further clarification or address any remaining questions.
Thank you for your answers and for completing the extra experiments.
The updated repository is helpful, though it still doesn't contain the baselines. The comparison of the two objectives is also helpful. However, the paper should explicitly state which objective was actually used. It is unclear as of now in Section 3.2, which frames per-view normalization as a design choice, so it requires a proper revision. Also, the concentration argument you used doesn't really fit since it assumes a common mask rate and a large d, but the training uses a wide range of mask rates and many datasets have few features.
I agree with your interpretation of the frequency baseline, so since this is the limit of your non-parametric estimator, it should be easy to run with your code. I think it's useful to run as because if simple per-feature frequencies, which ignore the other features, do as well as your full method on the low-dimensional datasets, we would see that conditional modellng is not what improves performance.
My second weakness about the small improvement over the strongest baseline on AUC-PR and about the mixed text results, was not addressed. I am maintaining my score for now.
Author Response (1/2)
We thank the reviewer for the careful and constructive assessment, and for recognizing the practical importance of the problem, the value of our parametric and non-parametric variants, and the breadth and strength of our tabular evaluation. We respond to the reviewer’s concerns and questions below.
Q1. Can you provide the code for the remaining experiments, particularly the baselines, representation controls, and text evaluation? It only includes a U2R demo as of now.
R1. Thank you for pointing this out. We have added the code for the remaining experiments to the anonymous folder https://anonymous.4open.science/r/supplementary-code-E485/. The repository now includes data-loading interfaces for the remaining 13 tabular datasets and all four text datasets, together with the text experiment and evaluation code. The datasets can be downloaded from their respective websites, with download links and setup instructions provided in the README.
For the representation controls, preprocess/tabular_preprocessor.py provides the ordinal_binned, onehot, and binned_onehot representations through the encode_method option. We have also added a Hamming-kNN baseline implementation. For the other baseline methods, we directly use the code released by their authors. Please refer to Appendix B.4 for the method references, preprocessing details, and evaluation protocols, and to Table 3 for the default hyperparameters used for the tabular baselines.
Q2. The code normalizes the training loss across the minibatch, while the paper describes normalization per masked view. Which objective was used for the reported experiments, and does this difference affect the results?
R2. Thank you for raising this point. We used minibatch normalization for the reported experiments, with the aim of improving training stability. When the minibatch loss is normalized by the total number of masked coordinates, it weights each view according to its masked-coordinate count, whereas Eq. (8) weights views equally. Theoretically, for views of length d generated with independent coordinate masking at a common probability p, the two objectives are close with high probability when dp is sufficiently large relative to logB, where B is the number of views in the minibatch. Under this condition, the masked-coordinate counts concentrate around their common expectation dp, making the weights approximately uniform.
We now further compared the two normalization choices; the following results show similar detection performance under the two normalization choices. These results are averaged across 14 tabular datasets with 5 random seeds. Compared with per-view normalization, minibatch improved ROC-AUC on 8 of 14 datasets, increasing mean ROC-AUC from 79.94% to 80.66% and improving the average ROC-AUC rank from 4.000 to 3.714. The largest gains occurred on chess and solar, with improvements of 8.23 and 1.55 percentage points, respectively.
Overall, the two choices yield similar rankings of our detector and leave our empirical conclusions unchanged. We will clarify the normalization used and the comparison between the two objectives in the final version.