acceptodds
Under review as a conference paper at ICLR 2027

Masked Diffusion Modeling for Anomaly Detection

Abstract

Anomaly detection aims to identify samples that deviate from the nominal data distribution and is central to many safety-critical applications. However, developing effective anomaly detection methods for categorical and discrete data remains challenging and relatively underexplored. Masked diffusion models provide a natural way to model such data by learning to recover masked values from remaining visible context. In this paper, we propose Masked Diffusion for Anomaly Detection (MaskDiff-AD), a forward-only method based on masked diffusion models trained only on nominal data. Given a test sample, MaskDiff-AD calculates anomaly scores from the difficulty of reconstructing randomly masked coordinates, yielding a content-sensitive score that operates directly on discrete state spaces while avoiding reverse-time sampling. We also develop a non-parametric variant of MaskDiff-AD and provide theoretical analysis by characterizing Type-I and Type-II errors under a fixed detection threshold. Experiments on fourteen tabular datasets show that MaskDiff-AD achieves the best overall average rank among all evaluated methods, and experiments on four text anomaly detection datasets also demonstrate competitive performance against the baselines.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Anomaly Detection, Diffusion Models, Masked Modeling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UaC8OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper tackles the problem of anomaly detection on categorical, mixed-type, and discrete-sequence data, where continuous and density methods tend to struggle. MaskDiff-AD trains a masked diffusion model (MDM) on normal data and scores each point by the difficulty of reconstructing a random mask over the feature space, averaged across masks, so that scoring acts directly on the discrete state space. The authors offer a non-parametric (training-free) variant alongside a parametric one. Over fourteen tabular and four text datasets and roughly twelve baselines, the parametric variant achieves the best overall average rank.

Critical strengths
  • The paper targets a practically useful and under-explored setting. Scoring directly on discrete data through masked reconstruction is a clean idea, and having both parametric and non-parametric variants is practical.

  • The tabular evaluation is broad, with relevant recent baselines, representation controls, and variance across seeds. I believe the empirical results provide a good case.

Critical weaknesses
  • The theoretical guarantee assumes separation between normal and anomalous scores for Type-II error control. It helps to explain the effect of averaging, but it does not tell when masked reconstruction will actually distinguish anomalies in practice.

  • The advantage over existing methods is more modest than the overall rank suggests. DTE-Gaussian remains stronger on mean PR-AUC, and the text results are mixed. The experiments do show that the method is competitive for categorical anomaly detection, but give less evidence that modeling the discrete space is generally better than using continuous representations.

Questions for authors
  • Can you provide the code for the remaining experiments, particularly the baselines, representation controls, and text evaluation? It only includes a U2R demo as of now.
  • The code normalizes the training loss across the minibatch, while the paper describes normalization per masked view. Which objective was used for the reported experiments, and does this difference affect the results?
  • I tried a simple frequency baseline using only normal training data, and it got better average precision than the model in the code on U2R. Could you try this baseline on the other low-dimensional categorical datasets, too?
6 comments
AuthorsOpenReview ↗

Author Response (1/2)

We thank the reviewer for the careful and constructive assessment, and for recognizing the practical importance of the problem, the value of our parametric and non-parametric variants, and the breadth and strength of our tabular evaluation. We respond to the reviewer’s concerns and questions below.

Q1. Can you provide the code for the remaining experiments, particularly the baselines, representation controls, and text evaluation? It only includes a U2R demo as of now.

R1. Thank you for pointing this out. We have added the code for the remaining experiments to the anonymous folder https://anonymous.4open.science/r/supplementary-code-E485/. The repository now includes data-loading interfaces for the remaining 13 tabular datasets and all four text datasets, together with the text experiment and evaluation code. The datasets can be downloaded from their respective websites, with download links and setup instructions provided in the README.

For the representation controls, preprocess/tabular_preprocessor.py provides the ordinal_binned, onehot, and binned_onehot representations through the encode_method option. We have also added a Hamming-kNN baseline implementation. For the other baseline methods, we directly use the code released by their authors. Please refer to Appendix B.4 for the method references, preprocessing details, and evaluation protocols, and to Table 3 for the default hyperparameters used for the tabular baselines.

Q2. The code normalizes the training loss across the minibatch, while the paper describes normalization per masked view. Which objective was used for the reported experiments, and does this difference affect the results?

R2. Thank you for raising this point. We used minibatch normalization for the reported experiments, with the aim of improving training stability. When the minibatch loss is normalized by the total number of masked coordinates, it weights each view according to its masked-coordinate count, whereas Eq. (8) weights views equally. Theoretically, for views of length dd generated with independent coordinate masking at a common probability pp, the two objectives are close with high probability when dpdp is sufficiently large relative to log⁡B\log B, where BB is the number of views in the minibatch. Under this condition, the masked-coordinate counts concentrate around their common expectation dpdp, making the weights approximately uniform.

We now further compared the two normalization choices; the following results show similar detection performance under the two normalization choices. These results are averaged across 14 tabular datasets with 5 random seeds. Compared with per-view normalization, minibatch improved ROC-AUC on 8 of 14 datasets, increasing mean ROC-AUC from 79.94% to 80.66% and improving the average ROC-AUC rank from 4.000 to 3.714. The largest gains occurred on chess and solar, with improvements of 8.23 and 1.55 percentage points, respectively.

Metricminibatchper-view (eq8)
ROC-AUC80.66%79.94%
PR-AUC32.39%32.54%

Overall, the two choices yield similar rankings of our detector and leave our empirical conclusions unchanged. We will clarify the normalization used and the comparison between the two objectives in the final version.

AuthorsOpenReview ↗

Author Response (2/2)

Q3. I tried a simple frequency baseline using only normal training data, and it got better average precision than the model in the code on U2R. Could you try this baseline on the other low-dimensional categorical datasets, too?

R3. Thank you for suggesting this baseline and sharing your U2R observation.

First of all, our best understanding of the "frequency baseline" you mentioned is that it estimates feature-wise empirical frequencies from normal training data and uses these frequencies to construct an anomaly score. Under this interpretation, the underlying probability estimator is closely related to our nonparametric approach. Specifically, assigning equal kernel weights to all training samples in Eq. (7), corresponding to the limit λ→0\lambda\to0, recovers the empirical marginal frequency of each category.

Our nonparametric estimator provides more flexibility by further allowing these frequencies to depend on the visible context: training samples that agree more closely with the visible coordinates receive greater weight when estimating the distribution of a masked coordinate. This allows the estimator to account for relationships among features, including unusual combinations of individually common values that feature-wise marginal frequencies alone would not capture.

We agree that simple frequency-based scoring may be effective on low-dimensional categorical datasets when the relevant frequencies can be estimated reliably from normal training data. However, our goal is to provide a general masked-conditional scoring framework for a range of categorical tabular and discrete-sequence datasets, including settings where complicated feature dependencies are informative for detection. In this regard, our existing experiments support our main methodological conclusion: masked-conditional modeling provides a practical framework for anomaly detection across diverse discrete-data settings, with both parametric and nonparametric implementations. We would also welcome any clarification if the frequency baseline you tested differs from our feature-wise interpretation above.

We thank the reviewer again for the thoughtful comments and constructive suggestions, and would be happy to provide further clarification or address any remaining questions.

Thank you for your answers and for completing the extra experiments.

The updated repository is helpful, though it still doesn't contain the baselines. The comparison of the two objectives is also helpful. However, the paper should explicitly state which objective was actually used. It is unclear as of now in Section 3.2, which frames per-view normalization as a design choice, so it requires a proper revision. Also, the concentration argument you used doesn't really fit since it assumes a common mask rate and a large d, but the training uses a wide range of mask rates and many datasets have few features.

I agree with your interpretation of the frequency baseline, so since this is the λ→0\lambda \rightarrow 0 limit of your non-parametric estimator, it should be easy to run with your code. I think it's useful to run as because if simple per-feature frequencies, which ignore the other features, do as well as your full method on the low-dimensional datasets, we would see that conditional modellng is not what improves performance.

My second weakness about the small improvement over the strongest baseline on AUC-PR and about the mixed text results, was not addressed. I am maintaining my score for now.

Reviewer_WaBpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MaskDiff-AD, an anomaly detection method for discrete and mixed-type data. The model is trained exclusively on nominal samples. At test time, it applies random masks at multiple levels and uses the conditional reconstruction negative log-probability at the masked positions as the anomaly score. The authors develop both parametric and non-parametric variants, and discuss the degeneracy that arises when Diffusion Time Estimation is directly applied to absorbing masked diffusion. The theoretical analysis provides finite-view sampling error bounds and Type-I/Type-II error bounds under model misspecification. Experiments cover 14 tabular datasets and 4 text datasets. The parametric variant achieves an average ROC-AUC of 80.66 on the tabular datasets and the best overall average rank. On text data, the method performs strongly on Email Spam and SMS Spam, but is weaker than some pretrained-embedding-based methods on AGNews and YelpReview.

Critical strengths

Anomaly scoring suitable for discrete data. The method directly evaluates masked conditional probabilities in the discrete state space, avoiding continuous embeddings and reverse-time diffusion sampling. This is conceptually natural and has practical value.

Broad experimental coverage. The authors evaluate the method on 14 tabular datasets and 4 text datasets, and provide a non-parametric variant, runtime comparisons, probe-mask sensitivity analyses, and NELBO scoring comparisons. The runtime efficiency of the parametric variant on the large Vehicle Claims dataset is also promising.

Critical weaknesses

The experiments only cover sample-level anomaly detection on categorical/mixed-type tabular data and text. They do not support general claims about anomaly detection, industrial anomaly detection, or industrial vision tasks. The authors should clarify the scope and limitations of the method in the introduction and conclusion, and avoid overstating its broader applicability.

The core scoring function is essentially a masked conditional negative log-probability and is highly related to existing masked-prediction-based anomaly detection methods. The current experiments do not directly compare ordinary masked-prediction training with masked-diffusion training under the same network architecture, training budget, and test-time masking scheme. Therefore, they do not establish that diffusion training provides an independent benefit. The authors should add controlled comparisons of the training objectives, as well as single-mask-rate versus multi-mask-rate comparisons under the same total number of masked views.

The proposed model uses a three-layer MLP, whereas NPT-AD uses a Transformer with both feature-wise and sample-wise attention. The two methods therefore differ in model size, inference mechanism, masking strategy, and number of views. The reported speed advantage may primarily result from the model architecture and scale, and cannot be attributed simply to NPT-AD’s “repeated feature reconstruction.” The authors already report NPT-AD performance results in Appendix Table 5; these results should be summarized in the main paper. In addition, Table 4 should include training time, inference time, and resource usage for NPT-AD on the same hardware and datasets, where feasible.

The parametric and non-parametric variants are both implementations of MaskDiff-AD, but they use different default total numbers of views: 144 and 32, respectively. Their probe-mask grids also differ. Table 8 further shows that changing the view configuration of the non-parametric variant affects performance. Therefore, the performance gap between the two variants in Table 1 cannot be directly attributed to parametric modeling. The authors should compare the two variants using the same mask-rate grid, total number of views, and computational budget, while reporting both detection performance and inference cost. Such experiments would evaluate the performance and scalability benefits of the parametric variant, but would not by themselves establish that parameterization is necessary in all settings.

Questions for authors

Please address the concerns raised in the weaknesses above.

4 comments
AuthorsOpenReview ↗

Author Response (1/2)

We thank the reviewer for the careful and constructive assessment, and for recognizing the practical value of our method, its broad experimental coverage, and computational efficiency. We respond to the reviewer’s concerns and questions below.

Q1. Can the authors clarify the scope of MaskDiff-AD and avoid unsupported claims about broader anomaly detection applications, particularly industrial and vision tasks?

R1. Thank you for this suggestion. Our evaluated benchmarks cover application domains including network intrusion detection (Probe and U2R), insurance-claims auditing (Vehicle Claims), and spam detection (Email Spam and SMS Spam). We will soften the statements about industrial and vision applications to better reflect the coverage of our numerical experiments.

We would also like to emphasize that our main contribution is methodological, providing a general framework for anomaly detection on discrete data across application domains. Specifically, we formulate masked-diffusion conditional surprisal as a forward-only detector operating natively on discrete data, explain why DTE-style time estimation degenerates under absorbing masking, and develop both parametric and nonparametric implementations with corresponding Type-I/Type-II analysis.

Q2. What advantages does masked-diffusion training offer over standard masked-prediction training, and does multi-rate probing outperform single-rate probing under matched architectures and computational budgets?

R2. Advantage of masked-diffusion. (1) The main practical advantage of the diffusion formulation is that it allows us to reuse a suitable pretrained masked diffusion model (MDM) for anomaly detection. MDMs have demonstrated strong generative performance on discrete data, reflecting their ability to learn distributional structure and dependencies (Sahoo et al., 2024; Shi et al., 2024). MaskDiff-AD uses these learned conditional distributions to assess whether a test sample is consistent with nominal data. (2) Our Eq. (8) also provides a detection-oriented training option when a suitable pretrained MDM is unavailable. It retains the same absorbing-masking process and conditional-prediction formulation, while focusing training on the probe levels used for detection. Thus, our framework accommodates both the reuse of pretrained MDMs and training a conditional predictor specifically for the detection task.

Compared with existing masked-prediction methods such as MCM (Yin et al., 2024), the masking mechanism and anomaly statistic differ. (1) MCM jointly learns an input-dependent mask generator and an autoencoder, whereas our MaskDiff-AD uses masks generated from a prescribed absorbing process. (2) MCM is formulated for numerical tabular vectors and scores Euclidean reconstruction error in a continuous representation. Our method instead operates directly on categorical states and token sequences by scoring masked conditional surprisal. (3) Empirically, parametric MaskDiff-AD outperforms MCM on 13/14 datasets in ROC-AUC and 11/14 datasets in PR-AUC, improving the mean ROC-AUC and PR-AUC by 7.97 and 7.98 percentage points, respectively.

Multi-rate probing under a matched inference budget. We compare single-rate probing with K=144K=144 against nine-rate probing with τ∈{0.1,…,0.9}\tau\in\{0.1,\ldots,0.9\} and K=16K=16 per rate. Both settings thus use 144 views per test sample.

Settingu2r ROC-AUCu2r PR-AUCbank ROC-AUCbank PR-AUC
Single-rate τ=0.198.7736.1664.9720.30
Single-rate τ=0.299.1039.3265.5321.31
Single-rate τ=0.399.3541.0565.9222.31
Single-rate τ=0.499.5343.6865.9723.01
Single-rate τ=0.599.5743.2765.8323.63
Single-rate τ=0.699.5244.2465.7924.09
Single-rate τ=0.799.4242.4965.4823.84
Single-rate τ=0.899.3342.8064.9923.29
Single-rate τ=0.999.2845.6864.6022.78
Multi-rate τ=0.1–0.999.5546.3765.9023.90

Multi-rate probing improves U2R PR-AUC by 0.69 percentage points over the best single rate and remains within 0.19 percentage points of the best single-rate results on the other three metrics. These results support our current interpretation that multi-rate aggregation provides robustness to the choice of mask level, achieving near-best performance without requiring dataset-specific selection.

AuthorsOpenReview ↗

Author Response (2/2)

Q3. Can the authors provide a fairer comparison with NPT-AD, including detection performance, training and inference time, and resource usage, while accounting for architectural differences?

R3. We thank the reviewer for this suggestion. We agree that the efficiency comparison should explicitly account for architectural differences rather than attributing the speed advantage solely to NPT-AD's repeated feature reconstruction.

We evaluated the official NPT-AD implementation on six datasets for which it was computationally feasible: bank, chess, cmc, probe, solar, and u2r. As reported in Appendix Table 5, Parametric MaskDiff-AD achieves mean ROC-AUC/PR-AUC of 80.45/31.45, compared with 76.46/27.97 for NPT-AD, outperforming it on five of the six datasets under each metric.

We now additionally compared training and inference times on the same machine equipped with a single NVIDIA RTX 5090 GPU (32 GB).

DatasetNPT-AD Training (s)NPT-AD Inference (s)Parametric MaskDiff-AD Training (s)Parametric MaskDiff-AD Inference (s)
bank713.9919.663.871.84
chess457.848.682.520.91
cmc530.660.330.800.12
probe424.8218.474.752.09
solar719.420.310.760.15
u2r422.6717.444.782.02

Using the summed runtimes across these six datasets, Parametric MaskDiff-AD achieves approximately 187× faster training and 9.1× faster inference under the evaluated configurations. We emphasize that these results demonstrate practical efficiency rather than an architecture-controlled speedup, since the two methods differ in model architecture, capacity, masking strategies, and number of inference views.

On several larger or higher-dimensional datasets, the official NPT-AD implementation exceeded the available GPU memory under our tested configuration. NPT-AD's sample-wise attention operates on flattened representations of dimension DhDh, where DD is the number of features and hh is the per-feature embedding width. The dense projection and feed-forward layers operating on these representations can have O(D2h2)O(D^2h^2) parameters, while sample-wise attention introduces additional memory requirements that depend on the number of reference samples. These architectural characteristics contribute to NPT-AD's resource demands and help explain the observed scalability limitations.

We will summarize the NPT-AD detection results in the main paper, extend Table 4 with the same-hardware runtime comparison and resource constraints, and revise the efficiency discussion to explicitly acknowledge the architectural differences.

Q4. Can the authors compare the parametric and non-parametric variants under matched probe grids, numbers of views, and computational budgets to isolate their performance and efficiency differences?

R4. Thank you for this suggestion. We now compare both variants under two matched probe configurations: the same four-level grid with K=8K=8 (32 total views), and the same nine-level grid with K=16K=16 (144 total views).

MethodProbe gridKKTotal viewsMean ROC-AUCMean PR-AUC
Nonparametric MaskDiff-AD{0.15,0.30,0.45,0.60}\{0.15,0.30,0.45,0.60\}83279.1231.64
Parametric MaskDiff-AD{0.15,0.30,0.45,0.60}\{0.15,0.30,0.45,0.60\}83278.8432.04
Nonparametric MaskDiff-AD{0.1,0.2,…,0.9}\{0.1,0.2,\ldots,0.9\}1614479.3632.51
Parametric MaskDiff-AD{0.1,0.2,…,0.9}\{0.1,0.2,\ldots,0.9\}1614480.6632.39

At 32 views, the nonparametric variant has a slightly higher mean ROC-AUC, while the parametric variant has a slightly higher mean PR-AUC. At 144 views, the parametric variant improves mean ROC-AUC by 1.30 percentage points, while the nonparametric variant leads in mean PR-AUC by only 0.12 points. Thus, both variants offer competitive performance under matched probe configurations, allowing users to choose according to their practical requirements and preferences.

We thank the reviewer again for the thoughtful comments and constructive suggestions, and would be happy to provide further clarification or address any remaining questions.

Thank you for the detailed response and additional experiments. I appreciate the authors’ efforts to address my concerns, particularly the matched-budget probing experiments and the additional comparisons with NPT-AD.

The additional results provide useful evidence for the practical effectiveness of MaskDiff-AD. However, my concerns regarding the methodological novelty and the benefit of masked-diffusion training over standard masked prediction remain only partially addressed.

Overall, while the rebuttal has clarified several aspects of the approach, I do not find the additional evidence sufficient to change my original assessment. I therefore maintain my rating of 2.

AuthorsOpenReview ↗

Additional Ablations on Training and Anomaly Scoring

Thank you for the thoughtful follow-up. We appreciate the recognition of our additional experiments. To address the remaining concerns regarding methodological novelty and the benefit of masked-diffusion training, we conducted additional comparisons examining the contributions of the training objective and anomaly scoring procedure. All ROC-AUC and PR-AUC results are averaged over five seeds and macro-averaged across 14 datasets, while training and inference times are summed across the 14 datasets.

MethodROC-AUC (%)PR-AUC (%)Training time (s)Inference time (s)
Parametric MaskDiff-AD80.6632.39455.85154.49
Hard masked prediction error75.0725.44455.85108.94
MDM reverse reconstruction73.0921.77455.85166.49
MDM-trained + reverse reconstruction72.2121.69601.07163.67
MDM-trained + MaskDiff-AD scoring80.2932.12601.07154.98

We describe each method included in the comparison below:

  • Hard masked prediction error: Reuses MaskDiff-AD checkpoints and replaces probability-based anomaly score with the fraction of incorrectly predicted masked features (argmax), averaged over nine mask rates (0.1–0.9) and 16 views per rate. This isolates probability-based scoring under identical checkpoints and probing budgets.

  • MDM reverse reconstruction: Reuses the same checkpoints but performs nine reverse-sampling steps from 90% masking, with 16 trajectories (144 model evaluations). Reconstruction errors are computed as the fraction of incorrectly reconstructed features, normalized by the number of initially masked positions.

  • MDM-trained + reverse reconstruction: We train a separate model from scratch using an MDM ELBO objective, retaining the same backbone, preprocessing, data splits, optimizer settings, and training epochs. Training uniformly samples ten discrete masking levels from 0.1 to 1.0 and minimizes masked-feature cross-entropy weighted by the inverse masking probability and normalized by the feature count. Evaluation uses the same reverse-sampling procedure.

  • MDM-trained + MaskDiff-AD scoring: We reuse the MDM-trained checkpoints and apply MaskDiff-AD’s probabilistic anomaly scoring without retraining, using nine mask rates and 16 views per rate, with normalization by the realized masked count.

We summarize the key findings from these results:

(1) Flexible use of pretrained MDMs or task-specific training. As described in our manuscript, MaskDiff-AD can directly reuse a pretrained MDM without additional training. Applying our scoring procedure to a standard MDM-trained checkpoint achieves 80.29/32.12 ROC-AUC/PR-AUC, close to our original 80.66/32.39. When no pretrained MDM is available, our task-specific training learns only the conditionals needed at the probe levels, achieving slightly better performance while reducing training time from 601.07s to 455.85s (1.32× faster). This demonstrates the flexibility of our framework across both settings.

(2) Probability-based scoring vs. hard masked prediction. Using identical trained checkpoints, masking rates, and 144 probe views per sample, replacing our conditional negative log-likelihood score with hard masked prediction errors reduces ROC-AUC from 80.66% to 75.07% and PR-AUC from 32.39% to 25.44%. This comparison isolates the benefit of retaining predictive probabilities rather than relying on hard reconstruction errors.

(3) Forward-only scoring vs. iterative reverse reconstruction. Using the same trained checkpoints and 144 model evaluations per sample, iterative reverse reconstruction achieves 73.09% ROC-AUC and 21.77% PR-AUC, compared with 80.66% and 32.39% for our forward-only score. Furthermore, applying MaskDiff-AD scoring to the standard MDM-trained model achieves 80.29% and 32.12%, compared with 72.21% and 21.69% using reverse reconstruction. These comparisons demonstrate the practical effectiveness of directly evaluating masked conditional probabilities rather than relying on sampled reconstruction discrepancies.

Clarification of our contribution. Our primary contribution is not a new masked-diffusion training objective, but a flexible, forward-only anomaly detection framework that leverages masked conditional probabilities learned by MDMs. It can directly reuse existing MDM checkpoints without additional training, or employ a lightweight task-specific training procedure when no pretrained model is available. This is complemented by our multi-probe scoring formulation, characterization of DTE degeneracy under absorbing masking, nonparametric method, and theoretical detection analysis.

We will incorporate these additional experimental results into the revised manuscript, emphasize the two practical settings, and further clarify our methodological contributions. We thank the reviewer again for the constructive feedback.

Reviewer_aqEjOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes MaskDiff-AD for anomaly detection in discrete data. Specifically, it learns masked conditional distributions from normal samples and scores test samples by the average negative log-probability of masked values across multiple masking levels and random masks, without reverse-time sampling. The authors also introduce a non-parametric variant and provide conditional bounds on false-positive and false-negative errors. Experiments show strong average rankings on tabular data and mixed results on text data.

Critical strengths
  1. MaskDiff-AD uses masked conditional probabilities to detect unusual feature relationships without reverse-time sampling. The analysis of why DTE-style time estimation degenerates under absorbing masking provides a clear motivation for this choice.

  2. The method achieves the best overall average rank across 14 tabular datasets under normal-only training. Additional comparisons and sensitivity analyses strengthen the evidence that its performance is robust across different datasets and evaluation settings.

  3. The writing of this manuscript is good and easy to follow.

Critical weaknesses
  1. The method uses masked conditional negative log-probabilities averaged over masking levels and random masks. Its distinction from conventional masked reconstruction is limited. Appendix C.4 also shows only modest gains over matched-compute NELBO scoring alternatives, leaving the specific benefit of the diffusion formulation unclear.

  2. The error bounds require sufficient separation between normal and anomalous oracle scores and controlled model approximation errors on anomalous inputs. These conditions are difficult to verify using normal-only data. The theory therefore provides limited guidance on when the method will successfully detect anomalies.

  3. The method performs well on the two spam datasets but falls substantially behind strong embedding-based baselines on AGNews and YelpReview. This limits the evidence that the proposed scoring method can reliably detect semantic anomalies across different text domains.

Questions for authors
  1. What distinguishes MaskDiff-AD from a standard masked reconstruction model trained on normal data with the same architecture and masking levels? Please clarify which benefits specifically require the diffusion formulation.

  2. Appendix C.4 shows modest average gains over NELBO scoring alternatives. Are these gains statistically significant across datasets and random seeds?

  3. Under what practical conditions are the assumed oracle-score separation and anomalous-side approximation error controlled? Please explain whether these conditions can be assessed using only normal data.

  4. What explains the large performance gap on AGNews and YelpReview? Please provide evidence that distinguishes limitations of the scoring rule from those of the learned text representation.

4 comments
AuthorsOpenReview ↗

Authors Response (1/2)

We thank the reviewer for the careful and constructive assessment, and for recognizing our motivating analysis of DTE degeneracy, computational efficiency, strong tabular results, and clear presentation. We respond to the reviewer’s concerns and questions below.

Q1. What distinguishes MaskDiff-AD from a standard masked reconstruction model trained on normal data with the same architecture and masking levels? Please clarify which benefits specifically require the diffusion formulation.

R1. Thank you for raising this question. (1) The main practical advantage of the diffusion formulation is that it allows us to reuse a suitable pretrained masked diffusion model (MDM) for anomaly detection. MDMs have demonstrated strong generative performance on discrete data, reflecting their ability to learn distributional structure and dependencies among coordinates (Sahoo et al., 2024; Shi et al., 2024). MaskDiff-AD uses these learned conditional distributions to assess whether a test sample is consistent with nominal data. When an MDM modeling the nominal distribution is available, we can directly construct anomaly scores from its conditional predictions, without training a separate reconstruction model or performing reverse-time sampling. (2) Our Eq. (8) also provides a detection-oriented training option when a suitable pretrained MDM is unavailable. It retains the same absorbing-masking process and conditional-prediction formulation, while focusing training on the probe levels used for detection. Thus, our framework accommodates both the reuse of pretrained MDMs and training a conditional predictor specifically for the detection task.

Compared with existing masked-reconstruction methods such as MCM (Yin et al., 2024), the masking mechanism and anomaly statistic differ. (1) MCM jointly learns an input-dependent mask generator and an autoencoder, whereas our MaskDiff-AD uses masks generated from a prescribed absorbing process. (2) MCM is formulated for numerical tabular vectors and scores Euclidean reconstruction error in a continuous representation. Our method instead operates directly on categorical states and token sequences by scoring masked conditional surprisal. This formulation also makes the detector compatible with the conditional predictions given by pretrained MDMs. (3) Empirically, parametric MaskDiff-AD outperforms MCM on 13/14 datasets in ROC-AUC and 11/14 datasets in PR-AUC, improving the mean ROC-AUC and PR-AUC by 7.97 and 7.98 percentage points, respectively.

Q2. Appendix C.4 shows modest average gains over NELBO scoring alternatives. Are these gains statistically significant across datasets and random seeds?

R2. The comparison in Appendix C.4 uses the same trained model and inference budget, with results over 5 random seeds on each of the 14 tabular datasets reported in Tables 13–14. To answer the reviewer's question on statistical significance, we conducted exact two-sided paired Wilcoxon tests with Holm correction across datasets for the MaskDiff-AD-vs-NELBO comparisons. For ROC-AUC, MaskDiff-AD shows a small but statistically significant advantage over Fixed-grid NELBO (adjusted pp-value = 0.0449). Although the average gain is modest, the larger dataset-wise differences predominantly favor MaskDiff-AD. The other pairwise comparisons, like those using ROC-AUC metric, do not show statistical significance.

Nevertheless, Parametric MaskDiff-AD achieves the highest mean ROC-AUC and PR-AUC, at 80.66% and 32.39%, respectively. MaskDiff-AD records 9/3/29/3/2 and 6/6/26/6/2 wins/losses/ties against Fixed-grid and MC-NELBO, respectively, for ROC-AUC, and 8/4/28/4/2 and 6/7/16/7/1 for PR-AUC. Therefore, the benefits of fixed-grid probing and realized-mask normalization remain heterogeneous across datasets.

AuthorsOpenReview ↗

Authors Response (2/2)

Q3. Under what practical conditions are the assumed oracle-score separation and anomalous-side approximation error controlled? Please explain whether these conditions can be assessed using only normal data.

R3. First of all, the assumed oracle-score separation is common and necessary to guarantee the effectiveness of anomaly detection. Such separation cannot be assessed beforehand using only normal data since it requires knowledge about the anomalous distribution. No detector can guarantee separation from arbitrary anomalies using only normal data if there are no restrictions on the anomalous distribution.

Our analysis actually quantifies the separation conditions explicitly. As shown in Remark 2, the oracle separation at probe level τℓ\tau_\ell is μ1,ℓ∗−μ0,ℓ∗=Dℓ+Hℓ′−Hℓ\mu_{1,\ell}^* - \mu_{0,\ell}^* = D_\ell + H_\ell' - H_\ell, where DℓD_\ell is the mask-averaged conditional KL divergence, and Hℓ′H_\ell' and HℓH_\ell are the corresponding conditional entropies under the alternative and null. Separation is positive when Dℓ>Hℓ−Hℓ′D_\ell>H_\ell-H_\ell'. In particular, when anomalies change conditional dependencies while leaving conditional entropies approximately unchanged, the gap is approximately DℓD_\ell. The multi-probe gap is the corresponding average over levels.

Second, for approximation error, Remark 3 in Appendix D shows that ϵ1≤L−1∑ℓηℓ\epsilon_1\le L^{-1}\sum_\ell\eta_\ell if p^j(a∣x~,τℓ)≤eηℓqj(a∣x~,τℓ)\hat p^j(a\mid\tilde{x},\tau_\ell)\le e^{\eta_\ell}q^j(a\mid\tilde{x},\tau_\ell) for all coordinates and values on masked contexts encountered under the alternative. Thus, the error is small when the learned model does not substantially overestimate nominal conditional probabilities on these contexts. More nominal training data can help when they yield accurate conditional estimates there.

These alternative-dependent conditions cannot generally be verified using normal data alone, as power guarantees throughout the anomaly detection literature inherently rely on assumptions about the anomalous distribution. Nevertheless, our empirical Type-I error control remains valid under model misspecification. We calibrate the threshold using independent nominal samples, with corrections for finite calibration data and probe views, without requiring knowledge of the anomalous distribution, the oracle nominal mean, or its approximation error.

Q4. What explains the large performance gap on AGNews and YelpReview? Please provide evidence that distinguishes limitations of the scoring rule from those of the learned text representation.

R4. Our current text model is trained from scratch using only normal text sequences in each dataset, whereas the strongest baselines on AGNews and YelpReview use OpenAI embeddings pretrained on large-scale external data. We therefore suspect that representation quality and model capacity contribute to the gap, particularly in capturing semantic and topic information.

Table 12 provides supporting evidence: keeping LOF as the downstream detector, changing from BERT to OpenAI embeddings increases ROC-AUC from 74.32% to 89.05% on AGNews and from 65.73% to 87.33% on YelpReview. Meanwhile, our masked-conditional scoring rule achieves the best ROC-AUC and PR-AUC on Email Spam and SMS Spam.

These observations support the importance of representation. We will clarify this interpretation and highlight pretrained, long-context masked diffusion backbones as an important future direction for improving semantic modeling.

We thank the reviewer again for the thoughtful comments and constructive suggestions, and would be happy to provide further clarification or address any remaining questions.

AuthorsOpenReview ↗

Rebuttal follow-up

Dear Reviewer aqEj,

Thank you again for your thoughtful review and constructive suggestions.

With roughly one day remaining in the discussion period, we would like to kindly follow up on our rebuttal, where we addressed your concerns regarding methodological novelty, statistical significance, theoretical assumptions, and text performance.

As an additional update to R1, we would also like to point you to our recent follow-up with Reviewer WaBp, titled Additional Ablations on Training and Anomaly Scoring. There, we report new experiments showing that MaskDiff-AD can directly reuse standard MDM checkpoints without additional training, while its probabilistic scoring substantially outperforms hard masked prediction errors and iterative reverse reconstruction under matched inference budgets.

We would greatly appreciate knowing whether you have any remaining concerns or questions. We understand that your time is limited, and our goal is to ensure that any outstanding issues can be addressed before the discussion closes. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.

Thank you again for your time and consideration.

Thank you for the careful responses. The detailed explanation has addressed my main concerns. I will maintain my original score of 3 (weak acceptance).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.