acceptodds
Under review as a conference paper at ICLR 2027

LAnoDet: Self-Supervised Lung Anomaly Detection with Transformer-Based Diffusion Models

Abstract

Detecting anomalies in chest X-rays is central to the timely diagnosis of lung disease, but labelled abnormal images are scarce and the abnormalities are often subtle. Reconstruction-based generative models trained only on normal images offer a label-free alternative in which an anomaly is revealed by the error between an image and its reconstruction. We identify three limitations of current diffusion-based detectors: convolutional U-Net backbones capture local texture but not the long-range structure of the thorax, whereas global self-attention costs quadratic time in the number of patches; the noise-prediction objective and stochastic sampler inject sampling variance into the very residual that serves as the anomaly score; and a fixed timestep code gives the network no control over how diffusion time is represented. We propose LAnoDet, in which each component answers one of the above limitations: (1) a time-embedded Transformer encoder–decoder whose attention is factorised into row and column multi-head attention (RC-MHA), reducing the cost of attention from O(N^2d) to O(N^1.5d) while preserving structured spatial dependencies; (2) a deterministic reverse process that reconstructs the clean image at every timestep with a mean-squared-error objective instead of predicting and subtracting noise; and (3) a learnable timestep embedding added to the patch and positional embeddings. Trained only on normal chest X-rays from COVIDx CXR-4, LAnoDet reaches 96.2% accuracy, 97.3% sensitivity, 96.9% F1 score and 96.51% AUC on COVID-19 detection and transfers without retraining to a pneumonia dataset with 96.78% accuracy, 98.34% sensitivity, 97.71% F1 score and 97.16% AUC, matching or exceeding supervised classifiers reported on these datasets. A sixteen-variant ablation shows that row–column attention, skip connections and the learnable time embedding each contribute, the full model exceeding its global self-attention counterpart by 4.17 AUC points.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
anomaly detection, self-supervised learning, denoising diffusion probabilistic models, vision transformer, chest X-ray, medical image analysis, attention mechanisms

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_C2J2OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper present a self-supervised based anomaly detection on chest X-rays. The method itself leverages transformer based encoder-decoder architecture that works based on row and column multi-head attention. Moreover, the method uses a learnable timestep embedding added to the patch and positional embeddings. Anomaly scores follow the convention of reconstruction based approaches. The authors evaluate their method on X-ray benchmarks including COVIDx CXR-4 (96.51% AUROC), with generalisation assessment on the Kaggle pneumonia dataset without retraining (97.16% AUROC). The authors used variant baselines including supervised classifiers supplemented by ablation studies, quantitative results and method components such as Attention mechanism and time embedding.

Critical strengths
  • Regarding the experimental results, reported performance on two different datasets, including evaluation on pneumonia without retraining, is promising

  • The ablation results are informative mainly in Table 2 on the effect of method components including the attention mechanism, the skip connections and the time embedding. Also, outperforming global attention (96.51% vs. 92.34% AUROC) while reducing computational complexity, making a strong use case.

Critical weaknesses
  • I believe there is an inconsistency in the algorithm which makes me confused. In Algorithm 1, page 15, the network is trained to predict the clean image, i.e., (f_\theta(x_t,t)\approx x_0). However, at test time for inference, this prediction is directly used as (x_{t-1}) and passed to the network for multiple steps. This is different from conventional diffusion update, which builds the next noisy state using the predicted clean image and the noise schedule, so I wonder after the first iteration, inputs may drift from the expected noise distribution. Additionally, the reported noise schedule leaves almost no original image information at T = 1,000 steps that authors reported in implementation details. Can authors clarify this?!

  • I am also unsure about the deterministic claim in the abstract, as algorithm builds teh initial state using stochastic forward diffusion, so eliminating noise injection during reverse sampling does not make the overall reconstruction deterministic. Am I right?

  • The main weakness is the experimental evidence, as it lacks many recent self-supervised baselines for chest x-ray anomaly detection. It only covers supervised classifiers while many well-known reconstruction-based or diffusion-based anomaly detectors are missing. It is also hard to judge the claim of superiority especially because the proposed components are already well-known in the field.

  • Also, some reported metrics in Table 1 just don't add up under the binary-classification definitions. For example, a sensitivity of 77.43% cannot end up with an F1 score of 90.67%, and a similar inconsistency for the pneumonia CNN baseline (83.33% sensitivity and 96.15% F1).

  • There is also ambiguity about threshold selection. In Section 3.4, it is written that the detection threshold is selected by maximizing validation F1, whereas Section 4.1 describes using only normal validation images. Authors needs to clarify these inconsistencies. I also could not find the actual runtime or memory measurements, which is another weakness of this paper.

Questions for authors
  • Please see my comments above regarding clarification of the reverse process in Algorithm 1, and Equation 9. Please clarify the exact implemented update.Also comparison w.r.t. DDIM sampler would be beneficial. Besides, how sensitive the predictions and anomaly score are w.r.t. initial forward-diffusion noise realization?

  • Please also find my above comments in critical weaknesses on metric inconsistency and evaluation protocols which need clarification. There is lack of recent diffusion-based reconstruction baselines as I described above and compared baselines are limited to supervised classifiers only.

Reviewer_QURBOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

LAnoDet detects anomalies in chest X-rays through reconstruction error, training its reconstruction model on normal images from COVIDx CXR-4. The model combines a Transformer encoder–decoder with parallel row and column attention, skip connections, learnable timestep embeddings, and direct clean-image prediction. At inference, the predicted image is repeatedly fed into the network before computing an image-level MSE score and a pixel-wise residual map. The paper reports AUCs of 96.51% on COVIDx CXR-4 and 97.16% on a pneumonia dataset without retraining the model. A sixteen-configuration ablation examines attention type, skip connections, and timestep embeddings.

Critical strengths
  1. The proposed row–column attention is a simple and reasonable design. It reduces the cost of global attention while still capturing spatial structure, which suits chest X-ray images and makes the method more practical for higher resolutions.

  2. The ablation study is fairly thorough. It examines the main design choices together rather than in isolation, which helps support the contribution of each component.

  3. The label-free setting is practically meaningful. Training only on normal images and generalizing to another dataset without retraining is valuable for medical screening, where abnormal labels are often scarce.

Critical weaknesses
  1. Limited novelty. The main components are largely adaptations of existing techniques. Row–column attention is closely related to axial and criss-cross attention, predicting the clean image instead of noise is a known diffusion parameterization, and learnable timestep embeddings and Transformer-based diffusion backbones are already common. The paper mainly combines these ideas for chest X-ray anomaly detection.

  2. Narrow scope and limited experiments. The method is evaluated only on chest X-rays, with two datasets and a binary normal-versus-abnormal setting. More importantly, Table 1 compares only against supervised classifiers and includes no other unsupervised or diffusion-based anomaly detection methods (e.g., AnoDDPM or f-AnoGAN), which are the most relevant baselines. It is therefore unclear how much the method advances anomaly detection itself, or whether it generalizes to other modalities and anomaly types.

  3. Some evaluation details are unclear. The threshold is selected by maximizing F1 on a validation set that appears to contain only normal images, which is hard to reconcile with the label-free claim. It is also unclear whether the baseline results were reproduced on the same test split or copied from the original papers.

Questions for authors
  1. Could the authors clarify how the reconstruction is obtained at inference? Since x_T is produced by forward diffusion to T = 1,000, it seems close to pure noise, which is hard to reconcile with a deterministic reconstruction of the input. More details on the initialization and update rule would be helpful.

  2. How was the decision threshold selected? The validation set appears to contain only normal images, so it is unclear how F1 was optimized without abnormal labels, and whether the same threshold was used for the pneumonia dataset.

  3. Were the supervised baselines in Table 1 evaluated under the same protocol, or were their results taken from the original papers? If feasible, a comparison with unsupervised anomaly detection methods would also be helpful, as they seem more relevant to this setting.

Reviewer_MxpxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

LAnoDet is a reconstruction based method for detecting anomalies in chest X rays (CXR) that is designed to train using only normal images. The model uses a transformer encoder decoder with parallel row column multi head attention, skip connections, and a learned timestep embeddings. During training, the network predicts the original image from a noisy version while during inference model repeatedly applies the network in a proposed deterministic reconstruction process. It uses reconstruction MSE to assign an anomaly score to each image and absolute pixel differences to locate anomalies. The paper reports AUCs of 96.51% on COVIDx CXR-4 and 97.16% on a pneumonia dataset without retraining. The model also achieves a reported 4.17 percent point AUC improvement over the global self attention variant.

Critical strengths

One of the strength of the paper seems to be the architecture design choices, done carefully by ablating through the four attention mechanism with and without skip connection along with the learned or sinusoidal timestep embeddings. The advantage of row column attention holds across the other settings. The paper also evaluates the model on a pneumonia dataset without retraining after training on COVIDx CXR-4. This also looks interesting since the fact that it examines whether the model can detect anomalies associated with a different disease. Further, 97.16% AUC suggests strong cross dataset performance particularly given that the method is designed to learn from normal images only.

Critical weaknesses
  1. In table 1, the paper reports the ResNet-50 results from Saber et al. on COVIDx CXR-4 as 96.04% accuracy, 96.70% sensitivity, 94.90% F1, and 95.77% AUC. However, Saber et al., table 2 reports the same values and uncertainties on the Cohen dataset as accuracy, precision, recall, and F1, respectively. The value presented as 95.77% AUC is actually reported as F1 on a different dataset in the cited source. Therefore, the claimed 0.74-percentage-point AUC improvement over ResNet-50 is not supported by this comparison. Can the authors explain this ?

  2. The process of selecting the threshold is also not clear. Section 3.4 states that the classification threshold is selected by maximizing validation F1, while Section 4.1 states that the validation set contains only normal images. Now, since abnormality is the positive class, a normal-only validation set cannot meaningfully optimize F1. Were abnormal labels used for threshold selection? Also, was the same threshold used for pneumonia detection on the target dataset?

  3. The paper compares mainly against supervised classifiers rather than established normal-only reconstruction or diffusion-based anomaly detectors. These comparisons do not establish an improvement over the methods the paper aims to replace. The authors should provide comparisons using the same datasets, splits, and supervision

  4. In eq. (8), the network is trained to predict the clean image (x_0). However, eq. (9) and Algorithm 1 directly use this prediction as (x_{t-1}) in the next inference step. After the first iteration, the network receives an estimated clean image while its timestep conditioning corresponds to a noisy input. This creates a mismatch between the training and inference distributions. Can you explain how this mismatch is handled?

  5. Afaik, DDIM uses the current noisy image, the predicted clean image, and the noise schedule to calculate the next state. Lanodet instead feeds the clean-image prediction directly into the network at the next timestep. Can the authors explain why this update is suitable or show that it improves reconstruction quality?

  6. The paper lacks comparisons with single step clean image prediction and a standard deterministic diffusion sampler using the same backbone, mainly to show whether the proposed reconstruction chain contributes to the improvement.

  7. On the novelty front, the paper basically combines the following ideas together: row column attention is closely related to existing axial attention(https://arxiv.org/abs/1912.12180) and criss-cross attention. U-ViT already uses skip-connected transformer architectures for diffusion models, while DDIM provides deterministic sampling.

Questions for authors
  1. Which Table 1 results were reproduced, and which were taken from prior papers? Can the authors explain the dataset and metric mismatch in Saber et al.?

  2. How were COVID negative images verified as normal? Were abnormal labels used for threshold selection? Was the same threshold used for pneumonia detection ?

  3. Does the implementation directly feed the predicted clean image as x_{t-1} or are intermediate updates missing from Equation (9)? What starting timestep and noise initialization were used? How does the method preserve patient-specific anatomy?

  4. What evidence shows that the proposed reconstruction improves anomaly detection? Were comparisons made with existing normal-only detectors, single step reconstruction, or standard deterministic diffusion sampling using the same backbone?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.