acceptodds
Under review as a conference paper at ICLR 2027

PriorDiffuDINO: Self-Trajectory Denoising for Diffusion-Based Detection Transformers

Abstract

We construct the diffusion process of a detection transformer as a denoising trajectory that starts from the noised encoder proposals and extends along the model's own predictions. We call this self-trajectory denoising and instantiate it on DINO as PriorDiffuDINO. The starting point of the trajectory does not depend on the ground truth and is shared by training and inference. Because the decoder is also trained on re-noised copies of its own predictions, which are the inputs of the later inference steps, the exposure bias between training and inference is reduced. We further adopt the matchability-aware loss and dense augmentation of DEIM, which also apply to diffusion-based detectors. With a ResNet-50 backbone and 12 training epochs, PriorDiffuDINO reaches 51.3 AP on the COCO 2017 validation set, 2.7 AP above DiffuDINO under the same schedule and close to DiffuDINO trained for 50 epochs (51.9 AP). With a ResNet-101 backbone it reaches 52.4 AP. On LVIS with ResNet-50, it substantially improves every metric over DINO and DiffuDINO, by 3.0 AP overall and 5.3 AP on the rare categories over DiffuDINO.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Object Detection, Detection Transformers, Diffusion Models, Exposure Bias, Self-Trajectory Denoising, Self-Conditioning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LRauOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the exposure bias of diffusion-based detection transformers and proposes self-trajectory denoising. The denoising trajectory starts from the top-k encoder proposals rather than random noise, and the noise is added in logit space, which bounds the perturbation. During training, each sample follows two denoising steps along the trajectory, where the second step takes a re-noised copy of the first step's prediction as input and gradients are stopped between the two steps. Ablations quantify the respective contributions of the encoder prior, dense augmentation, and the matchability-aware loss, and show that accuracy varies very little across different numbers of denoising steps.

Critical strengths

The paper identifies the exposure bias of diffusion-based detection transformers and proposes a solution: the denoising trajectory starts from encoder proposals shared by training and inference, and a second training step trains the decoder on re-noised copies of its own predictions, thereby providing an informative starting point for denoising. With a short training schedule, PriorDiffuDINO clearly surpasses DiffuDINO trained equally long and approaches its version trained several times longer. The gains appear exactly where the theoretical analysis predicts, which makes the improvements mechanistically convincing.

Critical weaknesses
  1. The appendix shows that a Gaussian start with more steps nearly matches the prior-based start, so the encoder prior may act mainly as an inference-efficiency device rather than an accuracy device. However, the paper reports no latency, throughput, or FLOPs for either training or inference.
  2. The two-step training design is not ablated. The paper does not justify why two training steps suffice, nor does it ablate the number of training steps or the maximum noise level.
  3. The matchability-aware loss and dense augmentation are borrowed from DEIM, yet DEIM itself is not included in the comparison.
Questions for authors
  1. Could you report the training time, inference latency, or FLOPs of PriorDiffuDINO, DINO, and DiffuDINO under different numbers of steps?
  2. Why does Figure 2 mark the steps as Step ≥ 2? Why using two training steps? Have you ablated the number of training steps or the maximum noise level?
  3. Could you include DEIM in the comparison?
  4. Could you provide more hyperparameter sensitivity analyses?
  5. Could you quantify the contribution of FiLM conditioning?
Reviewer_23MAOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PriorDiffuDINO, a diffusion-based object detection framework built upon DINO that addresses the training–inference mismatch in previous diffusion-based detectors, particularly DiffuDINO. The authors introduce self-trajectory denoising, which initializes the denoising process from noisy top-K encoder proposals instead of random Gaussian boxes. During training, the decoder performs two passes: the first predicts bounding boxes from noisy encoder proposals, while the second refines re-noised predictions from the first pass, with gradients stopped between passes. The method also propagates query features across passes and conditions the decoder on the noise level. Additionally, the authors incorporate matchability-aware loss and dense augmentation from DEIM. Experiments on COCO 2017 and LVIS demonstrate improvements over DINO and DiffuDINO. With a ResNet-50 backbone and 12 training epochs.

Critical strengths
  • Meaningful training–inference mismatch The paper identifies a limitation of diffusion-based detection transformers, where the decoder is trained on noisy ground-truth boxes but receives its own predictions during iterative inference. Training on self-generated predictions is a reasonable approach to addressing this exposure bias.

  • Simple and practical implementation The proposed method does not require major architectural modifications. Instead, it changes how reference points are initialized and refined while retaining the DINO architecture, making the approach relatively simple and practical.

  • Strong experimental results The method achieves strong improvements over DINO and DiffuDINO on COCO and LVIS. In particular, it improves DiffuDINO from 48.6 to 51.3 AP under the same 12-epoch schedule and shows substantial improvements on rare LVIS categories.

  • Stable performance across inference steps Unlike DiffuDINO, which experiences some performance degradation with additional denoising steps, PriorDiffuDINO maintains relatively stable performance from one to ten steps. This supports the motivation behind training on self-generated predictions.

  • Faster convergence in terms of training epochs PriorDiffuDINO achieves competitive performance in only 12 training epochs, approaching the performance of DiffuDINO trained for 50 epochs. This suggests improved convergence, although actual computational efficiency requires further evaluation.

Critical weaknesses
  • Meaningful training–inference mismatch The paper identifies a limitation of diffusion-based detection transformers, where the decoder is trained on noisy ground-truth boxes but receives its own predictions during iterative inference. Training on self-generated predictions is a reasonable approach to addressing this exposure bias.

  • Simple and practical implementation The proposed method does not require major architectural modifications. Instead, it changes how reference points are initialized and refined while retaining the DINO architecture, making the approach relatively simple and practical.

  • Strong experimental results The method achieves strong improvements over DINO and DiffuDINO on COCO and LVIS. In particular, it improves DiffuDINO from 48.6 to 51.3 AP under the same 12-epoch schedule and shows substantial improvements on rare LVIS categories.

  • Stable performance across inference steps Unlike DiffuDINO, which experiences some performance degradation with additional denoising steps, PriorDiffuDINO maintains relatively stable performance from one to ten steps. This supports the motivation behind training on self-generated predictions.

  • Faster convergence in terms of training epochs PriorDiffuDINO achieves competitive performance in only 12 training epochs, approaching the performance of DiffuDINO trained for 50 epochs. This suggests improved convergence, although actual computational efficiency requires further evaluation.

Questions for authors

Q1. Ablation of self-trajectory denoising: Could the authors provide an ablation removing the second self-trajectory training pass while keeping all other components unchanged, to isolate the contribution of the main proposed method?

Q2. Contribution of existing techniques: Could the authors evaluate DiffuDINO with matchability-aware loss and dense augmentation under the same training schedule to better distinguish the proposed improvements from existing techniques?

Q3. Dependence on encoder proposals: How does the proposed method handle objects missed by the initial encoder proposals, particularly compared with DiffuDINO, which starts from Gaussian noise?

Q4. Random-seed stability: How sensitive is the model to different random noise seeds during inference? Could the authors report the mean and standard deviation of AP across multiple seeds?

Q5. Computational efficiency: Could the authors report training GPU-hours, FLOPs, and peak GPU memory compared with DINO and DiffuDINO, considering the additional decoder pass during training?

Q6. Generalization: Can the authors test the proposed self-trajectory denoising approach generalization with other detection transformer architectures, such as Deformable DETR or Align-DETR?

Reviewer_BwyHOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends the object detection framework of DINO (Zhang et al., 2023) and DiffuDINO (Nawar et al., 2026) by defining an "Encoder Prior" for the diffusion process of the Transformer Decoder. The proposed method attempts to reduce the exposure bias by using a re-noised copy of the model's own prediction in training.

Critical strengths

The method extends previous approaches by introducing noise in the training process. The results are clearly presented and the comparison with other methods is fair. The method is fairly simple to implement with algorithms detailed in appendix.

Critical weaknesses
  1. The LLM generated "Introduction" and "Related Work" sections must be rewritten
  2. It remains unclear to what extend the exposure-bias is mitigated by prior + self-trajectory denoising
  3. From Table 1 it seems that the proposed method has no benefit over existing methods for ResNet-50 and for ResNet-101 it is unclear as comparison with best performing methods is missing (e.g. Mr. DETR). If a methods needs a handful more epochs for training is practically irrelevant and does not count as benefit to my opinion.
  4. The claim that "PriorDiffuDINO produces fewer missed and duplicate detections than DINO, and its results are close to those of DiffuDINO" does not seem to be supported by Fig. 4 (all models get the top sample right, for the second sample DINO misses one orange, while DiffuDINO andPriorDiffuDINO find all, but also have one false positive, third picture is perfectly detected by DINO and PriorDiffuDINO, but DiffuDINO has one double detection and one false positive, and the forth picture is too small for me to see count.)
Questions for authors

How does removing the trajectory adjustments change AP? Is it correct to assume that removing all 4 adjustments give the 48.6% AP stated in Table 1? It would be great to see other combinations. Especially something like: DiffuDINO -> DiffuDINO + DA -> DiffuDINO + DA + MAL -> DiffuDINO + DA + MAL + prior -> DiffuDINO + DA + MAL + prior + trajectory (yours)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.