- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies outlier tokens in diffusion transformers and observes that they occur in both pretrained vision encoders and diffusion models. The authors hypothesize that these outliers reflect degraded patch-level semantics rather than merely abnormally large feature norms. Based on this hypothesis, they introduce register-based interventions for both the vision encoder and the diffusion transformer. Experiments show that these interventions improve training stability and generation quality across ImageNet generation and large-scale text-to-image generation.
Critical strengths
-
The proposed register-based interventions are simple and effective, consistently improving training stability and generation quality across different model variants and generation settings.
-
The method is evaluated on both ImageNet generation and large-scale text-to-image generation, demonstrating its applicability across different tasks and model configurations.
Critical weaknesses
-
The motivation of the proposed method is not sufficiently justified. The authors hypothesize that outlier tokens reflect degraded patch-level semantics, but the evidence provided is not sufficient to establish this relationship. In particular, the authors use the improvements brought by their register-based interventions to support this hypothesis, while these improvements could potentially arise from other mechanisms. This makes the proposed insight less convincing, as the effectiveness of the method does not necessarily validate the underlying hypothesis that motivates it.
-
With unclear motivation, the novelty appears limited. The proposed approach primarily adapts register-based interventions from prior work to diffusion models, while the underlying motivation remains insufficiently established. Consequently, it is unclear whether the proposed method offers a novel insight beyond applying an existing technique to a new setting.
Questions for authors
Could the authors provide more direct evidence that outlier tokens are responsible for the degradation of patch-level semantics? In particular, how can the authors distinguish the proposed explanation from alternative mechanisms through which register-based interventions may improve feature representations?