acceptodds
Under review as a conference paper at ICLR 2027

Structural Text Recipients Route Same-Prompt Visual Realization in Diffusion Transformers

Abstract

Attention maps in text-to-image diffusion are widely used as a semantic-importance proxy for editing, pruning, and interpretability. Yet a growing line of work on attention sinks in language models suggests that high attention mass can also reflect routing infrastructure rather than semantic importance. We test this hypothesis in multimodal diffusion transformers (MM-DiTs) through a causal characterization framework: dynamic identification of high-mass recipients per image query, pre-softmax interventions including suppression, dose scaling, no-renormalization deletion, and static redirection, and a routing-leverage decomposition linking trajectory effect to attention mass, value contrast, and downstream sensitivity. On Stable Diffusion 3 (553 prompts 4 seeds), Contrastive Language–Image Pre-training (CLIP)-fused structural text recipients recover 98.1% of the all-sink visual shift, while content-text and image-side sinks remain near the no-op floor. Matched controls, static-receptacle redirections, and no-renormalization deletion, which preserves 95.6% of the effect, rule out generic text corruption and mass redistribution. Sink-mass dose scaling yields a continuous causal axis with CLIP delta statistically indistinguishable from zero (95% CI [-0.0015, +0.0011]), and a multi-VLM semantic audit confirms 94.6–98.4% prompt-semantics preservation for the clean SD3/FLUX carriers. Scope experiments show that the routing pattern recurs in FLUX with block-dependent carrier migration and extends to U-Net cross-attention models (Stable Diffusion 1.5/2.1/XL) with stronger semantic entanglement. These results show that high attention mass in diffusion transformers is not a reliable semantic-importance signal by itself: dynamically selected structural recipients act as routing interfaces for same-prompt visual realization.

Primary area:
interpretability and explainable AI
Keywords:
Diffusion Transformers, Text-to-Image Generation, Attention Sinks, Cross-Attention, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mda5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper asks which internal attention signals affect the t2i generation in sd3/3.5 mmdits. The authors find that some highly attended padding and special-token positions have a substantial effect on the final visual semantics like shape and layouts. These findings identify some influential text positions in the tested setting, but do not solidly explain what information they carry or why their role is distinctive.

Critical strengths
  1. The authors conducted extensive analytical experiments and presented massive details
  2. The manuscript carefully acknowledges the limitations of its controls and evidences.
Critical weaknesses
  1. The experiments show that influential attention signals are concentrated at dynamically selected structural text positions, but do not explain what information these positions carry or why they are distinctive. While the authors do acknowledge this limitation, the lack of an explainable mechanism makes it difficult for the community to learn useful insights from this work.
  2. The semantic preservation is insufficiently tested under more complex prompts. Tests on, for example, GenEval2/HPSv3 prompts, are crucial for validating the findings in real-world complex needs.
  3. The verified primary evidence is concentrated on sd3 / flux.1 (older mmdits in 2024). More results on modern backbones like flux.2 klein (newer mmdit) / z-image-base are needed.
  4. The presentation is bad. For example, the main text relies heavily on tables and prose, with only two figures and little visualization.
Questions for authors
  1. Why are these positions referred to as "routing" roles rather than high-impact contextualized representations?
  2. What does Proposition 1 explain or predict beyond a general perturbation argument. I don't quite understand the point of presenting this mathematics.
Reviewer_ci8cOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how structural text recipients affect image generation in Diffusion Transformers. The authors use dynamic attention sink selection, attention suppression, and value contribution deletion to analyze the effects of different token positions, intervention strengths, and denoising steps. The results show that structural text recipients can cause large visual changes while preserving most prompt semantics.

Critical strengths

This paper studies the difference between attention weights and their actual contributions to image generation. The research problem is clear and meaningful.

The experiments are well organized.

Critical weaknesses

The main research shows overlap with prior study [1].

Structural and normal text recipients have different attention masses. The current designs cannot clearly separate the effect of the structural identity.

[1] Wu, F., & Summa, B. (2026). Attention sinks in diffusion transformers: A causal analysis.

Questions for authors

Please clarify the differences between this paper and [1].

Please provide more experiments to show the differences in attention mass.

Reviewer_QRbvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines which attention recipients drive the visual changes caused by dynamic sink suppression in Stable Diffusion 3. It identifies the most attended recipient per head during generation and compares suppressing all selected sinks with suppressing only CLIP padding and special-token positions. CLIP structural recipients account for nearly all of the average visual displacement observed when all sinks are suppressed. Removing their value contributions without renormalizing attention preserves most of the structural-suppression effect. Two vision-language models indicate that prompt-relevant content is preserved in most evaluated image pairs.

Critical strengths
  • The study keeps the prompts and starting noise the same. It uses a no-op reference that matches the actual implementation. It also includes controls for structural positions and random text, with the same number of interventions. In addition, it reports uncertainty across prompts and checks the conditional and unconditional guidance branches separately.
  • Comparing renormalized suppression with deleting a contribution helps separate two effects that are often mixed together. The dose and timing experiments also show that the size of an immediate change does not predict the final change in the image by itself.
Critical weaknesses
  • [1] already studied dynamic sinks in SD3 using score- and value-path interventions, dose and phase analyses, and the dissociation between perceptual change and prompt alignment. Identifying CLIP structural recipients is a useful refinement, but the paper does not clearly establish a substantially new mechanism beyond that prior result.

  • The controls match intervention counts but do not jointly match incoming attention mass and value geometry. The paper also does not isolate CLIP from T5 through encoder-specific interventions or examine whether their different positional representations and padding behavior contribute to sink selection. The evidence therefore identifies where high-impact contributions occur in the tested setting, but does not establish why structural positions acquire that role.

  • LPIPS measures how much two images differ, while the VLM audit primarily tests whether prompt-relevant content remains present. Despite illustrative examples, the paper does not systematically describe whether the changes concern layout, color, texture, object shape, or artifacts, nor does it evaluate their perceptual quality with human judgments. This limits the interpretation and practical significance of the proposed “visual realization” effect.

[1] Attention Sinks in Diffusion Transformers: A Causal Analysis, ICML 2026

Questions for authors
  • Beyond showing that suppressing CLIP structural recipients changes the image, what mechanistic insight or practical capability does this finding provide?
1 comment

Thank you for providing additional clarifications and experimental results. While the author provided extensive revisions:

  • Figure 5 states only 'M' setting closely reproduces the full-deletion image, while the provided qualitative results seems to be nearly identical over all settings. This will not be the qualitative support for the claim and quantitative results on Tab. 2 provided on that section.
  • While the paper provides a detailed empirical characterization of structural attention sinks, I am not fully convinced that the paper provide sufficient scientific insight or novelty beyond prior work. The current work largely extends and connects these existing findings in the context of MM-DiT models, without clearly establishing a fundamentally new mechanism.
  • The added, multiple results are not well-organized so the new scientific understanding drawn from each experiment is largely unclear.

While the authors provide extensive experimental results throughout the paper, the main claim remains hard to get based on current version of manuscript. While the experimental results shows some interesting aspects and ablations, the manuscript not yet provides the clear takeaway or answer for specific research question they would like to discover. This would require major revision on presentation and organization of the manuscript. Thus I'm keeping my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.