Structural Text Recipients Route Same-Prompt Visual Realization in Diffusion Transformers
Abstract
Attention maps in text-to-image diffusion are widely used as a semantic-importance proxy for editing, pruning, and interpretability. Yet a growing line of work on attention sinks in language models suggests that high attention mass can also reflect routing infrastructure rather than semantic importance. We test this hypothesis in multimodal diffusion transformers (MM-DiTs) through a causal characterization framework: dynamic identification of high-mass recipients per image query, pre-softmax interventions including suppression, dose scaling, no-renormalization deletion, and static redirection, and a routing-leverage decomposition linking trajectory effect to attention mass, value contrast, and downstream sensitivity. On Stable Diffusion 3 (553 prompts 4 seeds), Contrastive Language–Image Pre-training (CLIP)-fused structural text recipients recover 98.1% of the all-sink visual shift, while content-text and image-side sinks remain near the no-op floor. Matched controls, static-receptacle redirections, and no-renormalization deletion, which preserves 95.6% of the effect, rule out generic text corruption and mass redistribution. Sink-mass dose scaling yields a continuous causal axis with CLIP delta statistically indistinguishable from zero (95% CI [-0.0015, +0.0011]), and a multi-VLM semantic audit confirms 94.6–98.4% prompt-semantics preservation for the clean SD3/FLUX carriers. Scope experiments show that the routing pattern recurs in FLUX with block-dependent carrier migration and extends to U-Net cross-attention models (Stable Diffusion 1.5/2.1/XL) with stronger semantic entanglement. These results show that high attention mass in diffusion transformers is not a reliable semantic-importance signal by itself: dynamically selected structural recipients act as routing interfaces for same-prompt visual realization.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.