acceptodds
Under review as a conference paper at ICLR 2027

Pixel Space Doesn't Save You: Two Mechanisms of Conditioning Homogenization in Diffusion Transformers

Abstract

A key motivation for pixel-space Diffusion Transformers is that deleting the VAE removes a lossy representation bottleneck. We show this fails for the conditioning pathway: the bottleneck was never in the VAE. Across 12 checkpoints and 9 architecture families, pixel- and latent-space models interleave along the homogenization axis, with conditioning vectors degenerating toward near-identical representations across instances. We trace this to two mechanisms—a frequency-schedule mismatch in pooled conditioning, and text-stream writeability in token conditioning—and derive from the first a training-free, checkpoint-level diagnostic, the norm ratio (, ), validated out-of-sample on four held-out architectures under pre-registered predictions. The same ratio predicts class-code fragility across those checkpoints (), showing functional consequences in models we did not train. We validate the second mechanism by perturbing only the prompt-specific residual, which changes the named attribute in 92–100% of trials on SD3.5 and FLUX. We then propose Conditioning Norm Balancing (CNB), a training-time regularizer that restores balanced conditioning geometry, and evaluate it under a shared-noise paired protocol (10,000 pairs per cell, McNemar's exact test, FID and accuracy from the same samples). CNB holds the ratio at parity and improves both metrics at each model's own optimal guidance, on both sides of the pixel/latent divide: top-1 on SiT-B/2 () and on PixelDiT-B, which has no VAE (), with FID improving in both cases. At lower guidance, the two metrics give opposite verdicts on identical weights. A single FID number cannot certify a conditioning pathway.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.