Patch-Sheaf Residual Matching for Local-to-global Coherence in Latent Diffusion Models
Abstract
Modern image generators produce locally plausible content while distorting structures that extend across spatial regions, such as roads, lanes, and object boundaries. We formulate this failure as a local-to-global compatibility problem over predicted-clean latent patch states. We construct a cellular sheaf with neighbouring-patch, parent-region, and sliding square-cell relations, whose residuals quantify incompatibility between local states. Clean latents can exhibit meaningful nonzero relation residuals at boundaries and transitions, motivating the use of clean relation energies as matching targets. We therefore match predicted-clean relation energies, i.e., the squared norms of the residuals, to clean-latent targets, calibrate per-patch incident-energy distributions, and weakly preserve their spatial organization. These auxiliary losses operate only within an intermediate signal-to-noise ratio window; the (+)tail extension adds timestep-conditioned tail calibration and localization, while the generative objective remains active at all timesteps. Using a DiT-XL/2 backbone, we evaluate on DeepGlobe Roads, LSUN-Church, and BDD100K against no-sheaf controls matched for training duration and learning-rate schedule. Across five independent phase-2 training seeds, (+)tail reduces mean FID relative to staged-LR controls by (11.8%), (12.9%), and (21.4%), respectively, while consistently improving KID, CMMD, recall, and coverage.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.