acceptodds
Published as a conference paper at NeurIPS 2026

RB-LDC: Redundancy-Balanced Latent Coding for Robust Diffusion

Hyunseok Jeong, Jaeho Jeon, Young-Sik Kim

Abstract

Latent diffusion models (LDMs) achieve high-fidelity generation by moving diffusion from pixels to a compact latent space, but standard tokenizers are optimized for compression, not robustness under diffusion-time corruption. We formulate latent tokenization as diffusion-aware joint source–channel coding and diagnose standard LDM pipelines as missing an explicit channel-coding step. A diffusion-aware rate–robustness functional admits an explicit spectral characterization in , yielding the Redundancy-Balance Principle: optimal redundancy is not uniform, but aligns with semantic sensitivity and equalizes marginal integrated-risk reduction across active directions. The resulting optima take weighted tight-frame and water-filling forms; isotropic bottlenecks are minimax-suboptimal under heterogeneous sensitivity, and a group-conditioned extension identifies rare groups as capacity-limiting under a fixed latent budget. We instantiate the theory as RB-LDC (Redundancy-Balanced Latent-Diffusion Coding), a lightweight tokenizer modification that inserts a learned coding layer before diffusion. Six controlled experiments validate the theorem sequence up to , with a alignment gap, minimax gap, and real-image VAE learnability via Adam recovery of the KKT spectrum with relative error below 0.5 percent. On SD-VAE and VA-VAE at ImageNet-256, RB-LDC outperforms isotropic uniform redundancy in all 20 perturbed metric cells over a 15-SNR grid; VA-VAE block-local obtains average pFID and peak pFID at , while preserving clean rFID parity (gap ). RB-LDC turns latent tokenizer design into a reliability problem for diffusion models.

Related papers

See more related submissions when the market opens.