Learning to Be Guided: Shaping Latent Geometry for Classifier-Free Guidance

Classifier-free guidance (CFG) is a standard component of conditional diffusion models, and guided performance has become the de facto measure for state-of-the-art models. In latent diffusion, recent work designs latent spaces that are easier to model. However, under CFG, generation additionally depends on how conditions are arranged in the latent space, revealing a factor that remains largely overlooked. We study this arrangement under discrete class conditions, termed class geometry, and investigate what class geometry allows a latent space to work well with CFG. By analyzing the oracle CFG correction and class distinguishability under noise, we show that greater inter-class separation and smaller intra-class spread are favorable for CFG, and derive a noisy posterior objective whose gradient aligns with the oracle correction. Guided by the analysis, we propose **P**rototype **R**egularization (**PR**), a lightweight surrogate for the intractable noisy posterior, which applies a cosine prototype classifier to spatially pooled latents. Under a controlled training budget (80 epochs), adding PR to an LDM baseline reduces guided gFID on ImageNet from 3.31 to 2.28, surpassing a DINOv3-aligned VA-VAE (2.44) with negligible overhead. Through rich extensions, we show that PR generalizes to various label choices, data modalities, and condition signals. Together, our findings highlight class geometry as a critical factor in guided generation and take a first step toward CFG-compatible representation learning.