How Excess Latent Dimensionality Delays Memorization in Diffusion Models

Latent diffusion models operate on representations whose dimension can exceed the data's intrinsic dimension. Because diffusion models can reproduce individual training samples, it is important to understand whether this excess width affects memorization. We find that widening the latent space at fixed hidden width delays memorization in the tested vector-latent multilayer perceptron (MLP) score models. In controlled synthetic experiments, the memorized fraction after 5M training steps falls from $30\%$ to $0.1\%$ as the latent dimension grows from 5 to 40. This decline persists when memorization is measured only in the known signal subspace, showing that declining detector sensitivity alone cannot explain the result. On CelebA, above the width at which the variational autoencoder (VAE) preserves sample identity, the mean time to reach $5\%$ memorization grows by $3.6\times$; CIFAR-10 provides supporting evidence. A frozen-feature model gives exact learning dynamics for individual feature directions. Its spectra and a controlled intervention support an explanation based on the variance entering the nonlinearity, although connecting this mechanism to memorization requires additional alignment assumptions. Spectral timing estimates calibrated on trained runs show dataset-dependent accuracy. These results identify latent width as a factor in memorization timing for vector-latent MLPs and motivate treating latent dimensionality as a variable when evaluating memorization and privacy risk in diffusion models. The mechanism in trained networks and transfer to other architectures remain unresolved.