V-Shaped Negative ELBO in VAEs: Rate Allocation and Aggregate Dependence

We uncover a counterintuitive pattern in variational autoencoders: in a controlled benchmark, reconstruction improves as encoder mutual information increases, yet the negative ELBO first decreases and then increases. Decomposing this objective reveals that aggregate total correlation (TC), a cost arising from dependence among code coordinates, grows enough to outweigh the combined improvement in all remaining terms, including reconstruction. Explaining this reversal requires understanding both the reconstruction benefit of additional information and the accompanying dependence cost. Our geometric analysis of optimal Gaussian coding quantifies how reconstruction gains diminish. To characterize the competing cost, we increase information by reducing Gaussian noise around a fixed representation and derive an exact law governing TC growth. Within this noise family, we establish an interior global minimum of the negative ELBO, bound the minimizing noise precision, and give sufficient conditions for uniqueness. Experiments support the predicted allocation of information and TC growth, while complete objective derivatives locate minima along controlled noise interventions. Two general image benchmarks exhibit similar V-shaped responses along measured KL-rate sweeps. These findings explain how increasing dependence can reverse the objective-level benefit of better reconstruction.