Architecture-Dependent Information Floors for Learned Boltzmann Sampling
Abstract
Learned Boltzmann samplers often fit a tractable proposal and correct the remaining mismatch to a target by importance reweighting. This raises a basic question: before training, what sampling efficiency is attainable by a given model class? We study the architecture-dependent information floor for classes specified by conditional-independence structure. For factorised, finite-context autoregressive, and graph-Markov families, admits exact target-only expressions in terms of total correlation, conditional mutual information, and the entropy gap to an -projection. Combining these identities with asymptotic importance-sampling thresholds yields class-level lower bounds on the number of proposal draws required for reweighting. Under a uniform interface-information condition, the floor is extensive for bounded-size partitions and therefore induces exponential scaling with system size. We verify the identities against numerical optimisation and transfer-matrix calculations, compare the two-dimensional Ising measurements with Onsager's exact solution, and evaluate molecular examples in regimes where the floors are small. Finally, a pair-marginal certificate provides a data-driven lower bound without fitting a proposal. The resulting quantities separate optimisation error from structural mismatch and can be evaluated before training.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.