Intrinsic Information Rates of Summaries in Neural Posterior Estimation

Neural posterior estimation (NPE) often encodes observations $Y$ into a fixed-dimensional summary $S$, with the neural posterior conditioning only on $S$. Any such summary imposes a finite information budget on inference, whereas exact preservation requires infinite information whenever posteriors vary continuously with the data. The budget typically emerges from architecture and training rather than an explicit measurement or choice. Better performance may therefore reflect better encoding or simply a larger budget. In this paper, we make the budget explicit and ask two questions: how large must it be, and how best is it spent? To answer them, we model the summary as a (possibly stochastic) channel from $Y$ to $S$. For the first question, we show that this budget, measured by the mutual information $I(Y;S)$, grows unboundedly as the posterior concentrates: in regular $d$-dimensional models, bounded posterior distortion requires $\frac{d}{2}\log n-O(1)$ nats, even when a fixed-dimensional sufficient statistic exists. More generally, the required budget is set by the distribution of the random posterior, and we bound it from above by quantizing posteriors and from below by converses that weight each posterior state by its probability. For the second question, we show that a finite budget is best spent stochastically. Under a common-support condition on the simulator, which covers the continuous observation spaces typical of simulators, a deterministic summary necessarily loses something: at every positive budget too small for exact preservation, some stochastic summary achieves _strictly_ lower distortion than every deterministic summary and every time-shared mixture of deterministic summaries within the same budget. Motivated by these results, we insert a binary bottleneck into NPE, measure each trained summary's rate, and compare summaries at matched rate, which removes the budget as a confounder in architecture comparisons. Across four simulator benchmarks and several encoder families, stochastic bottlenecks achieve favorable rate–loss tradeoffs, and architecture rankings change with the budget.