When Amortized Posteriors Become Priors: One-Shot Accuracy Does Not Certify Recursive Reuse
Abstract
A Bayesian posterior summarizes what data reveal about an unknown quantity, and it becomes the prior when more data arrive. An amortized posterior estimator outputs such a posterior in one forward pass. In sequential use it must take its own previous output as the next prior, as a language model reads its own text. On small regression tasks whose posteriors can be computed, we fix each dataset and vary how many calls it is split across. Low single-call error coexists with large recursive error, the error after repeated reuse. Training on intermediate priors reduces absolute recursive error by 43% over 16 calls for converged scalar Gaussian estimators. In logistic regression, using the network's own predictions as training priors reduces it by 66% over 32 calls, against 10% for exact intermediate posteriors. Each gain costs some single-call accuracy, in logistic regression only for short inputs. We also prove that fixed positive per-observation factors make every split agree, yet such learned factors can compose exactly while remaining far less accurate than Gaussian approximations built from the true likelihood or the reference posterior. One-shot accuracy does not certify recursive reuse, so a posterior estimator should be tested as an update rule before its predictions become priors.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.