Semi-Implicit Belief Approximation with Learned Transport and Reweighting for POMDPs

Belief estimation in POMDPs requires approximating the posterior over latent states given a history of actions and observations. In many domains, however, the observations carry no information about the state for many consecutive timesteps, and the belief's errors accumulate. Keeping these errors small requires consistency in carrying the belief forward until the next informative observation and expressiveness in correcting it in a single update when that observation arrives. Existing belief representations assume that informative observations arrive regularly: they either rely on ground-truth states during training, or lose the hypothesis that the next informative observation supports. We propose Generative Belief Variational Inference (GBVI), built on the insight that any change in a belief decomposes into transport, which moves probability mass, and reweighting, which adjusts it in place. Semi-implicit variational inference splits the belief into components that form a density without a closed form, a memory-conditioned Hamiltonian flow transports them across timesteps, and a gated recurrent memory reweights them, with both operations learned from observations alone. On Symmetric Light-Dark, occluded Atari, and Area Search, GBVI outperforms recurrent, variational, particle-based, and state-supervised baselines, and on Area Search its belief error against the exact posterior stays low as the period without informative observations grows while the baselines' error rises.