BRAG: BAYESIAN RETRIEVAL AUGMENTED GENERATION

Retrieval-augmented generation (RAG) ranks memory chunks by semantic similarity, but similarity is not what matters to the consumer of a retrieved chunk: what matters is whether the chunk actually gets used to answer the query. We introduce BRAG, a Bayesian stochastic filter that maintains, for every chunk in a RAG database, a Beta posterior over its latent utility, the probability that it will be retrieved and used. From the same stream of retrieval events, the filter also rotates each chunk's semantic keys toward the queries it actually answers, modeled as a mixture of von Mises–Fisher components fit by online EM. Utility relaxes toward a low-value, high-uncertainty equilibrium in the absence of evidence, in the tradition of Bayesian demand models from library science and rational-analysis models of human memory. The filter is closed form, stores two scalars per chunk, and costs $O(N)$ per query. In a simulation built from a 20,640-chunk corpus accumulated over nine months of daily use of a personal AI agent, BRAG raises Recall@5 on new queries targeting chunks that had been retrieved before from 0.73 to 0.97 and reduces the mean rank of the correct chunk from 12.5 to 3.8, with no significant loss on a control set of chunks that were never queried during training.