Evolving Belief Memory for Continual Learning Agents

Continual learning agents operating in environments with shared latent structure may need to carry forward reusable propositions whose evidential support remains incomplete at task boundaries. Most existing non-parametric approaches preserve raw interaction histories or compress experience into persistent solutions, strategies, or memories, leaving the evolving belief status of environment-level propositions largely implicit. We introduce Evolving Belief Memory (EBM), a non-parametric continual-learning framework that couples task-directed proposition proposal and evidence-based validation with a persistent belief state. EBM represents each belief as an environment-level proposition together with its current status and associated interaction evidence. We evaluate EBM on adapted continual Database and Poker environments across three backbone models. EBM achieves the highest mean Database reward among the compared non-oracle methods across all evaluated backbones, while remaining competitive on Poker, where performance exhibits substantially greater run-to-run variability. Persistent-state replay shows that the learned belief state benefits a fresh memoryless solver even without raw cross-task interaction history. Analyses on a controlled Database stream further show that reusable beliefs accumulate across tasks and support later task solving, while ablations support the utility of persistent belief storage and evidence-based validation. These results suggest that explicitly maintaining revisable, evidence-grounded beliefs offers a useful way to represent and carry forward learning state in continual language agents.