Growing Structured Memory with Learned Local Decisions for Online Ingestion

In online memory construction, a corpus arrives incrementally and each new item must be ingested upon arrival. Explicit structure over items, such as knowledge graphs, can substantially improve retrieval, but growing and maintaining such structure under incremental arrival remains a challenge. Recent methods address this agentically, prompting an off-the-shelf LLM to perform each ingestion; this yields flexible updates, but the cost of these calls is high and grows with corpus size. We instead grow a structured memory through learned local decisions, coupling a hierarchy over the corpus, here a tree, with a model trained to maintain it and a retrieval procedure that exploits it. We cast ingestion as a local maintenance problem: each decision depends only on a bounded region of the tree, not on the memory as a whole. A 0.5B model, supervised-finetuned on decisions labeled offline by a larger model, is enough to make these decisions. Retrieval in turn draws on the hierarchy as a signal beyond flat similarity, recovering the benefit of structure without any LLM calls. We evaluate on standard benchmarks under both offline and online arrival, where it matches state-of-the-art memory systems in R@10 at a fraction of their ingestion cost. On CloneMem, RealMem and NovelQA it reaches R@10 of 35.70, 71.46 and 60.67, against 32.76, 67.86 and 57.56 for the strongest competing system, while spending less than a tenth of the hosted-LLM input tokens of the cheapest competitor and building its memory at least eight times faster.