STAMP: Learning Cross-Session Memory in Native Recurrent States

Hybrid attention-recurrent language models process each session as a token sequence, using attention caches for local context while compressing part of the history into fixed-size native recurrent states. These states provide bounded memory within a sequence, but long-term memory requires information from a completed session to initialize later independent calls after the source tokens and attention key–value cache are discarded. We introduce STAMP, a learned cross-session interface that makes the backbone’s existing recurrent states persistent and reusable across independent calls. A learned write mechanism encodes future-useful information into terminal recurrent states, and a lightweight state adapter maps them into initial states for future sessions. STAMP is trained end-to-end over multi-session trajectories from a general-purpose context–question–answer corpus, but at inference it requires no per-session optimization. Experiments show that STAMP transfers without benchmark-specific retraining to LoCoMo, NextMem, PersonalMem, and LongMemEval, improving state-only recall over backbone-matched Metis while keeping the memory footprint fixed as sessions accumulate.