Functional Memory Compilation: Behavior-Level Migration of Aggregate LoRA Memory
Abstract
When a language model is upgraded, facts accumulated in its memory adapter may become inaccessible through the successor. Migration is difficult when the original interaction history is unavailable. Parameter mappings need not preserve individual answers, while response distillation requires target gradient training. We study migration from an already-trained aggregate low-rank adaptation (LoRA) memory. The source model and retained questions remain accessible, but the questions' original answers are unavailable. We propose Functional Memory Compilation (FMC), which transfers the source's observable answers rather than its parameter coordinates. FMC recovers responses at retained questions and retokenizes them for the successor. It turns selected answer positions into joint constraints on the target's output. Batched forward computation followed by one analytic low-rank fit installs a target-native output-head adapter without backpropagation or optimizer steps. Across the evaluated memory tasks, FMC improves task scores and lowers fixed-pool target installation cost over same-content distillation at the evaluated limited training budget.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.