RECON: Retrieval-Conditioned Norm Calibration for Lifelong Model Editing

Lifelong model editing requires a language model to absorb a growing sequence of factual updates while preserving prior knowledge and unrelated behavior. Residual-memory editing enables such scalable updates by storing each edit as a sparse residual. At scale, however, the hash-based readout of existing residual-memory editors induces two failure modes: routing fragility, because hash lookup is not invariant under paraphrase, and norm inflation, because collisions on shared columns become inevitable over long edit sequences. We present RECON, which repairs the readout with two complementary training-free mechanisms. First, semantic routing–support decoupling replaces rigid hash lookup with a gated dense semantic key while executing the sparse support verbatim, restoring robust retrieval across paraphrased queries. Second, retrieval-conditioned norm calibration rescales each retrieved dimension by a power of its relative column norm, directly counteracting collision-induced magnitude distortion to sustain locality under sequential edits. Across temporal, hallucination-correction, and sequential editing benchmarks, RECON nearly doubles out-of-distribution generalization over the hash-based readout at $T=10$, preserves locality across all evaluated horizons, and retains its advantages as edit sequences grow.