acceptodds
Published as a conference paper at NeurIPS 2026

Norm Anchors Make Model Edits Last

Mingda Liu, Zhenghan Zhu, Ze‘an Miao, Katsuki Fujisawa

Abstract

Sequential Locate-and-Edit (L&E) model editing can fail abruptly after many edits. We identify and formalize this failure as a positive _norm-feedback loop_, in which solved value vectors and edited MLP weights progressively amplify each other, degrading edit quality and eventually collapsing model capabilities. Our analysis shows that this feedback can yield approximately exponential norm growth under standard L&E dynamics, and can remain unresolved by existing increment-level regularizers or update clamps. We propose , a plug-in stabilizer that breaks this loop by rescaling each solved value vector to an original-model reference norm. Across multiple LLM backbones, datasets, and L&E editors, NAS extends the usable editing horizon by more than \textbf{4\times} and improves long-run editing performance by on average, while preserving single-edit efficacy, with only a **one-line modification and negligible computational overhead**. The code is available in the supplementary materials.

Related papers

See more related submissions when the market opens.