Editing Where It Matters Less: Capability-Aware Sequential Model Editing

Sequential model editing offers a practical way to update outdated knowledge in large language models, but many targeted edits applied over time can accumulate interference and degrade general behavior. Null-space-based editors improve stability by restricting parameter updates to directions that approximately preserve protected key–value associations, yet they typically select among the remaining writable directions with isotropic penalties. This creates a metric mismatch: admissible directions can still differ greatly in their effect on natural activations. We propose LessEdit, a capability-aware free-subspace editor for sequential model editing. At each edit, LessEdit excludes protected and previously used directions, projects the uncentered second moment of general-domain activations into the remaining free subspace, and uses the projected covariance as an anisotropic regularizer that favors low-drift updates, while keeping the reduced optimization closed form. Across 3,000 sequential edits on LLaMA-3 (8B) using ZsRE, CounterFact, and WikiBigEdit, LessEdit outperforms AlphaEdit by 13.9 points on average in LLM-as-a-judge reliability and generalization. The edited models retain stronger base capabilities, with a mean score of 60.3 compared to 46.7 for AlphaEdit. Our code is available at https://anonymous.4open.science/r/LessEdit.