acceptodds
Under review as a conference paper at ICLR 2027

Shape the Residuals, Not the Update: Spectrally Constrained Batch Editing for Subject-Centric Knowledge

Abstract

Locate-then-edit methods dominate batch model editing, yet they are evaluated almost exclusively on subject-diverse benchmarks where edits concern unrelated entities. Real-world updates are often subject-centric: many facts about the same entity change together. On HetionetEdit, a benchmark we construct from a biomedical knowledge graph, representative methods lose roughly 30 Efficacy points. We trace the failure not to the optimized residuals, which remain near-perfect, but to their internalization into the weights. A spectral analysis shows why: soft- and hard-constraint updates are one operator with different filters, and on directions where keys are nearly indistinguishable the soft filter discards the residual while the hard filter amplifies it without bound. Same-subject edits create exactly such directions while demanding different residuals. We therefore constrain the residuals rather than the update: a single reparameterization keeps them in the subspace the keys can express and bounds their separation by that of the keys. The resulting method, Spectrally Constrained Residual Optimization (SCRO), improves the average editing score on HetionetEdit from to across four LLMs, remains highly competitive on CounterFact and ZsRE, and preserves general language understanding. Code is available at https://anonymous.4open.science/status/SCRO-ED97.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.