RUR-Edit: Instance-Aware Residual Allocation and Utility-Aware Update Reconstruction for Knowledge Editing
Abstract
Knowledge editing aims to update specific factual knowledge in large language models while preserving unrelated knowledge. Existing multi-layer locate-then-edit methods typically distribute the remaining editing residual uniformly across candidate layers and directly apply each resulting parameter update as a whole. This coarse-grained editing overlooks heterogeneity at two levels: different layers vary in their ability to realize a target edit, while components within a layer-wise update differ in editing utility and interference with preserved knowledge. To address these limitations, we propose RUR-Edit, a fine-grained knowledge editing framework for both inter-layer and intra-update heterogeneity. At the inter-layer level, RUR-Edit introduces instance-aware residual allocation, which estimates a request-specific layer-importance profile and distributes the remaining target residual accordingly. At the intra-update level, it introduces utility-aware update reconstruction, which decomposes each layer-wise update into low-rank components and reconstructs it by balancing target-editing fidelity against interference with preserved knowledge. Experiments on CounterFact and ZsRE across LLaMA3, GPT-J, and Qwen2.5 show that RUR-Edit consistently achieves a stronger balance among editing efficacy, generalization, and specificity. On CounterFact, it achieves the highest overall editing score across all nine model–batch-size settings. On GPT-J, RUR-Edit improves the average overall score by 6.3 points over the strongest baseline while reducing editing time by 56.0%. Further analyses demonstrate the complementary effects of the two components, supporting fine-grained control over residual allocation and update composition for reliable and efficient knowledge editing.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.