RhoEdit: Balancing Model Editing and Capability Preservation via Relative Curvature

Model editing aims to change a specific fact or behavior while leaving the rest of the model working as before. This is difficult because parameter updates can damage unrelated knowledge and general capabilities. We introduce RhoEdit, which formulates capability-preserving model editing as constrained optimization and derives a relative-curvature gradient adjustment from a local second-order model. The adjustment jointly uses edit and capability curvature, attenuating each gradient component more strongly along directions of greater relative capability sensitivity. It thereby retains useful editing directions while protecting the behavior represented by a capability reference set. We approximate both curvatures with Kronecker-factored approximate curvature to make the adjustment practical for LLMs. Across ZsRE, CounterFact, and WikiBigEdit, RhoEdit achieves an average editing score of 68.8 while retaining nearly all of the unedited model's general capabilities (65.1 versus 65.4). Compared with AlphaEdit, RhoEdit improves editing by 1.8 points and general capabilities by 18.5 points; compared with CrispEdit, it improves editing by 6.8 points while slightly improving general capabilities. These results show that relative curvature improves the balance between model editing and capability preservation.