acceptodds
Under review as a conference paper at ICLR 2027

RhoEdit: Balancing Model Editing and Capability Preservation via Relative Curvature

11 views

Abstract

Model editing aims to change a specific fact or behavior while leaving the rest of the model working as before. This is difficult because parameter updates can damage unrelated knowledge and general capabilities. We introduce RhoEdit, which formulates capability-preserving model editing as constrained optimization and derives a relative-curvature gradient adjustment from a local second-order model. The adjustment jointly uses edit and capability curvature, attenuating each gradient component more strongly along directions of greater relative capability sensitivity. It thereby retains useful editing directions while protecting the behavior represented by a capability reference set. We approximate both curvatures with Kronecker-factored approximate curvature to make the adjustment practical for LLMs. Across ZsRE, CounterFact, and WikiBigEdit, RhoEdit achieves an average editing score of 68.8 while retaining nearly all of the unedited model's general capabilities (65.1 versus 65.4). Compared with AlphaEdit, RhoEdit improves editing by 1.8 points and general capabilities by 18.5 points; compared with CrispEdit, it improves editing by 6.8 points while slightly improving general capabilities. These results show that relative curvature improves the balance between model editing and capability preservation.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Large language model, Model edit

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_76eYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RhoEdit addresses the trade-off between model editing and capability preservation through a gradient adjustment based on capability curvature relative to edit curvature, using K-FAC to approximate both curvature matrices. Experiments cover three datasets on Llama-3-8B-Instruct, cross-model evaluation on Qwen2.5-7B-Instruct, and scaling to 10,000 ZsRE edit requests on Llama. The reported results show a competitive editing–preservation balance on the main model.

Critical strengths

1.The paper proposes an interpretable relative-curvature gradient adjustment that accounts for both editing and capability preservation. Generalized eigendecomposition clarifies the mechanism, while K-FAC enables implementation at LLM scale.

2.On Llama, RhoEdit combines strong editing with limited degradation in average capability benchmark performance across three datasets. At 10,000 ZsRE edits, it leads the reported baselines in editing performance while largely retaining average performance on the five capability benchmarks.

Critical weaknesses

1.Figure 2 only examines capability-set size and penalty strength, without isolating the contribution of edit curvature. Although capability-preserving baselines such as CrispEdit are compared, the existing experiments do not establish whether the gains arise specifically from incorporating edit curvature rather than capability curvature alone.

2.As acknowledged in Appendix B.1, the penalty–cost correspondence applies only to the local quadratic minimizer. The implementation instead uses Adam after relative-curvature adjustment, replacing Eq. (4)’s Newton preconditioner. How the local analysis translates into capability preservation under the actual optimizer updates remains insufficiently established.

Questions for authors

1.On Qwen2.5-7B-Instruct with CounterFact, RhoEdit underperforms LocFT-BF in both average editing and capability scores (Table 3). What explains this result?

2.Given the numerical instability noted in Section 3.1, how sensitive are editing performance and capability preservation to the damping applied to the edit-curvature factors?

3.Figure 2(a) examines the size of the capability reference set. What is the composition of this set, and how sensitive is capability preservation to the choice of reference data?

4.Appendix D.5 provides asymptotic complexity, but what are the measured runtime and peak GPU memory costs, including curvature estimation, eigendecomposition, and optimization?

2 comments

Thank you for your thoughtful review. We address your concerns below.


Weakness 1. Figure 2 only examines capability-set size and penalty strength, without isolating the contribution of edit curvature. Although capability-preserving baselines such as CrispEdit are compared, the existing experiments do not establish whether the gains arise specifically from incorporating edit curvature rather than capability curvature alone.

Response: Thank you for this suggestion. We conducted an additional ablation on Llama-3-8B-Instruct with 3,000 ZsRE edits, replacing edit curvature with the identity matrix. The gradient adjustment then becomes

=(I+\lambda G_{\mathrm{cap}})^{-1}g_{\mathrm{edit}}.$$ This variant retains our continuous attenuation mechanism while removing its dependence on edit curvature. We also include CrispEdit, which uses capability curvature to project gradients onto a low-curvature subspace. | Method | QA context Rel. | QA context Gen. | No context Rel. | No context Gen. | Edit Avg. | MMLU | IFEval | TruthfulQA | ARC-C | GSM8K | Base Avg. | |:--|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:| | CrispEdit | 81.9 | 70.4 | 60.4 | 51.0 | 65.9 | 69.7 | 67.8 | 53.0 | 57.3 | 77.1 | 65.0 | | RhoEdit without edit curvature | 83.17 | 66.47 | 64.23 | 51.30 | 66.29 | 69.39 | 70.05 | 52.59 | 56.48 | 77.33 | 65.17 | | RhoEdit | 86.40 | 69.60 | 65.87 | 54.27 | **69.04** | 69.43 | 73.01 | 52.51 | 56.48 | 77.94 | **65.89** | Incorporating edit curvature improves the average editing score by **2.75 points** and the average capability score by **0.72 points** over the ablated variant. RhoEdit also achieves higher editing and capability averages than CrispEdit. Together, these comparisons support the contribution of edit curvature beyond capability curvature alone: the improvement in editing is accompanied by stronger capability preservation. --- **Weakness 2. As acknowledged in Appendix B.1, the penalty–cost correspondence applies only to the local quadratic minimizer. The implementation instead uses Adam after relative-curvature adjustment, replacing Eq. (4)’s Newton preconditioner. How the local analysis translates into capability preservation under the actual optimizer updates remains insufficiently established.** **Response:** Thank you for raising this point. Our derivation separates the local quadratic solution into a relative-curvature gradient adjustment and subsequent Newton preconditioning. RhoEdit retains the former and uses Adam for the latter, since direct inverse-curvature preconditioning led to numerical issues in our preliminary experiments. The theoretical characterization of the adjustment remains valid: before entering Adam, gradient components are attenuated according to their relative capability curvature, with stronger attenuation in directions of greater relative capability sensitivity. The penalty–cost correspondence applies specifically to the local quadratic minimizer, while the preservation achieved by the complete optimization procedure is evaluated empirically. Our main experiments directly evaluate the complete method with Adam. Across the three datasets on Llama-3-8B-Instruct, RhoEdit achieves an average editing score of 68.8 while retaining a general-capability average of 65.1, compared with 65.4 for the unedited model. These results support the effectiveness of the theoretically motivated adjustment under the optimizer used in practice.

Question 1. On Qwen2.5-7B-Instruct with CounterFact, RhoEdit underperforms LocFT-BF in both average editing and capability scores (Table 3). What explains this result?

Response: Thank you for highlighting this result. LocFT-BF directly fine-tunes a localized layer, which can be particularly effective when the selected layer supports the requested edits while limiting interference with other capabilities. This may explain its strong performance on Qwen2.5-7B-Instruct.

However, its effectiveness appears highly model-dependent: its average editing scores fall from 57.3–76.7 on Qwen to 10.6–18.8 on Llama-3-8B-Instruct. RhoEdit provides more consistent performance across model families, ranking first or second in average editing performance across all six model–dataset combinations while generally retaining strong capability scores. Although LocFT-BF performs better in this particular setting, RhoEdit maintains a competitive editing–preservation balance across both models.


Question 2. Given the numerical instability noted in Section 3.1, how sensitive are editing performance and capability preservation to the damping applied to the edit-curvature factors?

Response: Thank you for raising this question. The numerical instability noted in Section 3.1 arose in our early attempts to use inverse edit curvature directly as a Newton preconditioner. With curvature estimated from a small edit dataset, inversion can strongly amplify components associated with very small eigenvalues, producing excessively large updates and preventing meaningful training. We tried damping in these preliminary experiments, but it did not sufficiently resolve the instability in the settings we tested. This motivated our choice to apply the relative-curvature gradient adjustment and let Adam convert the adjusted gradient into a parameter update.


Question 3. Figure 2(a) examines the size of the capability reference set. What is the composition of this set, and how sensitive is capability preservation to the choice of reference data?

Response: Following CrispEdit, we use Wikipedia samples as the capability set. We keep this corpus fixed and have not tried alternatives. We expect other broad pretraining corpora to provide similar preservation signals, given their shared coverage of general language and knowledge. The limited sensitivity to capability-set size in Figure 2(a) is consistent with this intuition.


Question 4. Appendix D.5 provides asymptotic complexity, but what are the measured runtime and peak GPU memory costs, including curvature estimation, eigendecomposition, and optimization?

Response: Thank you for requesting these measurements. We report the measured editing time and peak GPU memory on Llama-3-8B-Instruct with 3,000 ZsRE edits, using one NVIDIA L40 GPU. The table below reports editing time.

MethodEditing timePeak GPU memory
MEMIT16h 53m46 GB
FT8m 58s45 GB
AlphaEdit12h 30m42 GB
CrispEdit24m 9s45 GB
RhoEdit44m 47s47 GB

For RhoEdit, the stage breakdown is approximately 1h 39m for curvature estimation, 4m 5s for eigendecomposition, and 44m 47s for optimization.

Curvature estimation accounts for most of the preprocessing cost. For fixed model parameters, and reference and edit datasets, the curvature factors and decompositions can be cached and reused across penalty-strength settings, since they do not depend on the penalty strength. We will include these runtime and memory measurements in the revised manuscript.

Reviewer_JCbaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a knowledge editing method called RhoEdit, which aims to address the problem of general capability degradation in sequential editing setting. The approach falls under the direct fine-tuning-based editing paradigm; its core idea is to adjust the gradients before they are passed to the optimizer during fine-tuning, using a relative curvature mechanism derived from a local second-order approximation. This mechanism suppresses gradient directions that are likely to damage the model's general capabilities. Experimental results demonstrate that the proposed method achieves strong performance.

Critical strengths
  • The proposed method is built upon a direct fine-tuning framework, which appears to make it more adaptable to other domains beyond knowledge editing. This makes it a potentially valuable reference for the broader community.
  • The proposed framework effectively reduces the time complexity of implementation by incorporating K-FAC, keeping memory and computational overhead well under control. The efficiency makes real-world deployment feasible.
Critical weaknesses
  • A primary concern is whether the introduced constraint may conflict with other objectives in the editing task. For instance, if one wishes to optimize for additional capabilities within the current framework (e.g., multi-hop reasoning), could their optimization directions conflict with the relative curvature constraint designed to preserve general capabilities? How would the method behave under such conflicts, and is there a mechanism to regulate the trade-off between these competing objectives?
  • Under the experimental scales primarily considered in the paper, the method's advantage in preserving general capabilities appears limited, as some baselines in the main results reported in the paper already achieve rather strong performance in this regard.
  • The method appears to be sensitive to hyperparameters. For example, in Figure 2b, the penalty strength has a very pronounced impact on performance, which may lead to heavy reliance on careful hyperparameter tuning in practice.
Questions for authors
  • Could the authors comment on the robustness of their method to hyperparameter choices and provide practical guidance on how to set these hyperparameters effectively in new settings?
  • Could the introduced relative curvature constraint conflict with other optimization objectives in the editing task? How could the method handle such conflicts, and is there a way to balance the trade-off?

I would be happy to raise my score should the authors adequately address my concerns during the rebuttal.

2 comments

Thank you for your thoughtful review. We address your concerns below.


Weakness 1 and Question 2. A primary concern is whether the introduced constraint may conflict with other objectives in the editing task. For instance, if one wishes to optimize for additional capabilities within the current framework (e.g., multi-hop reasoning), could their optimization directions conflict with the relative curvature constraint designed to preserve general capabilities? How would the method behave under such conflicts, and is there a mechanism to regulate the trade-off between these competing objectives?

Response: We thank the reviewer for raising this important question. We believe it is useful to distinguish two scenarios, depending on whether multi-hop reasoning examples are included in the editing data.

(1) When the editing data contains multi-hop reasoning examples. In this case, the learning signals required for multi-hop reasoning are explicitly provided during editing. RhoEdit jointly considers edit and capability curvature, allowing directions to receive weaker attenuation when their edit curvature is sufficiently large. This supports learning the requested behavior while protecting existing capabilities. Therefore, we do not expect our method to introduce an inherent restriction on learning multi-hop reasoning patterns directly supported by the editing data.

(2) When the editing data does not contain multi-hop reasoning examples. This scenario is more subtle, as it concerns the source of compositional generalization after knowledge editing. We offer the following intuition under the assumption that the original model already possesses general multi-hop reasoning capabilities.

Suppose that standard fine-tuning on single-hop editing examples enables the model to answer multi-hop questions involving the edited knowledge, even though the fine-tuning data itself provides no explicit supervision for multi-hop reasoning. One plausible explanation is that such generalization arises from the combination of two capabilities: (i) retrieving the newly edited knowledge and (ii) applying the model's pre-existing reasoning capabilities to compose this knowledge with other relevant facts.

Under this interpretation, successful multi-hop generalization does not necessarily require learning new reasoning mechanisms during editing. Instead, it relies on the accessibility of the edited knowledge and the preservation of the model's existing reasoning capabilities. Our method aims to support both: it permits knowledge updates associated with the editing objective while protecting the model's pre-existing capabilities from interference. Consequently, we expect our method to remain compatible with this form of compositional generalization, potentially reducing the risk of degrading the reasoning capabilities on which it depends.

We acknowledge that this is a mechanistic intuition rather than a formal guarantee, as the interaction between knowledge retrieval and compositional reasoning may be more complex. Nevertheless, it provides a rationale for why the constraints introduced by our method need not impair multi-hop generalization involving edited knowledge.


Weakness 2. Under the experimental scales primarily considered in the paper, the method's advantage in preserving general capabilities appears limited, as some baselines in the main results reported in the paper already achieve rather strong performance in this regard.

Response: We agree that several baselines already preserve general capabilities well at the scale of the main experiments. The relevant question is how much editing each method achieves at that level of preservation. This motivates our use of relative curvature: by considering edit curvature alongside capability curvature, the gradient adjustment aims to retain directions that support the edit while limiting changes to existing capabilities. Table 1 provides the following comparison on Llama-3-8B-Instruct with 3,000 edits:

DatasetMethodEdit Avg.Base Avg.
ZsRECrispEdit65.965.0
ZsRERhoEdit69.065.9
CounterFactCrispEdit63.064.4
CounterFactRhoEdit65.964.4
WikiBigEditCrispEdit57.364.7
WikiBigEditUltraEdit59.365.5
WikiBigEditRhoEdit71.664.9

At this scale, RhoEdit's advantage is chiefly higher editing performance at a comparable level of measured capability preservation.

The larger ZsRE experiment in Table 2 shows a clearer preservation difference:

ZsRE, 10,000 editsEdit Avg.Base Avg.
Unedited model2.765.4
CrispEdit30.237.6
RhoEdit62.564.2

Here RhoEdit retains general capabilities close to the unedited model while achieving substantially more successful edits than the strongest tested baseline.

Weakness 3 and Question 1. The method appears to be sensitive to hyperparameters. For example, in Figure 2b, the penalty strength has a very pronounced impact on performance, which may lead to heavy reliance on careful hyperparameter tuning in practice. Could the authors comment on the robustness of their method to hyperparameter choices and provide practical guidance on how to set these hyperparameters effectively in new settings?

Response: Thank you for highlighting this concern. We have expanded the empirical analysis to clarify both the stability of capability preservation and the dependence of editing performance on λ.

The following table combines the ZsRE results in Figure 2(b) with additional evaluations on CounterFact and WikiBigEdit, using Llama-3-8B-Instruct with 3,000 edits. Edit Avg. averages the four editing metrics, and Base Avg. averages the five general-capability benchmarks.

Penalty strength λZsRE Edit Avg.ZsRE Base Avg.CounterFact Edit Avg.CounterFact Base Avg.WikiBigEdit Edit Avg.WikiBigEdit Base Avg.
0.0169.065.961.464.557.764.5
0.168.565.065.964.453.064.8
162.064.956.663.972.265.0
1051.664.955.063.655.364.9

Average capability preservation remains stable across the evaluated penalty strengths. Although λ spans three orders of magnitude, Base Avg. varies by at most 1 point on each dataset and remains close to the unedited model's average. These results support stable average capability preservation over a broad range of penalty strengths.

Editing performance exhibits greater dependence on λ. On ZsRE, the two smallest tested penalties yield similarly strong editing performance despite differing by a factor of ten. CounterFact favors an intermediate penalty, while WikiBigEdit benefits substantially from a larger penalty, with little change in its capability average. We therefore acknowledge that selecting λ matters for obtaining the best editing performance, especially on WikiBigEdit. The additional results make the scope of this sensitivity clearer: the preferred penalty is task-dependent, while measured average capability preservation is comparatively stable.

The role of λ follows from our Lagrangian formulation,

Ledit+λRcap,\mathcal{L}_{\mathrm{edit}}+\lambda\mathcal{R}_{\mathrm{cap}},

where λ controls the relative weighting of the editing and capability-preservation objectives, thereby governing the trade-off between acquiring the requested edits and retaining existing capabilities. Larger values place greater weight on capability preservation relative to editing. The resulting gradient adjustment scales each generalized coordinate by 1/(1+λρi)1/(1+\lambda\rho_i), where ρi\rho_i measures capability curvature relative to edit curvature. Increasing λ therefore strengthens attenuation and can also suppress directions needed for successful editing. Because these relative curvatures depend on the task, the appropriate penalty strength can differ across datasets.

We deliberately included λ=10 to examine the effect of a strong penalty. Across the reported results, this endpoint yields lower editing scores without higher capability averages than the respective settings used in the paper. It illustrates the cost of overly strong attenuation in these experiments. We used this broad sweep to examine both effective penalty settings and the degradation caused by an overly strong penalty. For practical tuning, we recommend the smaller candidate set below.

For a new setting, we recommend starting with the coarse logarithmic grid 0.01,0.1,1\\{0.01,0.1,1\\}. These candidates include the settings used in our main experiments on all three datasets. For fixed model parameters, and datasets, the curvature factors and generalized eigendecompositions are independent of λ and can be reused across candidates, avoiding repeated preprocessing.

We will incorporate these additional results and our recommendations for selecting λ into the revised manuscript.


We hope these clarifications address your concerns. We would be happy to provide further clarification if needed.

Reviewer_8TBgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RhoEdit, which adjusts editing gradients using the relative curvature of editing and capability preservation objectives. Experiments show an improved balance between editing performance and retention of the model’s general capabilities.

Critical strengths

The joint use of editing and capability curvature is well motivated and offers an interesting perspective on balancing editing effectiveness with the preservation of general capabilities.

Critical weaknesses

Although the paper provides a complexity analysis, it reports neither actual editing time nor runtime comparisons with existing methods, leaving its additional time overhead unclear.

Questions for authors

Could the authors clarify how the experiments with 3,000 and 10,000 edits are conducted? Are edits applied jointly, sequentially one at a time, or sequentially in batches? If batches are used, what is the editing batch size?

1 comment

Thank you for your thoughtful review. We address your concerns below.


Weakness 1. Actual editing time and runtime comparisons.

Response: Thank you for pointing out the need for a practical runtime comparison. Below, we supplement the ZsRE results from Table 1 with the actual editing time for each method on Llama-3-8B-Instruct with 3,000 edits. Due to time constraints, we report timing results only for ZsRE here. The editing and base-capability averages allow the runtime to be considered alongside performance.

MethodEdit Avg.Base Avg.Editing time
LLaMA-3-8B-Instruct2.765.4-
MEMIT0.019.916h 53m
FT20.946.98m 58s
AlphaEdit55.146.712h 30m
CrispEdit65.965.024m 9s
RhoEdit69.065.944m 47s

All reported editing times were estimated using one NVIDIA L40 GPU.


Question 1. How are the 3,000 and 10,000 edits applied?

Response: Thank you for the opportunity to clarify. The edits are applied jointly in a single training run, using the full set of 3,000 or 10,000 edit examples. As in standard fine-tuning, we shuffle the dataset and divide it into mini-batches of 32 examples. We train for up to 25 epochs, stopping early when the average edit loss is sufficiently low. Thus, the mini-batches are optimization batches within one joint editing run, rather than separately completed edits applied sequentially.


We hope these clarifications address your concerns. We would be happy to provide further clarification if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.