acceptodds
Under review as a conference paper at ICLR 2027

Generalizable Model Editing Could Be More Than Direct Generalization

Abstract

A central desideratum in model editing is generalization to complex tasks, which prior work formalizes under a *direct generalization* paradigm: treating and evaluating LLM behavior as an input-output mapping and expecting a factual update to alter outputs on reasoning tasks. Yet despite substantial empirical effort, the field consensus remains that such generalization is highly challenging and elusive. However, when we deploy the latest models in an in-the-wild chatbot-user setting, we surprisingly find that even classical editing methods exhibit remarkably fluent generalization behavior. This motivates us to revisit the obstacles to generalization and the limitations of the prevailing paradigm. We find that direct generalization is fundamentally constrained both theoretically and practically: theoretically, it suffers from an identifiability issue; and practically, existing approaches attempting to circumvent this are further hindered by the ambiguity of their constructed biases and by vulnerability to being overridden by other factors such as optimization dynamics. We instead identify *compositional generalization*, empowered by the increasingly strong reasoning and composition abilities of modern LLMs, as a theoretically grounded and empirically validated alternative. We validate this phenomenon across multiple models and editing methods, showing that such generalization behavior is widespread. Overall, we argue that generalizable model editing could be more than direct generalization, and call for a paradigm shift toward more generalizable and practically deployable knowledge update techniques.

Primary area:
interpretability and explainable AI
Keywords:
model editing, continual learning, large language models, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qmbGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits generalization in knowledge editing through an analysis of non-identifiability and optimization effects. It further shows that models judged to overfit under conventional evaluation can still use edited facts in multi-hop conversations, motivating a compositional view of generalization.

Critical strengths

The paper raises a useful question about whether conventional evaluations reflect how edited models are used in practice. The observations across several models and editing methods suggest that this issue deserves further attention.

Critical weaknesses
  1. The main comparison changes both the input format and the scoring method. This makes it hard to tell how much of the observed difference comes from compositional generalization. A comparison under a common evaluation criterion would make the main conclusion more convincing.
  2. The theoretical discussion explains why fitting edited facts does not guarantee generalization, but its connection to the behavior observed in the experiments is still unclear. In particular, the assumptions about the model’s composition process are not sufficiently checked. The proposed paradigm shift therefore seems stronger than the current evidence supports.
Questions for authors
  1. Would the main conclusions still hold if both settings were evaluated using the same criterion?
  2. Which assumptions in Section 4.1 are supported by the experiments? It would help to clarify whether the required knowledge and composition ability remain intact after editing.
2 comments

Thank you for your thorough and careful review. We are pleased to see that you find the core research question of this paper interesting and worthy of further attention. We hope the following response will clarify your concerns about the paper.


W1 / Q1 - Changes in Input Format and Evaluation Paradigm

Thank you for raising this concern. We first want to point out that the argument of this paper does not require the two sets of results to be numerically comparable. Our core contribution is to argue a position: the prevailing direct-generalization setting and its evaluation protocol—which naturally encompasses many elements such as the input template and scoring method—is conceptually unsound and fails to reflect the model's actual generalization capability in practice. The old protocol evaluates generalization via teacher-forced scoring on plain-text fixed prefixes, judging generalization by the probability of answer tokens without ever asking the model to actually generate a response; real users do not interact with edited models this way. Our setting, by contrast, simulates how real users query models in conversation, allows the model to generate freely, and then evaluates whether the generated response is correct. Under this new setting, the scoring method naturally needs to change. The key insight is that while the old protocol yields a consensus that edited models cannot generalize, in-the-wild generation demonstrates that they actually can. This divergence highlights the inherent limitation of the old protocol, and proving this point does not require numerical comparability between the two sets of scores.

W2 / Q2 - The Connection Between Theory and Experimental Behavior, and Whether the Evidence Is Sufficient to Warrant a Call for a Paradigm Shift

Thank you for raising this question. We will more clearly articulate in the paper the connection between the theoretical discussion and the experiments. This paper does not directly infer "models can exhibit compositional generalization in the in-the-wild setting" from "fitting edited facts does not guarantee generalization." Instead, we advance three interrelated claims. First, the old paradigm expects local fact-fitting to directly produce the target mapping on unseen compositional queries; however, unidentifiability renders this research objective theoretically under-constrained and thus difficult to achieve. Second, compositional generalization offers a theoretically grounded alternative path: newly edited facts can interact with the model's existing knowledge and compositional capabilities, making the desired generalization feasible under certain conditions—conditions that the old paradigm's mapping-based evaluation happens to preclude. Third, once we adopt a new setting that allows the model to more fully leverage its compositional capabilities, we do indeed observe this generalization behavior.

In the specific experimental setting, for a multi-hop query (s, r1, o1, r2, o2): the completeness of the first hop (s, r1, o1) is verified by the efficacy test; the compositional capability is evidenced by the edited model's ability to actually complete the reasoning chain; and the output pattern analysis in RQ4 further demonstrates that the model retains the ability to reason via chain-of-thought after editing. As for whether the second hop (o1, r2, o2) remains intact after editing, we conducted a supplementary experiment evaluating the second-hop factuality of Qwen3.5 before and after ROME and MEMIT editing. The results confirm that the second-hop knowledge does indeed remain intact:

ModelQwen3.5 (Unedited)ROMEMEMIT
Second-Hop Factuality90.190.690.6

That said, we do not deny that the precise internal mechanism underlying the model's compositional generalization has not been—and is indeed difficult to—deeply explained. Research on compositional generalization in LLMs is itself a significant field in its own right, still at a very early stage, with the entire community remaining highly curious about the underlying mechanisms. Resolving a central open problem in another field is clearly beyond the scope of a position paper in the knowledge editing domain. Nevertheless, we emphasize that this does not undermine the support that the theoretical reasoning and the observed experimental phenomena provide for our claims. We already know that the old paradigm is theoretically difficult to realize and has stalled the editing field without substantive progress for years. Given this, why should compositional generalization—a theoretically grounded and empirically observed direction of generalization—be required to fully explain its internal mechanisms before it is worth calling for a paradigm shift? The insufficiency of mechanistic understanding should be one of the open problems that this new agenda calls upon the community to address; it is not a prerequisite that must be resolved before the agenda itself can be proposed.

Reviewer_MHo2OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The generalization ability of edited LLMs is often reported to be poor under conventional evaluation protocols. The authors argue that direct generalization, measured through structured completion of downstream queries, does not fully capture a model's ability to use edited knowledge in natural-language interactions. As an alternative, they claim that the community should focus on compositional generalization, showing that LLMs can compose the edited fact with other knowledge to answer natural-language questions.

Critical strengths
  • The authors focus on a standard evaluation protocol for generalization, highlighting that it is not well-posed, as it measures the ability to predict a target answer under a fixed completion protocol, which is not equivalent to the ability to use an edited fact in a natural conversation. This is an interesting topic

  • The experimental discussion is well organized. I particularly appreciated the specific research-question paragraphs. Moreover, the experimental evaluation considers instruction-tuned assistants, with their default chat templates and without additional reasoning instructions. This is a convincing and relevant evaluation setting (Section 4.2, lines 399–406, and Appendix C.3, lines 910–928). Measuring the phenomenon quantitatively increases its credibility (Tables 2 and 5).

  • I evaluate positively the way in which the LLM-judge is used, as its automated scoring is supported by a correlation analysis with human judgement (Appendix E.2 and Table 6 compare ratings on the first 100 outputs, reporting a Pearson correlation of 0.884 with human assessment).

Critical weaknesses
  • The technical contribution provided by the paper is thin. From a theoretical perspective, it primarily brings together existing results rather than developing substantially new theoretical insights. From an experimental point of view, the analysis is not extensive, with a single benchmark and a relatively small set of editing methods (4) and model families (4).

  • The paper puts together several theoretical results, each with its own assumptions, without discussing applicability in the investigated setting. The underdetermination perspective of Sec.3.1 and Reizinger et al. (2024) (https://proceedings.mlr.press/v235/reizinger24a.html) is compatible with learned compositional priors. Wiedemer et al. (2023, Theorem 1 and Sec.5) (https://proceedings.neurips.cc/paper_files/paper/2023/file/15f6a10899f557ce53fe39939af6f930-Paper-Conference.pdf) motivates composition under structural and support conditions. However, neither establishes the assertion that chat completion yields compositional generalization (lines 360–367). That assertion should be presented as a hypothesis.

  • The literature is broader than the immediate-completion characterization in the introduction (lines 80–100) and Sec.4.1. Zhong et al. (2023, Sec.4.1 and Table 3) (https://aclanthology.org/2023.emnlp-main.971.pdf) evaluates ROME and MEMIT using generated multi-hop answers, including chain-of-thought (CoT). Hua et al. (2024, Table 4 and Sec.5) (https://aclanthology.org/2024.findings-acl.743.pdf) evaluate direct and CoT prompting and analyze fact recall and reasoning coherence.

  • in several points the sentence formulation is confusing (e.g. lines 59-60). Moreover some notation is used before being defined (e.g. lines 46-49)

Questions for authors

Would it be possible to provide a more in-depth discussion of the causal relationships underlying the observations? Successful multi-hop answers after editing do not necessarily demonstrate that the model composes the new fact with existing knowledge in the proposed manner. Is it possible to design controlled counterfactual tests to isolate the contribution of the edited fact?

Does compositional generalization persist on independent benchmarks and longer reasoning chains?

2 comments

We thank the reviewer for the careful reading and for the recognition of our work. We are glad that the central point came through clearly: the standard generalization protocol measures whether a model can predict a target answer under a fixed completion format, which is not equivalent to whether it can use an edited fact in natural conversation. This distinction is the main motivation of the paper, and we appreciate that the reviewer found it interesting. We also appreciate the reviewer's recognition of our experimental design: the research-question structure, the evaluation of instruction-tuned assistants with their default chat templates and no additional reasoning instructions, and the quantitative measurement of the phenomenon in Tables 2 and 5.

The reviewer's concerns concern the scope of the contribution, how the theory is framed, how we position the prior literature, and the causal interpretation of the results. We address each of them in detail below, with additional experiments where they were requested.


W1 – Whether the paper proposes new theory, and the scope of experiments

With the utmost respect, we firmly disagree with this assessment. This paper is, by design, a position paper; its purpose is to surface and rigorously argue for a problem that the community has largely overlooked. The absence of a fundamentally new theory cannot, in itself, constitute a weakness of such a paper. We believe that articulating these insights is crucial to progress in knowledge editing as a field: if we have not yet established whether the generalization target we pursue is well-posed, there is little value in continuing to chase technical novelty, or at times mere complexity, around that target.

Regarding the experimental scope, we would also push back on the characterization of it as insufficient. Our study covers all mainstream parameter-modifying paradigms, evaluated across four model families — a breadth that is, to our knowledge, quite rare in this subfield. For a position paper whose primary contribution lies in reframing an existing problem, we believe this constitutes thorough empirical grounding. That said, reliance on a single data source (EVOKE) is a fair criticism, and we have supplemented our experiments with MQuAKE (see our response to Q2) to verify that the phenomenon holds under a different data distribution.


W2 – Existing theory does not prove that "conversational completion yields compositional generalization"; should this be treated as a hypothesis only?

Thank you for raising this point. It was indeed our intent to frame this as a hypothesis, and we have revised the presentation to prevent any misreading. At the same time, we would like to take this opportunity to clarify the specific role that the theoretical component plays in supporting our claims.

The theoretical sections of this paper serve two distinct functions. First, the non-identifiability argument explains why the old protocol provides no guarantee: OOD generalization must be supplied by some form of inductive bias, and the editing objective alone cannot guarantee it. Second, the compositional generalization theory points to a viable pathway under known conditions: because it supports the view that novel compositions are not fully OOD given the right prior, the LLM's own compositional ability can serve precisely as such an internal bias. The old evaluation protocol, however, never gives the model the opportunity to invoke that ability — and therefore cannot observe the generalization it enables. The theoretical component does not, and does not claim to, establish that generalization will occur under conversational conditions; that empirical question is addressed by our experiments, whose results are largely consistent with the theoretical prediction. We will further revise the presentation to make this distinction clearer.

W3 – Some prior work has studied generative multi-hop and chain-of-thought settings; how does this paper position itself relative to those?

This is a fair concern, and we will add a more explicit discussion of related work and the distinctions involved. That said, we wish to clarify that having studied a generative setting and the contributions of this paper are still substantially different.

First, to our knowledge, the central finding of this paper represents a genuinely new observation for the field as a whole. MQuAKE and ReCoE did adopt generate-then-evaluate metrics on older models such as GPT-J and Tülu, yet those studies did not arrive at conclusions analogous to ours. We attribute this, at least in part, to the fact that those earlier models lacked the compositional capabilities of modern systems — as is illustrated by the GPT-J outputs we include in our response to Reviewer 5w1p: even in a generative setting, those models fail in ways that make the phenomenon invisible. As far as we are aware, no prior work has identified and foregrounded the generalization phenomenon we report.

Second, our position retains importance for the field. Despite the existence of generative benchmarks, direct-generalization metrics remain the primary standard for judging whether edited knowledge transfers, and this is precisely what we challenge. We observe, in modern assistants used under their default conversational conditions, a capability that those metrics systematically fail to capture. Our argument is therefore that generalization should be understood as more than direct generalization. We hope this also addresses the broader concern raised.


W4 – Several expressions are unclear

Thanks! We have now fixed the ones you mentioned and several others we caught, and we will definitely give it another polish should our work reach camera-ready.


Q1 – Does multi-hop success genuinely stem from the edited fact? Could a controlled counterfactual test be added?

We believe the controlled counterfactual test the reviewer is requesting is, in fact, already built into our experimental design — though perhaps our description made this less apparent, so please let us know if we have misunderstood the concern. All samples in our study are counterfactual by construction, meaning an unedited model cannot answer them correctly by chance. To further guard against confounds, we do not report post-edit score in isolation; we evaluate and score the same questions on the pre-edit model, targeting the new, counterfactual multi-hop answers. Unedited models consistently fail to produce these answers, while post-edit models show substantial improvement, ruling out the explanation that the model was already predisposed toward the new answers.


Q2 – Can compositional generalization be observed on an independent benchmark and with longer reasoning chains?

This is a reasonable question. We have reproduced the post-edit compositional generalization phenomenon on an independent MQuAKE subset, covering both two-hop and three-hop reasoning chains. The supplementary experiments use Qwen 3.5-9B on MQuAKE-CF: after modifying (s, r, o), we query multi-hop questions of the form (s, r₁, r₂, r₃), with an editing setup fully aligned with EVOKE. We also include a canonical in-context baseline for reference, which typically ranks among the strongest classical baselines under the direct-generalization setting.

Method2-Hop3-Hop
Qwen 3.5 Unedited0.40.8
IKE29.232.0
ROME69.255.0
MEMIT46.040.0

As expected, factuality score decreases with chain length, but the three-hop numbers remain non-trivial. The multi-hop usage phenomenon beyond the old protocol therefore does not arise only in the EVOKE data source, nor is it confined to the two-hop setting. We would also emphasize that exhaustively testing every chain length is not a prerequisite for advancing compositional generalization as a research direction. If two-hop results already reveal a capability that the old protocol misses, the appropriate next step is to study how to extend and exploit it; not to dismiss the phenomenon because longer chains have not yet been fully characterized.

Reviewer_5w1pOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether standard model-editing tests miss what edited models can do in conversation. The authors test four editing methods on four models using EVOKE. With standard chat templates, the models can use edited facts to answer multi-hop questions, even when the original metrics show overfitting. The authors argue that the models' existing reasoning skills help them combine new facts with old knowledge. They also show that changing MEMIT's optimization settings can reduce overfitting.

Critical strengths
  1. The paper raises a useful concern about what model-editing benchmarks actually measure. Testing generated answers in the model's normal chat format is relevant to how these models are used. The results show that poor performance on the original overfitting metrics can coexist with successful use of edited knowledge. For example, ROME on Qwen3.5 achieves a mean factuality score of 65.8/100, compared with 6.4 for the unedited model. This observation is worth investigating.

  2. The MEMIT experiments give a useful practical result. In Figure 2, the MEMIT variant with a lower learning rate reduces all four overfitting rates on Llama-3.1-8B while keeping edit success at 100%. The optimized hidden states also have norms closer to those of the unedited model. These results show that basic optimization settings can make a substantial difference.

Critical weaknesses
  1. The main comparison changes both the input format and the scoring method. The original evaluation uses answer probabilities, while the new evaluation uses an LLM judge to score a full response. These scores are not directly comparable. Table 2 shows that edited models can answer some questions in chat, but it does not isolate the effects of chat formatting, intermediate reasoning, and scoring. Testing both formats with the same answer-correctness metric would make the comparison clearer. Locality is also not tested in chat, so we do not know how often the edited models give wrong answers to unrelated questions in this setting.

  2. Section 4.1 argues that edited facts can support multi-hop reasoning if the other required knowledge and the ability to combine facts remain intact. The paper reports locality results, but it does not directly check whether the second-hop knowledge and composition ability are preserved. It also does not isolate whether plain-text inputs suppress this ability. The examples show that edited models can combine facts in chat, but they do not establish why chat works better. The proposed explanation is plausible, but it needs more direct evidence.

Questions for authors
  1. Can you compare plain-text and chat inputs on the same multi-hop questions, using free generation and the same answer-correctness metric? In chat, does asking for only the final answer change the results? How well does the edited model preserve unrelated knowledge under the same chat setting?

  2. Can you check whether the model recalls both the edited fact and the second-hop fact correctly before and after editing? After editing, how often does it answer the multi-hop question correctly when it knows both facts? What evidence shows that plain-text inputs suppress the ability to combine facts?

3 comments

Thank you for the careful review and for recognizing the value of this work. The reviewer notes that the paper raises a useful concern about what model-editing benchmarks actually measure. This is the main motivation of our work, and we are glad it came through. We address each of the reviewer's concerns in turn below.

First, we would like to offer a brief clarification, as we suspect it is the source of the main misunderstanding: the key factor in this paper is not the chat template itself. Our argument is that the prevailing paradigm treats the model as an input-output mapping and evaluates it as one. Under this view, the edit objective cannot determine OOD behavior, so generalization has no theoretical guarantee, and in practice it is indeed not observed. A model's own compositional ability, however, theoretically allows the desired generalization, and when the evaluation setting lets the model invoke it, generalization does appear. So the core distinction is likely not plain text vs. chat, but whether the evaluation lets the model use its compositional ability.


W1 / Q1 - Input format and scoring change together, so their effects are not isolated

Thank you for raising this concern. We first want to point out that the argument of this paper does not require the two sets of results to be numerically comparable. Our core contribution is to argue a position: the prevailing direct-generalization setting and its evaluation protocol—which naturally encompasses many elements such as the input template and scoring method—is conceptually unsound and fails to reflect the model's actual generalization capability in practice. The old protocol evaluates generalization via teacher-forced scoring on plain-text fixed prefixes, judging generalization by the probability of answer tokens without ever asking the model to actually generate a response; real users do not interact with edited models this way. Our setting, by contrast, simulates how real users query models in conversation, allows the model to generate freely, and then evaluates whether the generated response is correct. Under this new setting, the scoring method naturally needs to change. The key insight is that while the old protocol yields a consensus that edited models cannot generalize, in-the-wild generation demonstrates that they actually can. This divergence highlights the inherent limitation of the old protocol, and proving this point does not require numerical comparability between the two sets of scores.

With this in mind, while we do not regard the absence of such fine-grained analysis as a major weakness of the paper, we agree that a more granular ablation to disentangle which factors have a greater influence on the model's ability to invoke its compositional capability would be a valuable additional contribution. We therefore provide factuality scores on a subset of EVOKE multi-hop questions where the edited model is prompted with plain-text input (no chat template) and asked to continue the text freely. ROME and MEMIT still show post-edit generalization:

MethodPlain TextChat Template
Qwen 3.5 Unedited7.86.4
ROME51.463.4
MEMIT57.868.4

We further analyzed the actual model outputs and found that Qwen3.5, as a model trained extensively with agentic RL, spontaneously produces chain-of-thought-like reasoning even under plain-text input. We also compared GPT-J, a model from an earlier generation commonly used in the knowledge-editing literature that has not undergone any SFT or RL. The following shows their free-generation outputs on EVOKE case #1—after editing the knowledge Spike Hughes originates from Philadelphia—in response to the plain-text prompt What famous food is associated with the city where Spike Hughes originates from?:

GPT-J:

Spike Hughes is a fictional character in the British television series Buffy the
Vampire Slayer. He is portrayed by actor Nicholas Brendon.
Spike is a vampire who is the leader of the vampire gang known as the "Slayers".
He is the only character in the series to be a recurring character.
Spike is a character who is often associated with the food he eats. He is a
vampire who is a fan of the food of the human world, and he often

Qwen 3.5-9B:

Okay, the user is asking about a famous food associated with the city where Spike
Hughes originates from. Let me start by recalling who Spike Hughes is. He's a
well-known jazz saxophonist, right? I remember he was born in Philadelphia. So
the city in question is Philadelphia.
Now, what's the famous food from Philadelphia? The most iconic one is the
cheesesteak. But I should make sure there aren't other contenders. Maybe the
Philly cheesesteak is the primary

Under the free-generation setting, GPT-J produces fluent but entirely irrelevant text, whereas Qwen spontaneously generates CoT-style reasoning. This further supports our emphasis that it is not the chat template per se that matters, but rather the model's compositional ability (and whether the evaluation allows it to be invoked). By contrast, despite this striking difference in behavior, evaluations conducted under the old setting with newer base models have not yielded meaningful or consistent conclusions—they still broadly conclude that generalization after editing remains poor. This is largely an artifact of the flawed evaluation setting, and it has substantially hindered progress in the field.

Regarding locality of edited models under the chat setting: while this is somewhat out of scope for the paper's main focus, it is nonetheless a reasonable concern. We evaluated several locality tasks using the standard factuality metric described in the paper:

MethodSubject Specificity⇡Relational Specificity⇡Prefix Distraction⇡
Qwen 3.5 Unedited57.666.270.8
ROME29.861.665.8
MEMIT29.662.264.6

Overall, locality in the chat setting is indeed somewhat lower than what probability-based metrics suggest, because probability-based metrics are in practice quite permissive and do not necessarily reflect actual generation behavior. That said, the general trends are consistent: for example, the subject-overfitting issue known for ROME and MEMIT is also apparent in the more pronounced drop in Subject Specificity under the chat setting. This further underscores the need for more practically grounded evaluation metrics in the field.

W2 / Q2 - The second-hop knowledge is not directly checked, and it is not shown that plain text suppresses it

Thank you for raising this point. The paper indeed does not provide direct verification that the models have retained second-hop knowledge prior to editing. To address this, we have supplemented our experiments by constructing the second-hop knowledge as standalone questions and evaluating four unedited LLMs using the factuality metric. The results are as follows:

ModelQwen3.5 (Unedited)ROMEMEMIT
Second-Hop Factuality90.190.690.6

Overall, except for the small 0.5B-scale model, these models are largely able to accurately recall second-hop knowledge, and a large proportion of them exhibit compositional capability after editing—further indicating that the compositional ability of the edited models remains intact.

At the same time, we wish to reiterate that the problem should not be reduced simply to "plain text is bad, chat template is good." What truly needs to be reconsidered is the old protocol as a whole, which evaluates LLMs as input-output mappings. This encompasses the use of plain-text fixed prefixes, teacher-forced answer probabilities, and the absence of real free generation, among other issues. Such a protocol clearly cannot capture many behaviors that emerge during actual generation, and it is theoretically very hard to observe generalization in this setting. We will make this point more explicit in the revision, so as not to give the impression that all the shortcomings of the complete protocol are attributed solely to the input format.

If we have misread your concern, or if there are specific results you believe would further change your assessment of the paper, please let us know and we will respond faithfully.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.