acceptodds
Under review as a conference paper at ICLR 2027

Beyond Reproduction: Edit-Conditioned Adaptation for Unstructured Knowledge Editing

Abstract

In unstructured knowledge editing, successfully reproducing a target document does not necessarily ensure that a model can use its individual facts to answer new queries. To narrow this reproduction–usability gap, we propose Edit-Conditioned Adaptation (ECoA), which trains lightweight adapters conditioned on the weight updates produced by an existing editor. ECoA freezes the edited base weights and trains low-rank adapters on continuation and cloze views constructed from the target document. It selects adapter layers using gradients of a document-derived training objective evaluated before and after editing, and modulates adapter rank-channel contributions using the responses of the editing updates to the current input activations. Evaluations across three language models and three unstructured knowledge editing benchmarks show consistent improvements in question answering over the corresponding base editors on UnKEBench and AKEW-CounterFact, with gains maintained in batch settings where multiple documents are edited into a single model. These results suggest that conditioning subsequent adaptation on existing editing updates provides an effective approach to improving query-conditioned use of edited knowledge.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Knowledge Editing, Unstructured Knowledge Editing, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YXxuOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+2)
Read review
Paper summary

In this paper, the authors examine the reproducibility-usability gap in unstructured knowledge editing, in which models can reproduce a document with high accuracy but are unable to answer related questions. To address the identified issue, the authors propose ECoA, a wrapper for existing base editors that applies low-rank adapters to selected layers. The empirical results across two benchmarks and three language models show that ECoA consistently improves the base editors' performance in both single-document and batched document editing.

Critical strengths
  1. The highlighted issue of the gap between factual recall and actually usable knowledge is a timely and important issue in knowledge editing.
  2. The authors motivate their paper well and explain their proposed method clearly.
  3. The empirical analysis of the proposed method covers three language models and two datasets. In addition to editing performance, the authors investigate the effect of the conditioning signal, perform component ablation, and examine the preservation of general capability.
Critical weaknesses
  1. The questions about the document are highly underspecified. The authors motivate their work by emphasizing the distinction between reproducing knowledge and using it. However, the evaluation does not sufficiently characterize the types of questions used to assess the usability of knowledge. Different question types probe different levels of knowledge use. For example, a question might simply require recalling a fact explicitly stated in the document, while another might require applying the acquired knowledge to solve a more complex reasoning task. As these different levels of knowledge use may exhibit substantially different behavior, it is important for the authors to clarify which aspects of knowledge usability their evaluation captures and align their claims about the capabilities enabled by their method accordingly.
  2. The empirical analysis evaluates the method in the setting of independently editing single documents and batched documents. The setting of editing only a single document represents a strong simplification compared to real-world settings. In reality, editing sequences of individual documents or batches of documents is the standard use case. This is especially important as sequential updates have been shown to be a real issue for knowledge editing and particularly locate-then-edit methods [1,2].
  3. The results are missing an RAG baseline. While RAG is not directly comparable to editing, it provides a practical alternative for solving the underlying problem of answering questions relating to a given document. In addition to a RAG baseline, an oracle ICL baseline (where the model answers the given questions with the corresponding document in context) would provide an informative soft upper boundary reference point.
  4. While I appreciate the authors' effort to evaluate the preservation of general capabilities, the current evaluation is focused on evaluating post-edit language understanding. To align the content of the section with the section title, the authors either need to extend their evaluation using an evaluation suite that covers general capabilities more holistically (e.g., [3]) or change the section title and the corresponding claims from general capabilities to language understanding.

[1] Gupta et al., 2024, Model Editing at Scale leads to Gradual and Catastrophic Forgetting [2] Gupta et al., 2024, Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing [3] Thede et al., 2026, CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training

Questions for authors
  1. Which type of questions are used to test the usability of knowledge? What level of "knowledge usability" does ECoA facilitate?
5 comments
AuthorsOpenReview ↗

Response to Reviewer YXxu (1/2)

We thank the reviewer for the positive assessment of the importance of the reproduction–usability gap and of ECoA’s motivation and methodology. The reviewer’s points on the clarity of the evaluation scope, the connection to real-world settings, and the need for reference baselines have helped us make the paper's claims more precise.

W1 & Q1. Types of evaluation questions and the scope of knowledge usability

As the reviewer points out, knowledge usability spans multiple levels, from direct factual recall to reasoning that applies acquired knowledge, and the current paper does not describe the scope of our evaluation clearly enough. We agree and clarify it as follows.

(1) Types of evaluation questions

Our evaluation uses the fact-specific questions provided by the three benchmarks, each of which asks about a single fact stated in the edited document in a form different from the original editing prompt. The UnKEBench questions were constructed by decomposing each document into sub-question and sub-answer pairs via knowledge decomposition [1], and the AKEW-CounterFact questions ask about the counterfactual fact contained in the document (the object of a subject–relation–object triple) [2]. For AKEW-MQuAKE, the paper uses single-hop sub-questions. For example, the document in Appendix C.7 states that Nathan Purdee managed the cultivation and harvesting of grapes, berries, and other fruits, and the evaluation question is “What crops has Nathan Purdee managed the cultivation and harvesting of?” The evaluation questions are not used for training; ECoA is trained only on continuation and cloze views constructed from the target document.

(2) Level supported by ECoA

The level directly demonstrated by our results is therefore single-hop factual use: recalling an individual fact stated in the edited document in response to a new question, without the original context. To examine levels beyond this, we also evaluated the multi-hop questions of AKEW-MQuAKE, which require composing multiple facts (Qwen2.5-7B-Instruct).

MethodROUGE-LBERTScore
UnKE+0.0420.127
UnKE+ w/ ECoA0.2620.280
AlphaEdit+0.1410.162
AlphaEdit+ w/ ECoA0.2570.278

ECoA improves ROUGE-L on multi-hop questions by 0.220 and 0.116, respectively. However, ECoA’s training views do not directly train the composition of multiple facts, and absolute performance is also lower than on fact-specific questions.

Accordingly, we will replace the term “knowledge usability” with “query-conditioned factual use” in the paper and state explicitly that our claims concern the ability to use individual facts of an edited document in new question contexts. We will describe the results on complex reasoning, together with the multi-hop analysis above, as a limitation.

W2. Sequential editing

As the reviewer points out, edits are made sequentially in real-world settings, and the fact that successive updates cause forgetting and model collapse in locate-and-edit methods [3,4] is an important consideration. We agree and have added a sequential editing experiment.

We sequentially edited 100 UnKEBench documents one at a time, updating the adapters at each step with conditioning on the cumulative update Δt=θt−θ0\Delta_t = \theta_t - \theta_0, and evaluated on all edited documents after the final step (Qwen2.5-7B-Instruct).

MethodROUGE-LBERTScore
UnKE+0.1790.224
UnKE+ w/ ECoA0.3540.296
AlphaEdit+0.1460.163
AlphaEdit+ w/ ECoA0.2730.246

In sequential editing as well, ECoA improves the ROUGE-L of both base editors by about 2×. This appears to be because ECoA does not modify the edited base weights and updates only separate adapters, so it does not accumulate additional modifications on top of the base-weight changes caused by successive updates. We will add this result to the paper.

AuthorsOpenReview ↗

Response to Reviewer YXxu (2/2)

W3. RAG and oracle ICL baselines

As the reviewer notes, RAG and oracle ICL access external documents at inference time and thus cannot be directly compared with parameter-based editing, but they serve as practical alternatives and reference points for the same problem. We therefore added both baselines. RAG prepends the top-1 document retrieved by BM25 from the 500 edited documents to the question, and oracle ICL directly provides the gold document for the question.

MethodDocument access at inferenceROUGE-LBERTScore
UnKE+✗0.2860.260
UnKE+ w/ ECoA✗0.4310.321
AlphaEdit+✗0.3610.273
AlphaEdit+ w/ ECoA✗0.4510.325
RAG✓0.6970.491
Oracle ICL✓ (gold document)0.6990.492

Methods that provide external documents at inference time outperform parameter-based editing, and oracle ICL provides a soft upper bound. This result supports the motivation of our work. The same model can answer these questions with a ROUGE-L of 0.699 when the gold document is given in context, and the base editor reproduces the document with high accuracy (AlphaEdit+ 0.956, Figure 1). Nevertheless, the edited model's question performance remains at 0.361. The bottleneck is thus neither the model's ability to answer nor the incorporation of knowledge itself, but the use of knowledge stored in the parameters in new question contexts. ECoA closes 35% (UnKE+) and 27% (AlphaEdit+) of this gap.

RAG performs nearly on par with oracle ICL on this benchmark because BM25 retrieves the gold document for most questions. Meanwhile, RAG and ICL require adding a document to the context for every question and maintaining a document store and retrieval index, whereas parameter-based editing uses knowledge inside the model without additional context. We view the two approaches as complementary methods with different trade-offs and will add this comparison to the paper.

W4. General capability preservation

We agree with the reviewer. Since the current GLUE-based evaluation focuses on language understanding, we will rename Section 4.7 to “Language Understanding Preservation” and restrict the related claims accordingly.

To broaden the evaluation, we also added the MMLU-based evaluation of UnKEBench used in prior work [1].

MethodMMLU accuracy
Pre-edit0.732
LoRA0.226
FT-L0.657
AlphaEdit0.700
MEMIT0.702
UnKE0.730
COIN0.732
UnKE+0.734
AlphaEdit+0.566
UnKE+ w/ ECoA0.731
AlphaEdit+ w/ ECoA0.728

UnKE+ w/ ECoA maintains the pre-edit level. The drop for AlphaEdit+ largely stems from output-format failures (valid response ratio 50.1%); after applying ECoA, the valid response ratio recovers to 88.9%, and the accuracy drop on questions valid for both the edited and pre-edit models decreases from 0.070 to 0.017. On the other hand, on GLUE (Figure 4), the mean accuracy of AlphaEdit+ w/ ECoA is 0.024 lower than that of the base editor, and we will report this result in the paper as well.

We will incorporate the above results and revisions into the revised paper. We thank the reviewer again for the careful review.

[1] Deng et al., Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models, ICLR 2025.

[2] Wu et al., AKEW: Assessing Knowledge Editing in the Wild, EMNLP 2024.

[3] Gupta et al., Model Editing at Scale leads to Gradual and Catastrophic Forgetting, Findings of ACL 2024.

[4] Gupta et al., Rebuilding ROME: Resolving Model Collapse during Sequential Model Editing, EMNLP 2024.

Thank you for the extensive rebuttal. The clarifications and new analyses address most of my concerns. The revision clearly explains what the evaluation measures, and the sequential experiment shows that ECoA improves factual recall over both base editors after 100 edits.

My remaining concern stems primarily from the low absolute performance of the underlying parameter-editing methods. ECoA delivers clear gains over both editors, but those gains start from a low baseline, and performance remains limited, particularly on multi-hop questions. I would appreciate more discussion of which use cases the resulting models could realistically support at the demonstrated performance level.

Reviewer_jCLROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the gap between reproducing an edited document and using its individual facts to answer questions. The proposed Edit-Conditioned Adaptation (ECoA) adds a post-edit adaptation stage to an existing knowledge editor. It trains low-rank adapters on sentence-level continuation and cloze tasks, selects adapter layers using gradients before and after editing, and modulates adapter channels using input-dependent responses to the editing weight updates. Experiments evaluate factual question answering across three models, three benchmarks, and single-document and batch editing settings.

Critical strengths
  1. The paper identifies a meaningful limitation of unstructured knowledge editing: successful document reproduction does not guarantee fact-specific question answering.
  2. The method improves both evaluated base editors across all three models on UnKEBench and AKEW-CounterFact, with gains extending to batch editing.
Critical weaknesses
  1. The motivation concerns retrieving relevant facts under new question contexts. However, the conditioning signal, ΔWh\Delta W h, measures the local effect of an editing update on the current activation. This does not directly indicate which fact a question requires, whether that fact has been successfully retrieved, or how the adapter should correct the response.

  2. Continuation and cloze tasks reasonably expose facts under different partial contexts. However, they do not directly train question-conditioned retrieval. The paper provides limited analysis of how the proposed conditioning mechanism bridges this difference

Questions for authors
  1. What information in ΔWh\Delta W h supports question-conditioned factual retrieval? Can the authors provide analysis connecting this signal to successful use of edited facts?

  2. How does ECoA compare with parameter-matched adapters gated directly by input activations, using identical training views, layer locations, and optimization budgets?

  3. Does edit conditioning improve generalization to unseen questions beyond its effect on continuation and cloze performance?

  4. How does gradient-based selection compare with selecting the first five eligible layers or using a fixed layer set across documents?

5 comments
AuthorsOpenReview ↗

Response to Reviewer jCLR (1/2)

We thank the reviewer for recognizing the gap between document reproduction and query-conditioned factual use as a meaningful problem, and for the positive assessment of ECoA’s consistent improvements across models and batch editing settings.

W1 & Q1. Relationship between the edit-response signal and query-conditioned factual use

Role of the signal. We agree with the reviewer that the edit-response code is not a signal that explicitly identifies which fact a question requires or determines whether retrieval succeeded. As stated in Section 3.3, rather than determining whether editing succeeded, the gate modulates adapter contributions using input-dependent edit responses. In ECoA, what to correct is learned by the adapters from the continuation and cloze views, while the edit-response code determines how much each adapter rank channel contributes for the current input. We will state this division of roles explicitly in Section 3.3.

Information carried by the signal. The AnyEdit-based editors we use split the target document into sequential chunks and edit the key token of each chunk, so the weight update Δd(e)\Delta_d^{(e)} accumulates key directions associated with different parts of the document. When the activation he(x)h_e(x) of a question token aligns with the keys of a particular chunk, Δd(e)he(x)\Delta_d^{(e)} h_e(x) responds along the corresponding value changes. The edit response thus reflects, at the token level, which part of the edited document the current question engages, providing partial information about which fact is relevant.

Connecting the signal to factual use. Three results show that the edit-response signal directly contributes to factual use. First, an activation-only variant that removes only Δd(e)\Delta_d^{(e)} from the same gate structure fits the training views as well as or better than Full ECoA, yet performs worse on the evaluation questions (Q3). The contribution of Δ\Delta therefore appears in factual use on new questions rather than in fitting the views. Second, replacing Δd(e)\Delta_d^{(e)} with the activation itself or with a random matrix of the same shape reduces performance below Full ECoA, and on Llama-3.1-8B, activation-only gating performs on par with the variant without conditioning (Q2). Third, feeding the trained adapters the signal of another document reduces ROUGE-L from 0.431 to 0.355 for UnKE+ and from 0.451 to 0.403 for AlphaEdit+ (Table 2), indicating that the adapters use the signal corresponding to the edited content of the current document.

W2 & Q3. Connection between continuation/cloze training and question-based factual use

As the reviewer notes, the continuation and cloze views do not directly train question-based retrieval. This is a deliberate design choice: ECoA constructs its training data solely from the target document, without using evaluation questions or questions generated by an external model (Section 3.1). The gap between training views and questions is therefore inherent to this setting, and the question is what bridges it.

Why the views transfer to questions. A fact-specific question asks for a particular fact given only partial cues (e.g., an entity and a relation), without the original document context. Continuation views train the model to recover facts from short prefixes, and cloze views from partial context on both sides, which reduces reliance on the original editing prompt and the full document context. In Figure 3, ECoA's gains appear at all positions in the document, and unlike COIN and the base editors, performance is maintained for facts in the latter part of the document. Table 6 also shows that the two view types are complementary, as removing either reduces QA performance.

Role of conditioning. The gate is computed from the activations at the edited layers, not from the format of the input. Even when a question differs in form from the training views, it can produce a similar gate pattern if it engages the same edited directions, allowing the adapters to apply the corrections learned from the views to the question as well.

AuthorsOpenReview ↗

Response to Reviewer jCLR (2/2)

W2 & Q3 (continued)

To test whether edit conditioning improves generalization to new questions beyond its effect on fitting the training views, we measured the view loss and evaluation-question performance of Full ECoA and the activation-only variant, which removes only Δd(e)\Delta_d^{(e)} from the same gate structure, on the same checkpoints.

VariantView loss ↓QA R-L
UnKE+ w/ ECoA0.1510.431
UnKE+ activation-only0.1570.394
AlphaEdit+ w/ ECoA0.1470.451
AlphaEdit+ activation-only0.1430.443

Activation-only fits the training views as well as or better than Full ECoA (UnKE+ 0.157 vs. 0.151, AlphaEdit+ 0.143 vs. 0.147), yet its ROUGE-L on the evaluation questions is lower by 0.037 and 0.008, respectively. That is, even with equal or better fit to the views, the model performs worse on new questions without the editing update. This indicates that edit conditioning contributes to transferring what is learned from the views to new questions, rather than to fitting the views themselves.

The scope of generalization we validate is the use of knowledge in fact-specific question contexts not seen during training; we do not claim generalization to complex reasoning.

Q2. Comparison with input activation-based gating

As the reviewer suggested, we trained an activation-only variant that replaces Δd(e)he(x)\Delta_d^{(e)} h_e(x) in Eq. (9) with he(x)h_e(x). Since the projection PeP_e has the same shape, the variant is parameter-matched, and the training views, adapter locations, rank, optimization steps, and λd\lambda_d are all identical. We additionally compared a variant using a random matrix RR of the same shape as Δd(e)\Delta_d^{(e)}, i.e., PeLN(Rhe(x))P_e \mathrm{LN}(R h_e(x)). Results are mean ± std over 3 seeds on UnKEBench.

Qwen2.5-7B-Instruct:

VariantUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Full ECoA0.432 ± 0.0010.321 ± 0.0010.453 ± 0.0020.325 ± 0.001
Activation-only0.394 ± 0.0010.312 ± 0.0020.437 ± 0.0100.316 ± 0.001
Random R0.393 ± 0.0020.310 ± 0.0020.436 ± 0.0030.308 ± 0.001
w/o edit conditioning0.375 ± 0.0070.295 ± 0.0020.415 ± 0.0020.306 ± 0.002

Llama-3.1-8B-Instruct:

VariantUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Full ECoA0.493 ± 0.0010.334 ± 0.0010.510 ± 0.0030.321 ± 0.001
Activation-only0.425 ± 0.0010.302 ± 0.0010.425 ± 0.0010.306 ± 0.001
w/o edit conditioning0.414 ± 0.0070.299 ± 0.0020.429 ± 0.0030.303 ± 0.002

On both models, Full ECoA achieves the highest score on all metrics. On Qwen2.5-7B-Instruct, activation-only and Random R perform nearly identically and fall below Full ECoA in ROUGE-L by 0.038–0.039 (UnKE+) and 0.016–0.017 (AlphaEdit+). On Llama-3.1-8B-Instruct, activation-only performs on par with w/o edit conditioning, whereas Full ECoA improves by +0.079 and +0.081. ECoA's gain therefore stems from the editing update rather than from the input-dependent gating structure.

Q4. Necessity of gradient-based layer selection

ECoA selects adapter layers with the score Iℓ(d)I_\ell(d) in Eq. (6), which measures how well the gradient change induced by editing aligns with the learning direction of the edited model.

Following the reviewer’s suggestion, we compared against using the five layers directly above the last edited layer for all documents (first-5 eligible). Since this uses the same layer set regardless of the document, it also corresponds to a fixed layer set.

VariantUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Gradient-based (Full ECoA)0.4310.3210.4510.325
Fixed first-5 eligible layers0.4020.3200.4350.319
Random selection0.3880.3170.4150.313

Gradient-based selection outperforms first-5 eligible layers in ROUGE-L by 0.029 (UnKE+) and 0.016 (AlphaEdit+), and random selection by 0.043 and 0.036. Although selections on Qwen2.5-7B and Llama-3.1-8B tend to concentrate on the lower eligible layers (Appendix C.2), the document-specific variation in selection leads to additional gains.

We will incorporate the above results and clarifications into the revision. We thank the reviewer again for the careful review.

Thank you for the parameter-matched ablations. They show that the edit-response signal improves QA. Could you test whether it helps at the fact level? For questions whose answers come from a known document chunk, could you keep the edited model and adapters fixed and compute the gate using either the update from the answer-bearing chunk or the update from an unrelated chunk of the same document? Does using the answer-bearing chunk improve factual answer accuracy?

Reviewer_qStgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the gap between document reproduction and query-conditioned factual use in unstructured knowledge editing. The authors first show that locate-and-edit-based unstructured knowledge editors can reproduce target documents with high ROUGE-L scores but perform poorly on fact-specific questions. They further demonstrate that simply increasing the edit strength does not close this gap. To address this problem, the authors propose Edit-Conditioned Adaptation (ECoA), which freezes the edited model and trains lightweight LoRA adapters using continuation and cloze views constructed from the target document. Adapter layers are selected using an edit-aware layer selection criterion, while the contribution of each adapter rank channel is modulated by an edit-response code derived from the editing updates and current input activations. Experiments show that ECoA improves query-conditioned factual use over both base editors on UnKEBench and AKEW-CounterFact across three LLMs, although the results on AKEW-MQu

Critical strengths
  1. Well-motivated problem and novel plug-in approach: The paper provides evidence that existing unstructured knowledge editors can reproduce target documents accurately but perform poorly on fact-specific questions, and that simply scaling the editing updates does not resolve this gap. The key idea of using editing updates to condition post-edit adaptation, rather than designing a new editor, is novel and promising. The approach requires neither modifications to the underlying editor nor evaluation queries for training, making it potentially applicable to future editing methods.

  2. Strong and scalable empirical improvements: ECoA consistently improves all evaluation metrics for both base editors across three LLMs on UnKEBench and AKEW-CounterFact, with moderate computational overhead. On UnKEBench, its ROUGE-L improvement over UnKE+ increases from approximately 1.5-fold in single-document editing to 2.0-fold when editing 500 documents jointly, where several baselines exhibit substantial performance degradation.

Critical weaknesses
  1. Insufficient validation of edit conditioning: The main contribution of edit conditioning is not sufficiently validated. Table 3 shows that additional training with the same views already provides substantial gains, while removing edit conditioning reduces ROUGE-L by 0.04–0.06. Several design choices also lack justification. The layer-selection criterion is compared only with random selection, and the restriction to layers above the edited layers and the use of LayerNorm in Eq. (9) are not adequately validated.

  2. Insufficient validation of input-dependent adapter conditioning: In Section 3.3, the proposed gating mechanism uses an edit-response code (c(x)). However, it remains unclear whether using editing updates is more effective than directly conditioning on input activations (h_e(x)). A comparison with input-dependent gating that does not use (\Delta_d^{(e)}) would help establish the contribution of edit conditioning.

Questions for authors
  1. Table 3's “w/o edit conditioning” variant uses ECoA's hyperparameters. How does it perform when tuned independently? Could the authors also compare it with gating based on input activations or a random matrix of the same shape as the editing update? These comparisons would help isolate the contribution of the editing updates.

  2. Does ECoA preserve document reproduction and locality on unrelated knowledge? How does it perform on the multi-hop questions in AKEW-MQuAKE? These evaluations would help determine whether ECoA genuinely bridges the reproduction–usability gap or improves factual use at the expense of document reproduction.

  3. Do the main ablation findings generalize across other models, datasets, and multiple random seeds? What explains the substantially smaller gains on Qwen3-14B?

3 comments
AuthorsOpenReview ↗

Response to Reviewer qStg (1/3)

We thank the reviewer for the positive assessment of our motivation and approach, in particular for recognizing the gap between document reproduction and query-conditioned factual use in unstructured knowledge editing, and the potential of performing post-edit adaptation without modifying the underlying editor.

Q1. Independent contribution of edit conditioning

(1) Contribution of edit conditioning

As the reviewer notes, additional training on document-derived views alone already yields substantial gains, and we regard it as an important component of ECoA. However, the core contribution of ECoA is not additional adapter training itself, but conditioning post-edit adaptation on the weight updates produced by an existing editor.

In Table 3, removing edit conditioning while keeping the same training views and settings reduces ROUGE-L from 0.431 to 0.372 for UnKE+ and from 0.451 to 0.415 for AlphaEdit+, a relative improvement of about 15.9% for UnKE+. In Table 2, with the trained adapters fixed, replacing the conditioning signal with that of another document (UnKE+ 0.355, AlphaEdit+ 0.403) or removing it (0.342, 0.389) both reduce performance. This indicates that the adapters use the signal corresponding to the current edit.

(2) Comparison with activation-only gating

We agree that these results alone do not distinguish whether the gain comes from the editing update or from the input-dependent gating structure itself. To separate the two, we trained an activation-only variant that replaces Δd(e)he(x)\Delta_d^{(e)} h_e(x) in Eq. (9) with he(x)h_e(x). Since the projection PeP_e has the same shape, the variant is parameter-matched, and the training views, selected layers, rank, steps, and λd\lambda_d are identical. Following the reviewer's suggestion, we also trained a variant that uses a random matrix RR of the same shape as Δd(e)\Delta_d^{(e)}, i.e., PeLN(Rhe(x))P_e \mathrm{LN}(R h_e(x)). Results are mean ± std over 3 seeds (Qwen2.5-7B-Instruct, UnKEBench).

VariantUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Full ECoA0.432 ± 0.0010.321 ± 0.0010.453 ± 0.0020.325 ± 0.001
Activation-only0.394 ± 0.0010.312 ± 0.0020.437 ± 0.0100.316 ± 0.001
Random R0.393 ± 0.0020.310 ± 0.0020.436 ± 0.0030.308 ± 0.001
w/o edit conditioning0.375 ± 0.0070.295 ± 0.0020.415 ± 0.0020.306 ± 0.002

Activation-only and Random R improve over w/o edit conditioning but perform nearly identically to each other, and both fall below Full ECoA in ROUGE-L by 0.038–0.039 for UnKE+ and 0.016–0.017 for AlphaEdit+. This is consistent with the fact that, because the gate passes through a learnable projection PeP_e, a random matrix provides little more than another linear map of the activation. Therefore, ECoA's gain cannot be explained by the input-dependent gating structure or the shape of Δ\Delta alone, and stems from the information contained in the actual editing update. This effect is even clearer on Llama-3.1-8B-Instruct (see Q3-(1)).

AuthorsOpenReview ↗

Response to Reviewer qStg (2/3)

Q1 (continued)

(3) Independent hyperparameter tuning of w/o edit conditioning

We independently searched training steps {30, 50} and learning rates {2e-4, 4e-4, 8e-4} for the w/o edit conditioning variant.

VariantStepsLRUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Full ECoA (default)304e-40.4310.3210.4510.325
Full ECoA504e-40.4400.3430.4680.344
w/o edit conditioning302e-40.2920.2830.2950.285
w/o edit conditioning502e-40.3160.2910.3420.310
w/o edit conditioning (default)304e-40.3720.2960.4150.308
w/o edit conditioning504e-40.3730.3140.4150.312
w/o edit conditioning308e-40.3670.3150.4180.316
w/o edit conditioning508e-40.3950.3220.4370.324

Even at the best setting within the search range (50 steps, 8e-4), w/o edit conditioning remains below the default Full ECoA in ROUGE-L by 0.036 (UnKE+) and 0.014 (AlphaEdit+). Under the same training budget (50 steps), Full ECoA without any learning-rate tuning outperforms the tuned w/o edit conditioning by 0.045 and 0.031 in ROUGE-L and by 0.021 and 0.020 in BERTScore. The gain from edit conditioning is therefore not explained by hyperparameter differences.

Q2. Document reproduction, locality, and multi-hop evaluation

(1) Preservation of document reproduction

ECoA freezes the edited base weights and trains only lightweight adapters, using a distillation objective with the edited model (adapters disabled) as the teacher to limit deviation from its predictive distribution. Measuring document reproduction under the original editing prompt, we find that reproduction is almost fully preserved after ECoA while QA improves substantially (Qwen2.5-7B-Instruct, UnKEBench).

MethodReproduction R-LQA R-L
UnKE+0.9760.286
UnKE+ w/ ECoA0.9670.431
AlphaEdit+0.9560.361
AlphaEdit+ w/ ECoA0.9520.451

Reproduction ROUGE-L drops by at most 0.009, whereas QA ROUGE-L improves by 0.145 and 0.090. ECoA thus improves query-conditioned factual use without sacrificing document reproduction.

(2) Locality

We performed the MMLU-based evaluation of UnKEBench used for locality assessment in prior work [1].

ConditionUnKE+AlphaEdit+
Pre-edit0.7320.732
Base editor0.7340.566
Base editor w/ ECoA0.7310.728

For UnKE+, accuracy stays at the pre-edit level with and without ECoA. The drop for AlphaEdit+ largely stems from output-format failures. To separate the two, we measured the ratio of valid answer choices and the accuracy on questions where the edited model produces a valid answer.

ConditionValid response ratioAccuracy on valid questions (pre-edit model on the same questions)Change
AlphaEdit+50.1%0.687 (0.757)−0.070
AlphaEdit+ w/ ECoA88.9%0.724 (0.741)−0.017

After applying ECoA, the valid response ratio recovers from 50.1% to 88.9%, and the accuracy drop on the same basis decreases from 0.070 to 0.017. ECoA thus mitigates both the output-format collapse and the knowledge loss caused by the base editor. On the other hand, on GLUE (Figure 4, batch editing), the mean accuracy of AlphaEdit+ w/ ECoA is 0.024 lower than that of the base editor, and we will report this difference in the paper as well.

(3) Multi-hop evaluation on AKEW-MQuAKE

The AKEW-MQuAKE experiments in the paper use fact-specific sub-questions to evaluate the use of individual facts in new query contexts. Following the reviewer's suggestion, we also evaluated the multi-hop questions.

MethodROUGE-LBERTScore
UnKE+0.0420.127
UnKE+ w/ ECoA0.2620.280
AlphaEdit+0.1410.162
AlphaEdit+ w/ ECoA0.2570.278

ECoA improves ROUGE-L on multi-hop questions by 0.220 and 0.116 for the two base editors. However, since ECoA's training views do not directly train the composition of multiple facts, our claims focus on the use of individual facts, and we will state the analysis of multi-hop reasoning as a limitation.

[1] Deng et al., Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models, ICLR 2025.

AuthorsOpenReview ↗

Response to Reviewer qStg (3/3)

Q3. Generalization of ablations and smaller gains on Qwen3-14B

(1) Generalization across models and random seeds

In Table 1, ECoA consistently improves ROUGE-L, BERTScore, and Judge scores for both base editors on UnKEBench and AKEW-CounterFact across all three models. Since the component ablations were mainly conducted on Qwen2.5-7B-Instruct, we repeated the key ablations with 3 seeds and additionally ran them on Llama-3.1-8B-Instruct. The Qwen2.5-7B-Instruct results are shown in the table in Q1-(2). Due to computational cost, the component ablations were conducted on UnKEBench; the overall effect of ECoA is consistent on AKEW-CounterFact in Table 1.

VariantUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Full ECoA0.493 ± 0.0010.334 ± 0.0010.510 ± 0.0030.321 ± 0.001
Activation-only0.425 ± 0.0010.302 ± 0.0010.425 ± 0.0010.306 ± 0.001
w/o edit conditioning0.414 ± 0.0070.299 ± 0.0020.429 ± 0.0030.303 ± 0.002

On both models, Full ECoA achieves the highest score on all metrics, with differences exceeding the standard deviation across seeds (≤ 0.010). In particular, on Llama-3.1-8B-Instruct, activation-only gating performs on par with w/o edit conditioning (ROUGE-L: UnKE+ +0.011, AlphaEdit+ −0.004), whereas Full ECoA improves by +0.079 and +0.081, respectively. This shows that ECoA's gain stems from the editing update rather than from the input-dependent gating structure.

(2) Smaller gains on Qwen3-14B

It is true that the ROUGE-L gains on Qwen3-14B are relatively small. However, in terms of factual correctness, the gains are comparable to those on the other models.

First, the ratio of Judge scores over the base editor on UnKEBench is as follows.

Qwen2.5-7BLlama-3.1-8BQwen3-14B
UnKE+ w/ ECoA1.92×1.70×1.47×
AlphaEdit+ w/ ECoA1.39×1.38×1.46×

For AlphaEdit+, the relative gain on Qwen3-14B is the largest among the three models. The BERTScore gains (+0.042, +0.041) are also comparable to those on Llama-3.1-8B (+0.061, +0.042). On Qwen3-14B, ECoA thus improves factual correctness more than lexical overlap.

Second, on Qwen3-14B, both ECoA configurations still outperform all baselines in ROUGE-L, BERTScore, and Judge on UnKEBench and AKEW-CounterFact.

Third, as possible causes, the hyperparameters were set on Qwen2.5-7B-Instruct and were not retuned for Qwen3-14B. In addition, as shown in Appendix C.2, the selected layers in Qwen3-14B are more broadly distributed, so the same k may cover fewer relevant layers. We will also state in the paper that the gains are small or inconsistent for AlphaEdit+ on AKEW-CounterFact and on AKEW-MQuAKE.

Additional clarification: layer selection and LayerNorm

(1) Restricting candidates to layers above the edited layers

The conditioning code c(x)c(x) averages the update responses of all edited layers (Eq. 10), so it is available only after the last edited layer has been computed. An adapter placed below that layer would require c(x)c(x) before it is computed in the same forward pass. The restriction in Eq. (7) is therefore a constraint imposed by the order of computation rather than a tunable design choice, and we will clarify this in Section 3.2.

(2) Validation of layer selection and LayerNorm

In addition to random selection, we compared against using the five layers directly above the last edited layer for all documents (first-5 eligible), and against a variant without LayerNorm in Eq. (9).

VariantUnKE+ R-LUnKE+ BERTAlphaEdit+ R-LAlphaEdit+ BERT
Full ECoA0.4310.3210.4510.325
First-5 eligible layers0.4020.3200.4350.319
w/o LayerNorm0.3930.3050.4380.316

Gradient-based layer selection outperforms first-5 eligible layers in ROUGE-L by 0.029 (UnKE+) and 0.016 (AlphaEdit+). Although selections on Qwen2.5-7B and Llama-3.1-8B tend to concentrate on the lower eligible layers (Appendix C.2), the document-specific variation in selection contributes to performance. Removing LayerNorm reduces ROUGE-L by 0.038 and 0.013. LayerNorm normalizes the scale of update responses, which varies widely across edited layers, editors, and documents, preventing the responses of particular layers from dominating the conditioning code or saturating the tanh gate.

We will incorporate the above results and clarifications into the revision. We thank the reviewer again for the careful review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

Dear Reviewers,

We thank all reviewers for their careful reading and constructive feedback. We have conducted additional experiments and analyses to address the concerns raised and have uploaded a revised manuscript. All changes are marked in blue. The main changes are as follows.

  • Validating the contribution of edit conditioning (Section 4.6, Table 3, Appendix C.7–C.9): We added comparisons with activation-only and random-matrix gating, results over three seeds and two models, independent hyperparameter tuning of the variant without edit conditioning, and a comparison of training-view fit against question performance. These show that the gains are not explained by additional training or by the gating structure alone.

  • Justifying design choices (Sections 3.2–3.3, Section 4.6, Table 3): We clarified why adapter candidates are restricted to layers above the edited layers and how the roles of the gate and the adapters are divided, and added comparisons with fixed first-5 eligible layers and with removing LayerNorm.

  • Clarifying the evaluation scope and extending the evaluation (Section 1, Sections 4.2–4.3, Appendix C.10, C.12–C.14): We replaced "knowledge usability" with "query-conditioned factual use" to make the scope of our claims precise, and added results on the preservation of document reproduction, multi-hop questions, sequential editing, and comparisons with RAG and oracle ICL.

  • Language understanding preservation and limitations (Section 4.7, Figure 4, Appendix A, C.11): We renamed Section 4.7 to "Language Understanding Preservation," added an MMLU evaluation, and described the limitations of our evaluation scope in Appendix A.

We hope these revisions address the reviewers' concerns and convey the contribution and empirical support of ECoA more clearly. We thank the reviewers again for their valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.