- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AlignEdit studies knowledge editing in post-trained language models, where successful edits should remain effective under chat formatting and open-ended generation. The paper introduces CHATEVAL, which applies model-specific chat templates and uses an LLM judge to assess the correctness of generated answers for Reliability and Generality. Locality is measured through agreement between the original and edited models’ top-1 predictions on unrelated prompts.
The method constructs each teacher independently from the original model using localized fine-tuning, then transfers its behavior to a shared student. The student minimizes token-level reverse KL to the teacher along student-generated trajectories, together with a weighted target-likelihood loss. In the batch implementation, one teacher is constructed for each edit batch, and all student layers are updated.
Critical strengths
The main results show a substantial improvement in the edit–preservation trade-off.
The evaluation extends beyond fixed target continuations and a single editing setting.
Critical weaknesses
1. The training schedule is not sufficiently specified to support the sequential-editing comparison. Algorithm 1 places the epoch loop outside the loop over the complete edit set, and Table 10 sets the number of epochs to five. As written, the algorithm appears to revisit earlier edits after processing later ones, rather than completing training on each arriving batch before moving to the next. This distinction matters: revisiting old edits can directly improve their final retention.
The paper does not establish whether the reported experiments follow this global multi-pass schedule or whether the pseudocode is intended to describe multiple updates within each arriving batch. It also does not clearly specify comparable historical-edit access for the baselines. I therefore cannot yet interpret the performance gap as evidence of better retention under a matched sequential arrival protocol.
The current ablations vary the target-loss weight, batch size, and external context, but do not isolate independent teacher construction with a matched teacher-reset control. Such a comparison would help establish whether the proposed mechanism, rather than differences in data access or training schedule, explains the improvement.
2. The preservation claim is stronger than the Locality evaluation establishes. CHATEVAL uses semantic judgments of generated answers for Reliability and Generality, but retains top-1 argmax agreement for Locality. Equations 2 and 4 do not clearly specify whether this agreement concerns a next token, answer tokens under a supplied prefix, or a complete decoded response. These alternatives measure different forms of preservation.
This ambiguity is especially consequential in reasoning mode. Table 6 reports CounterFact Locality near 100 for several methods, even when their edit success is very low. These numbers do not establish a metric artifact, but they make it important to verify that agreement reflects preserved answer content rather than shared formatting or response prefixes.
The five general-capability benchmarks provide valuable complementary evidence, but they do not directly test whether unrelated factual answers remain unchanged. Their average also masks individual losses, including a 7.78-point AIME decline on SmolLM3 after CounterFact editing and a 6.65-point IFEval decline after ZsRE editing. The evidence supports limited average capability degradation more clearly than broad alignment preservation.
Questions for authors
- Clarification of the training protocol: Does Table 1 use five complete passes over the 2,000-edit set, as Algorithm 1 appears to specify, or five updates/passes within each arriving batch before moving on? Please state when historical edits can be revisited, when evaluation occurs, and which baselines have equivalent access. If the current results use global replay, a matched no-revisit comparison would materially affect my assessment.
- Clarification and validation of Locality: What exactly is compared in the argmax agreement in Equations 2 and 4? Are chat-control tokens, reasoning delimiters, and other response-prefix tokens included? An answer-level preservation evaluation on the unrelated prompts would help determine whether the reported Locality reflects semantic preservation, especially in Table 6.
- Assessment-changing mechanism control: How does AlignEdit perform when teachers are updated cumulatively rather than independently reset, with student training and data access held fixed? A comparison using teacher-generated rather than student-generated trajectories would also help distinguish the contributions of teacher construction and on-policy supervision.
- Clarification of the optimization claim: Is Equation 11 intended as a conditional-prefix distillation surrogate, rather than an unbiased gradient estimator of the sequence-level objective in Equation 10? The implementation detaches generated trajectories and normalizes by response length. Please clarify the scope of the claimed equivalence.
4 comments
Rebuttal by Authors (2)
Definition and validation of Locality
-
Our token-level Locality evaluation follows the teacher-forced prediction-agreement approach used in EasyEdit [1]. For each locality question, the pre- and post-edit models receive the same formatted prompt and preceding reference-answer tokens. At each reference-answer position, we compare the two models’ top-ranked next-token predictions. We average this agreement first across answer positions and then across examples. The reference answer provides the shared conditioning sequence and identifies the positions to evaluate.
-
We initially explored using an LLM judge for Locality as well. However, the reference for Locality is the pre-edit model’s response, and we empirically observed frequent errors when the judge assessed whether that response was preserved. Our subsequent validation against human annotations yielded a Krippendorff’s α of 0.697, providing quantitative evidence of the judge’s limited reliability in this setting. To make the evaluation more robust, we therefore adopted our current Locality evaluation protocol. Prior work has also raised similar concerns about Locality evaluation and adopted separate evaluation protocols for this metric [2][3][4].
-
In our implementation, non-reasoning evaluation includes an empty thinking block ending with
</think>in the input, while reasoning-enabled evaluation prefills<think>. We observed that generated responses frequently begin with phrases such as “Let me think…” and “We need to…”. These common openings may lead the pre- and post-edit models to predict similar tokens at the beginning of the reference-answer span, contributing to the high token-level Locality scores in the reasoning-enabled setting. -
We consequently added an answer-level preservation evaluation that compares the factual content of the pre- and post-edit models’ final responses, using the text after
</think>. The results are shown below and are also reported in Table 10. The detailed setup is described in Lines 1158-1165.
Table 10. LLM-Locality scores with reasoning mode disabled and enabled.
| Model | Reasoning mode | FT-M | ICE | LocFT | AlphaEdit | WISE | RLEdit | AlignEdit |
|---|---|---|---|---|---|---|---|---|
| Qwen3-4B | Off | 10.64 | 2.87 | 0.05 | 6.33 | 8.56 | 5.16 | 24.41 |
| Qwen3-4B | On | 0.00 | 0.00 | 0.00 | 2.09 | – | 0.00 | 24.72 |
| SmolLM3-3B | Off | 22.69 | 24.81 | 17.59 | 0.37 | 10.10 | 74.02 | 43.04 |
| SmolLM3-3B | On | 31.18 | 37.59 | 1.00 | 0.00 | – | 59.52 | 38.98 |
- With reasoning enabled, AlignEdit ranks first on Qwen3-4B (24.72) and second on SmolLM3-3B (38.98) in answer-level Locality. RLEdit achieves the highest Locality on SmolLM3-3B at 59.52, while its Reliability and Generality remain close to zero. These results support AlignEdit’s favorable balance between incorporating target knowledge and preserving unrelated knowledge under answer-level evaluation as well.
Individual changes in general capabilities
-
We acknowledge that some benchmarks show performance declines after editing. However, the results also show improvements: Qwen3-4B improves by 2.64 percentage points on LiveCodeBench after CounterFact editing, while SmolLM3-3B improves by 9.69 and 4.80 percentage points on BFCL after CounterFact and ZsRE editing, respectively. These gains should also be considered when assessing the overall impact of editing on post-trained capabilities.
-
Knowledge editing modifies model parameters and, in this respect, resembles an additional stage of post-training. Such updates can lead to trade-offs across capabilities, with improvements on some benchmarks and declines on others. While individual performance losses should not be overlooked, focusing on these losses alone would give an incomplete picture of the method’s effects. We therefore believe that the average performance across the evaluated benchmarks provides a meaningful basis for assessing the method’s overall effectiveness.
Rebuttal by Authors (3)
Ablations of the main components
- We conducted separate ablations of independent teacher construction, on-policy distillation, full-parameter student updates, and the direct target-loss term. The results for independent teacher construction, on-policy distillation, and full-parameter student updates are shown below and are also reported in Section 7.1 and Table 3 of the revised manuscript. The direct target-loss term is examined through the λ ablation presented in Lines 498-503 and Figure 3.
Table 3. Component ablation results on CounterFact.
| Model | Method | Reliability | Generality | Locality | Score |
|---|---|---|---|---|---|
| Qwen3-4B | AlignEdit | 77.80 | 59.65 | 54.50 | 62.54 |
| Qwen3-4B | w/o Independent teacher | 78.85 | 58.10 | 7.85 | 19.07 |
| Qwen3-4B | w/o on-policy distillation | 12.85 | 9.45 | 34.40 | 14.10 |
| Qwen3-4B | w/o full parameter update | 14.10 | 12.85 | 68.10 | 18.36 |
| SmolLM3-3B | AlignEdit | 73.10 | 65.80 | 77.98 | 71.94 |
| SmolLM3-3B | w/o Independent teacher | 7.8 | 8.4 | 64.0 | 11.41 |
| SmolLM3-3B | w/o on-policy distillation | 54.50 | 37.85 | 68.70 | 50.57 |
| SmolLM3-3B | w/o full parameter update | 3.60 | 2.90 | 96.95 | 4.74 |
-
Replacing independent teacher construction with cumulative teacher updates substantially degrades preservation. On Qwen3-4B, Reliability and Generality are largely maintained, while Locality collapses. On SmolLM3-3B, Reliability and Generality collapse, while Locality also decreases. These results support our motivation for initializing each teacher from the original checkpoint to limit interference across edits.
-
Replacing on-policy distillation with off-policy distillation using teacher-generated trajectories consistently degrades all three metrics. This supports the contribution of teacher supervision along student-generated trajectories.
-
Restricting student updates to the MLP parameters of a single layer, as in teacher fine-tuning, substantially reduces Reliability, Generality, and the overall Score on both models, while increasing Locality. This supports the importance of full-parameter student updates for balancing edit success with preservation in our setting.
-
The λ ablation shows that increasing the weight of the direct target-loss term improves Reliability and Generality at the cost of reduced Locality. In particular, when the target-loss term is omitted, SmolLM3-3B exhibits low Reliability but high Locality. These results highlight the role of direct target supervision in acquiring target knowledge and balancing edit success with preservation.
Scope of the optimization equivalence
-
Thank you for highlighting this distinction. Yes, Equation 11 should be understood as a length-normalized conditional-prefix distillation surrogate. The equivalence in Equation 10 holds at the level of the expected sequence objectives under the teacher-induced reward. The sequence-level reverse KL decomposes into an expected sum of token-level reverse-KL terms over student-generated prefixes.
-
In our implementation, sampled trajectories are treated as fixed during backpropagation. Computing the KL over the full vocabulary gives the exact categorical KL and its gradient conditional on each fixed prefix. However, the update does not include the gradient contribution arising from the student-dependent probability of generating those prefixes. Furthermore, dividing each response’s loss by its length changes the relative weighting of responses of different lengths. Consequently, the resulting update is not, in general, an unbiased gradient estimator of the sequence-level objective in Equation 10.
-
Our claim of exact computation in Appendix A is therefore limited to the conditional KL and its gradient at a fixed prefix. Equation 10 provides the objective-level motivation, while Equation 11 defines the practical distillation surrogate used for training.
We thank the reviewer once again for the constructive feedback, which has helped us improve our experimental analysis and the clarity of the manuscript. We would be happy to discuss any remaining questions.
References
[1] Wang et al. EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models. ACL 2024, System Demonstrations.
[2] Pohl et al. Towards a Principled Evaluation of Knowledge Editors. First Workshop on Large Language Model Memorization (L2M2), 2025.
[3] Yang et al. The Mirage of Model Editing: Revisiting Evaluation in the Wild. ACL 2025.
[4] Yang et al. Fine-tuning Done Right in Model Editing. ICLR 2026.
Follow-up on Our Responses and Revisions
Dear Reviewer 7rdz,
Thank you again for your careful evaluation and constructive feedback. We have addressed the points you raised through additional experiments and manuscript revisions, with changes highlighted in blue. Our rebuttal and manuscript revisions address the following points:
- The training schedule and access to historical edits, including baseline comparisons under the breadth-first pipeline (Appendix G and Table 11).
- The definition and validation of Locality, including answer-level preservation evaluation with reasoning disabled and enabled (Table 10 and Lines 1158–1174).
- Changes in general capabilities across individual benchmarks, including both improvements and declines.
- Ablations of the main components (Section 7.1).
- The scope of the optimization equivalence, distinguishing the sequence-level objective from the practical distillation surrogate and clarifying the scope of exact KL computation.
We would sincerely appreciate any further feedback you may have on our responses and revisions. If you feel that your concerns have been adequately addressed, we would be grateful if you would consider revisiting your assessment. Should any points require further clarification or discussion, we would be happy to elaborate.
Best regards,
The authors
Rebuttal by Authors (1)
We appreciate the reviewer’s careful evaluation and constructive feedback. In response to the concerns raised, we have performed additional experiments and updated the manuscript, highlighting the revisions in blue. We address each point below.
The AlignEdit results in Table 1 use five complete passes over the entire 2,000-edit set. Earlier edits are therefore revisited in subsequent epochs, after later edits have been processed.
We perform five complete passes over the entire edit set, drawing on existing training practices in standard on-policy distillation and LocFT [4]. Specifically, we adopt a breadth-first (BF) pipeline that simply swaps the number of epochs and the number of gradient updates per batch, spreading the updates across multiple epochs while preserving the total number of student gradient updates. This scheduling strategy was a key aspect of LocFT, which was well received at ICLR 2026. We simply adopt the same approach here.
However, to ensure a fairer comparison, we adapted the baselines to the breadth-first schedule and evaluated them alongside AlignEdit with comparable access to historical edits. The results are shown below and are also reported in Table 11 of the revised manuscript. The experimental setup is described in Appendix G.
Table 11. Results for methods evaluated under the breadth-first (BF) pipeline.