acceptodds
Under review as a conference paper at ICLR 2027

AlignEdit: Alignment-Preserving Knowledge Editing for Post-Trained Language Models

Abstract

Knowledge editing updates specific knowledge while preserving unrelated behavior. This is challenging for post-trained language models, where edits must work with model-specific chat templates and open-ended generation without disrupting acquired capabilities. Yet most methods and evaluations target language models used directly after pretraining and assess fixed target continuations. We introduce an evaluation protocol combining model-specific input formatting with response-level correctness. It shows that representative methods struggle to balance Reliability, Generality, and Locality during sequential editing, even with chat-formatted editing inputs. We then propose AlignEdit, which independently constructs knowledge-specific teachers from the original model through localized fine-tuning and distills them into a shared student using dense supervision along student-generated trajectories. We evaluate AlignEdit on Qwen3-4B and SmolLM3-3B using CounterFact and ZsRE, together with five general-capability benchmarks. Across both editing benchmarks, AlignEdit achieves a better balance among Reliability, Generality, and Locality than the evaluated baselines, with limited degradation in average general-capability performance. These results underscore the importance of accounting for post-training behavior in practical knowledge editing.

Primary area:
foundation or frontier models, including LLMs
Keywords:
knowledge editing, post-training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7rdzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AlignEdit studies knowledge editing in post-trained language models, where successful edits should remain effective under chat formatting and open-ended generation. The paper introduces CHATEVAL, which applies model-specific chat templates and uses an LLM judge to assess the correctness of generated answers for Reliability and Generality. Locality is measured through agreement between the original and edited models’ top-1 predictions on unrelated prompts.

The method constructs each teacher independently from the original model using localized fine-tuning, then transfers its behavior to a shared student. The student minimizes token-level reverse KL to the teacher along student-generated trajectories, together with a weighted target-likelihood loss. In the batch implementation, one teacher is constructed for each edit batch, and all student layers are updated.

Critical strengths

The main results show a substantial improvement in the edit–preservation trade-off.

The evaluation extends beyond fixed target continuations and a single editing setting.

Critical weaknesses

1. The training schedule is not sufficiently specified to support the sequential-editing comparison. Algorithm 1 places the epoch loop outside the loop over the complete edit set, and Table 10 sets the number of epochs to five. As written, the algorithm appears to revisit earlier edits after processing later ones, rather than completing training on each arriving batch before moving to the next. This distinction matters: revisiting old edits can directly improve their final retention.

The paper does not establish whether the reported experiments follow this global multi-pass schedule or whether the pseudocode is intended to describe multiple updates within each arriving batch. It also does not clearly specify comparable historical-edit access for the baselines. I therefore cannot yet interpret the performance gap as evidence of better retention under a matched sequential arrival protocol.

The current ablations vary the target-loss weight, batch size, and external context, but do not isolate independent teacher construction with a matched teacher-reset control. Such a comparison would help establish whether the proposed mechanism, rather than differences in data access or training schedule, explains the improvement.

2. The preservation claim is stronger than the Locality evaluation establishes. CHATEVAL uses semantic judgments of generated answers for Reliability and Generality, but retains top-1 argmax agreement for Locality. Equations 2 and 4 do not clearly specify whether this agreement concerns a next token, answer tokens under a supplied prefix, or a complete decoded response. These alternatives measure different forms of preservation.

This ambiguity is especially consequential in reasoning mode. Table 6 reports CounterFact Locality near 100 for several methods, even when their edit success is very low. These numbers do not establish a metric artifact, but they make it important to verify that agreement reflects preserved answer content rather than shared formatting or response prefixes.

The five general-capability benchmarks provide valuable complementary evidence, but they do not directly test whether unrelated factual answers remain unchanged. Their average also masks individual losses, including a 7.78-point AIME decline on SmolLM3 after CounterFact editing and a 6.65-point IFEval decline after ZsRE editing. The evidence supports limited average capability degradation more clearly than broad alignment preservation.

Questions for authors
  1. Clarification of the training protocol: Does Table 1 use five complete passes over the 2,000-edit set, as Algorithm 1 appears to specify, or five updates/passes within each arriving batch before moving on? Please state when historical edits can be revisited, when evaluation occurs, and which baselines have equivalent access. If the current results use global replay, a matched no-revisit comparison would materially affect my assessment.
  2. Clarification and validation of Locality: What exactly is compared in the argmax agreement in Equations 2 and 4? Are chat-control tokens, reasoning delimiters, and other response-prefix tokens included? An answer-level preservation evaluation on the unrelated prompts would help determine whether the reported Locality reflects semantic preservation, especially in Table 6.
  3. Assessment-changing mechanism control: How does AlignEdit perform when teachers are updated cumulatively rather than independently reset, with student training and data access held fixed? A comparison using teacher-generated rather than student-generated trajectories would also help distinguish the contributions of teacher construction and on-policy supervision.
  4. Clarification of the optimization claim: Is Equation 11 intended as a conditional-prefix distillation surrogate, rather than an unbiased gradient estimator of the sequence-level objective in Equation 10? The implementation detaches generated trajectories and normalizes by response length. Please clarify the scope of the claimed equivalence.
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1)

We appreciate the reviewer’s careful evaluation and constructive feedback. In response to the concerns raised, we have performed additional experiments and updated the manuscript, highlighting the revisions in blue. We address each point below.

Training schedule and access to historical edits

  • The AlignEdit results in Table 1 use five complete passes over the entire 2,000-edit set. Earlier edits are therefore revisited in subsequent epochs, after later edits have been processed.

  • We perform five complete passes over the entire edit set, drawing on existing training practices in standard on-policy distillation and LocFT [4]. Specifically, we adopt a breadth-first (BF) pipeline that simply swaps the number of epochs and the number of gradient updates per batch, spreading the updates across multiple epochs while preserving the total number of student gradient updates. This scheduling strategy was a key aspect of LocFT, which was well received at ICLR 2026. We simply adopt the same approach here.

  • However, to ensure a fairer comparison, we adapted the baselines to the breadth-first schedule and evaluated them alongside AlignEdit with comparable access to historical edits. The results are shown below and are also reported in Table 11 of the revised manuscript. The experimental setup is described in Appendix G.

Table 11. Results for methods evaluated under the breadth-first (BF) pipeline.

ModelMethodReliabilityGeneralityLocalityScore
Qwen3-4BICE0.000.000.180.00
Qwen3-4BLocFT2.451.400.450.90
Qwen3-4BAlphaEdit25.8522.7526.6524.97
Qwen3-4BWISE17.8011.500.381.08
Qwen3-4BRLEdit0.000.050.150.00
Qwen3-4BAlignEdit77.8059.6554.5062.54
SmolLM3-3BICE11.9511.800.381.07
SmolLM3-3BLocFT66.7055.4036.1549.42
SmolLM3-3BAlphaEdit28.7529.350.000.00
SmolLM3-3BWISE37.9529.650.431.26
SmolLM3-3BRLEdit0.550.100.030.07
SmolLM3-3BAlignEdit73.1065.8077.9871.94
  • AlignEdit continues to achieve the highest overall Score in this comparison, supporting its advantage even under the evaluated multi-pass protocol.
AuthorsOpenReview ↗

Rebuttal by Authors (2)

Definition and validation of Locality

  • Our token-level Locality evaluation follows the teacher-forced prediction-agreement approach used in EasyEdit [1]. For each locality question, the pre- and post-edit models receive the same formatted prompt and preceding reference-answer tokens. At each reference-answer position, we compare the two models’ top-ranked next-token predictions. We average this agreement first across answer positions and then across examples. The reference answer provides the shared conditioning sequence and identifies the positions to evaluate.

  • We initially explored using an LLM judge for Locality as well. However, the reference for Locality is the pre-edit model’s response, and we empirically observed frequent errors when the judge assessed whether that response was preserved. Our subsequent validation against human annotations yielded a Krippendorff’s α of 0.697, providing quantitative evidence of the judge’s limited reliability in this setting. To make the evaluation more robust, we therefore adopted our current Locality evaluation protocol. Prior work has also raised similar concerns about Locality evaluation and adopted separate evaluation protocols for this metric [2][3][4].

  • In our implementation, non-reasoning evaluation includes an empty thinking block ending with </think> in the input, while reasoning-enabled evaluation prefills <think>. We observed that generated responses frequently begin with phrases such as “Let me think…” and “We need to…”. These common openings may lead the pre- and post-edit models to predict similar tokens at the beginning of the reference-answer span, contributing to the high token-level Locality scores in the reasoning-enabled setting.

  • We consequently added an answer-level preservation evaluation that compares the factual content of the pre- and post-edit models’ final responses, using the text after </think>. The results are shown below and are also reported in Table 10. The detailed setup is described in Lines 1158-1165.

Table 10. LLM-Locality scores with reasoning mode disabled and enabled.

ModelReasoning modeFT-MICELocFTAlphaEditWISERLEditAlignEdit
Qwen3-4BOff10.642.870.056.338.565.1624.41
Qwen3-4BOn0.000.000.002.09–0.0024.72
SmolLM3-3BOff22.6924.8117.590.3710.1074.0243.04
SmolLM3-3BOn31.1837.591.000.00–59.5238.98
  • With reasoning enabled, AlignEdit ranks first on Qwen3-4B (24.72) and second on SmolLM3-3B (38.98) in answer-level Locality. RLEdit achieves the highest Locality on SmolLM3-3B at 59.52, while its Reliability and Generality remain close to zero. These results support AlignEdit’s favorable balance between incorporating target knowledge and preserving unrelated knowledge under answer-level evaluation as well.

Individual changes in general capabilities

  • We acknowledge that some benchmarks show performance declines after editing. However, the results also show improvements: Qwen3-4B improves by 2.64 percentage points on LiveCodeBench after CounterFact editing, while SmolLM3-3B improves by 9.69 and 4.80 percentage points on BFCL after CounterFact and ZsRE editing, respectively. These gains should also be considered when assessing the overall impact of editing on post-trained capabilities.

  • Knowledge editing modifies model parameters and, in this respect, resembles an additional stage of post-training. Such updates can lead to trade-offs across capabilities, with improvements on some benchmarks and declines on others. While individual performance losses should not be overlooked, focusing on these losses alone would give an incomplete picture of the method’s effects. We therefore believe that the average performance across the evaluated benchmarks provides a meaningful basis for assessing the method’s overall effectiveness.

AuthorsOpenReview ↗

Rebuttal by Authors (3)

Ablations of the main components

  • We conducted separate ablations of independent teacher construction, on-policy distillation, full-parameter student updates, and the direct target-loss term. The results for independent teacher construction, on-policy distillation, and full-parameter student updates are shown below and are also reported in Section 7.1 and Table 3 of the revised manuscript. The direct target-loss term is examined through the λ ablation presented in Lines 498-503 and Figure 3.

Table 3. Component ablation results on CounterFact.

ModelMethodReliabilityGeneralityLocalityScore
Qwen3-4BAlignEdit77.8059.6554.5062.54
Qwen3-4Bw/o Independent teacher78.8558.107.8519.07
Qwen3-4Bw/o on-policy distillation12.859.4534.4014.10
Qwen3-4Bw/o full parameter update14.1012.8568.1018.36
SmolLM3-3BAlignEdit73.1065.8077.9871.94
SmolLM3-3Bw/o Independent teacher7.88.464.011.41
SmolLM3-3Bw/o on-policy distillation54.5037.8568.7050.57
SmolLM3-3Bw/o full parameter update3.602.9096.954.74
  • Replacing independent teacher construction with cumulative teacher updates substantially degrades preservation. On Qwen3-4B, Reliability and Generality are largely maintained, while Locality collapses. On SmolLM3-3B, Reliability and Generality collapse, while Locality also decreases. These results support our motivation for initializing each teacher from the original checkpoint to limit interference across edits.

  • Replacing on-policy distillation with off-policy distillation using teacher-generated trajectories consistently degrades all three metrics. This supports the contribution of teacher supervision along student-generated trajectories.

  • Restricting student updates to the MLP parameters of a single layer, as in teacher fine-tuning, substantially reduces Reliability, Generality, and the overall Score on both models, while increasing Locality. This supports the importance of full-parameter student updates for balancing edit success with preservation in our setting.

  • The λ ablation shows that increasing the weight of the direct target-loss term improves Reliability and Generality at the cost of reduced Locality. In particular, when the target-loss term is omitted, SmolLM3-3B exhibits low Reliability but high Locality. These results highlight the role of direct target supervision in acquiring target knowledge and balancing edit success with preservation.

Scope of the optimization equivalence

  • Thank you for highlighting this distinction. Yes, Equation 11 should be understood as a length-normalized conditional-prefix distillation surrogate. The equivalence in Equation 10 holds at the level of the expected sequence objectives under the teacher-induced reward. The sequence-level reverse KL decomposes into an expected sum of token-level reverse-KL terms over student-generated prefixes.

  • In our implementation, sampled trajectories are treated as fixed during backpropagation. Computing the KL over the full vocabulary gives the exact categorical KL and its gradient conditional on each fixed prefix. However, the update does not include the gradient contribution arising from the student-dependent probability of generating those prefixes. Furthermore, dividing each response’s loss by its length changes the relative weighting of responses of different lengths. Consequently, the resulting update is not, in general, an unbiased gradient estimator of the sequence-level objective in Equation 10.

  • Our claim of exact computation in Appendix A is therefore limited to the conditional KL and its gradient at a fixed prefix. Equation 10 provides the objective-level motivation, while Equation 11 defines the practical distillation surrogate used for training.

We thank the reviewer once again for the constructive feedback, which has helped us improve our experimental analysis and the clarity of the manuscript. We would be happy to discuss any remaining questions.

References

[1] Wang et al. EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models. ACL 2024, System Demonstrations.

[2] Pohl et al. Towards a Principled Evaluation of Knowledge Editors. First Workshop on Large Language Model Memorization (L2M2), 2025.

[3] Yang et al. The Mirage of Model Editing: Revisiting Evaluation in the Wild. ACL 2025.

[4] Yang et al. Fine-tuning Done Right in Model Editing. ICLR 2026.

AuthorsOpenReview ↗

Follow-up on Our Responses and Revisions

Dear Reviewer 7rdz,

Thank you again for your careful evaluation and constructive feedback. We have addressed the points you raised through additional experiments and manuscript revisions, with changes highlighted in blue. Our rebuttal and manuscript revisions address the following points:

  • The training schedule and access to historical edits, including baseline comparisons under the breadth-first pipeline (Appendix G and Table 11).
  • The definition and validation of Locality, including answer-level preservation evaluation with reasoning disabled and enabled (Table 10 and Lines 1158–1174).
  • Changes in general capabilities across individual benchmarks, including both improvements and declines.
  • Ablations of the main components (Section 7.1).
  • The scope of the optimization equivalence, distinguishing the sequence-level objective from the practical distillation surrogate and clarifying the scope of exact KL computation.

We would sincerely appreciate any further feedback you may have on our responses and revisions. If you feel that your concerns have been adequately addressed, we would be grateful if you would consider revisiting your assessment. Should any points require further clarification or discussion, we would be happy to elaborate.

Best regards,

The authors

Reviewer_m1BbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes chat-based evaluation and teacher–student distillation for sequential knowledge editing in post-trained models, improving editing performance while largely preserving general capabilities.

Critical strengths

The setting is practically relevant, with substantial reported gains and evaluations covering general capabilities and unstructured knowledge editing.

Critical weaknesses
  1. The problem’s distinction from conventional knowledge editing is unclear. Chat templates and response-level assessment mainly change evaluation, while baseline failures are not clearly linked to post-training characteristics.

  2. The ablations do not sufficiently isolate the contributions of teacher resetting, on-policy distillation, and full-parameter student updates. Differences in training budgets also make the source of the gains less clear.

Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1)

We thank the reviewer for the constructive feedback and for recognizing the practical relevance of our setting and the breadth of our evaluation. In response to the concerns raised, we have conducted additional experiments and revised the manuscript, with changes highlighted in blue. We provide our point-by-point responses below.

Relevance of the method to post-trained models

  • Our work focuses on knowledge editing under two conditions central to the practical use of post-trained models: model-specific chat templates and open-ended response generation. We designed ChatEval to reflect these conditions and found that existing editing methods struggle to balance edit success and preservation under this evaluation protocol.

  • These conditions also guide the design of AlignEdit. Section 4 formulates editing as reward maximization over complete responses generated from chat-formatted inputs. Building on this formulation, AlignEdit provides dense teacher supervision along student-generated trajectories, extending supervision beyond the short target answer. This connects the evaluation setting to the editing objective and explicitly accounts for open-ended generation during editing.

  • The same reward-maximization framework could also be applied under BaseEval. However, Figure 1 shows that existing methods generally achieve higher Reliability and Generality under BaseEval than under ChatEval, making further improvements under BaseEval a less pressing objective. This gap highlights the need to design editing methods that account for the practical characteristics of post-trained models from the beginning. AlignEdit was developed to address this need.

Ablations of the main components

  • We conducted separate ablations of independent teacher construction, on-policy distillation, full-parameter student updates, and the direct target-loss term. The results for independent teacher construction, on-policy distillation, and full-parameter student updates are shown below and are also reported in Section 7.1 and Table 3 of the revised manuscript. The direct target-loss term is examined through the λ ablation presented in Lines 498-503 and Figure 3.

Table 3. Component ablation results on CounterFact.

ModelMethodReliabilityGeneralityLocalityScore
Qwen3-4BAlignEdit77.8059.6554.5062.54
Qwen3-4Bw/o Independent teacher78.8558.107.8519.07
Qwen3-4Bw/o on-policy distillation12.859.4534.4014.10
Qwen3-4Bw/o full parameter update14.1012.8568.1018.36
SmolLM3-3BAlignEdit73.1065.8077.9871.94
SmolLM3-3Bw/o Independent teacher7.88.464.011.41
SmolLM3-3Bw/o on-policy distillation54.5037.8568.7050.57
SmolLM3-3Bw/o full parameter update3.602.9096.954.74
  • Replacing independent teacher construction with sequential teacher construction substantially degrades preservation. On Qwen3-4B, Reliability and Generality are largely maintained, while Locality collapses. On SmolLM3-3B, Reliability and Generality collapse, while Locality also decreases. These results support our motivation for initializing each teacher from the original checkpoint to limit interference across edits.

  • Replacing on-policy distillation with standard off-policy distillation consistently degrades all three metrics. This supports the importance of providing teacher supervision along trajectories generated by the current student.

  • Restricting student updates to the MLP parameters of a single layer, as in teacher fine-tuning, substantially reduces Reliability, Generality, and the overall Score on both models, while increasing Locality. This supports the importance of full-parameter student updates for balancing edit success with preservation in our setting.

  • The λ ablation shows that increasing the weight of the direct target-loss term improves Reliability and Generality at the cost of reduced Locality. In particular, when the target-loss term is omitted, SmolLM3-3B exhibits low Reliability but high Locality. These results highlight the importance of direct target supervision for acquiring the target knowledge and its role in balancing edit success with preservation.

AuthorsOpenReview ↗

Rebuttal by Authors (2)

Unequal training budgets complicate the interpretation of the gains

  • Constructing independent teachers and distilling them into a shared student through on-policy trajectories introduces additional computational cost compared to baselines. In the main experiments, AlignEdit uses approximately 4.5 times the FLOPs of FT-M, the least computationally expensive baseline evaluated.

  • To examine whether this difference in computational resources contributes to the observed gains, we performed additional comparisons matching per-batch editing time and FLOPs separately across methods. The experimental setup and results are presented in Appendix H of the revised manuscript, with the results shown in Figure 4.

  • When editing-time budgets are matched, AlignEdit obtains the highest overall Score, demonstrating the strongest balance among Reliability, Generality, and Locality, although some baselines outperform it on individual metrics. Adjusting the baselines’ relevant hyperparameters to use the available budget does not eliminate this overall advantage. Moreover, the Score gap in favor of AlignEdit widens as the per-batch editing-time budget increases.

  • Comparisons under matched FLOPs budgets yield similar findings, with AlignEdit consistently attaining the highest overall Score. Together, these results show that its overall performance advantage remains under both runtime- and FLOPs-matched comparisons.

We again thank the reviewer for the constructive suggestions, which have strengthened our experimental analysis and improved the clarity of the manuscript. We would be happy to continue the discussion.

AuthorsOpenReview ↗

Follow-up on Our Responses and Revisions

Dear Reviewer m1Bb,

Thank you again for your constructive feedback. We have addressed the points you raised through additional experiments and manuscript revisions, with changes highlighted in blue. Our rebuttal and manuscript revisions address the following points:

  • Why AlignEdit is relevant to post-trained models.
  • Ablations of the main components (Section 7.1).
  • Comparisons under matched computational budgets: (Appendix H and Figure 4).

We would sincerely appreciate any further feedback you may have on our responses and revisions. If you feel that your concerns have been adequately addressed, we would be grateful if you would consider revisiting your assessment. Should any points require further clarification or discussion, we would be happy to elaborate.

Best regards,

The authors

Reviewer_WvfBOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies knowledge editing for post-trained LLMs and introduces a chat-template-aware, response-level evaluation protocol. It then proposes AlignEdit, which constructs teachers from the original model using localized fine-tuning and distills their behavior into a shared student on student-generated trajectories. Experiments on Qwen3-4B and SmolLM3-3B show strong sequential editing results on CounterFact and ZsRE, with additional evaluations of general-capability retention and several extended settings.

Critical strengths
  1. The paper addresses a meaningful gap in knowledge editing. Evaluating post-trained models with their chat templates and generated responses is more representative of how these models are actually used than relying only on fixed continuations.
  2. The method has a clear motivation. Constructing teachers from the original checkpoint avoids accumulating errors across teacher updates, while on-policy distillation gives the student supervision beyond the short target answer.
  3. The main editing results are strong across both models and datasets. In Table 1, AlignEdit obtains the highest overall Score in all four settings, with particularly large margins over the evaluated baselines on sequential editing.
  4. The paper also examines preservation beyond factual editing. Table 2 covers reasoning, instruction following, code, and tool use, and the additional experiments include unstructured editing, reasoning-enabled evaluation, and an 8B model.
Critical weaknesses
  1. The computational cost of the method is insufficiently characterized. The reported 2,000-edit setting constructs a separately fine-tuned teacher for each batch of ten edits, followed by student distillation on generated trajectories. Without runtime, memory, or cost comparisons, the practicality of this approach for sequential editing remains unclear.
  2. The ablations do not clearly isolate the main sources of improvement. Figure 3 varies the loss weight and batch size, but there is no controlled comparison separating independent teacher construction, on-policy distillation, and the direct target-likelihood term. It is difficult to tell which component explains the large gains in Table 1.
  3. The description of knowledge-specific teachers is not fully aligned with the implementation. Section 5.1 discusses a separate teacher for each edit, while Algorithm 1 and Appendix A fine-tune one teacher for each batch of ten edits. The effect of mixing edits within a teacher and the sensitivity to batch composition are not examined.
  4. The evidence for preserving alignment is narrower than the title suggests. Locality on CounterFact and ZsRE is based on changes in top-token predictions for unrelated prompts, and general-capability retention covers five tasks rather than safety or preference alignment. The response-level judge is also a single LLM, without an independent agreement study.
Questions for authors
  1. Could the authors report GPU memory, and model calls or training steps for 2,000 edits, alongside representative baselines? How does this cost scale with the number of edits?
  2. Could the authors ablate the teacher-construction strategy, on-policy reverse-KL loss, and direct target loss separately, using the same editing budget and evaluation protocol?
  3. Is one teacher constructed per edit or per edit batch in the reported experiments? Could the authors test whether teacher batch size and edit ordering affect locality or interference?
  4. Could the authors validate the LLM judge on a sampled set of responses and provide an additional behavior-preservation evaluation beyond token-level locality, particularly for instruction or safety-related behavior?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1)

We thank the reviewer for the thoughtful assessment and constructive suggestions. To address the concerns raised, we have conducted additional experiments and revised the manuscript, with changes highlighted in blue. Our point-by-point responses are provided below.

Computational cost

  • AlignEdit constructs independent teachers and distills them into a shared student using on-policy trajectories. This incurs additional computational cost. Under the settings reported in the main experiments, AlignEdit requires approximately 4.5 times the FLOPs of FT-M, the least computationally expensive method among the evaluated baselines.
  • To address the concern that differences in training budgets may contribute to the reported gains, we conducted additional comparisons with matched per-batch runtime and FLOPs budgets. These experiments are reported in Appendix H and Figure 4 of the revised manuscript.
  • Under time-matched settings, AlignEdit achieves the highest overall Score, reflecting the most favorable balance among Reliability, Generality, and Locality, although individual baselines perform better on some metrics. Even after increasing the baselines’ key training hyperparameters, they do not match this overall balance. AlignEdit’s Score advantage also grows as the per-batch editing-time budget increases. The FLOPs-matched experiments show a similar pattern, with AlignEdit consistently achieving the highest overall Score.
  • These results show that AlignEdit’s overall performance advantage persists when the computational budgets are matched.

Ablations of the main components

  • We conducted separate ablations of independent teacher construction, on-policy distillation, full-parameter student updates, and the direct target-loss term. The results for independent teacher construction, on-policy distillation, and full-parameter student updates are shown below and are also reported in Section 7.1 and Table 3 of the revised manuscript. The direct target-loss term is examined through the λ ablation presented in Lines 498-503 and Figure 3.

Table 3. Component ablation results on CounterFact.

ModelMethodReliabilityGeneralityLocalityScore
Qwen3-4BAlignEdit77.8059.6554.5062.54
Qwen3-4Bw/o Independent teacher78.8558.107.8519.07
Qwen3-4Bw/o on-policy distillation12.859.4534.4014.10
Qwen3-4Bw/o full parameter update14.1012.8568.1018.36
SmolLM3-3BAlignEdit73.1065.8077.9871.94
SmolLM3-3Bw/o Independent teacher7.88.464.011.41
SmolLM3-3Bw/o on-policy distillation54.5037.8568.7050.57
SmolLM3-3Bw/o full parameter update3.602.9096.954.74
  • Replacing independent teacher construction with sequential teacher construction substantially reduces preservation. On Qwen3-4B, Reliability and Generality are largely maintained, while Locality collapses. On SmolLM3-3B, Reliability and Generality collapse, while Locality also decreases. These results support our motivation for constructing each teacher from the original checkpoint to limit interference across edits.
  • Replacing on-policy distillation with standard off-policy distillation consistently degrades all three metrics. This supports the contribution of teacher supervision along student-generated trajectories.
  • Restricting student updates to the MLP parameters of a single layer, as in teacher fine-tuning, substantially reduces Reliability, Generality, and the overall Score on both models, while increasing Locality. This supports the importance of full-parameter student updates for balancing edit success with preservation in our setting.
  • The λ ablation shows that increasing the weight of the direct target-loss term improves Reliability and Generality at the cost of reduced Locality. In particular, when the target-loss term is omitted, SmolLM3-3B exhibits low Reliability but high Locality. These results highlight the role of direct target supervision in acquiring the target knowledge and balancing edit success with preservation.
AuthorsOpenReview ↗

Rebuttal by Authors (2)

Teacher construction per edit or per batch

  • In the main experiments, we construct one teacher per batch of ten edits. Each teacher is initialized from the original checkpoint and fine-tuned on its corresponding batch.
  • We originally described the motivation and formulation in Section 5.1 using the single-edit setting for clarity. Algorithm 1 describes the batch-editing procedure used in the experiments, which accommodates the hundreds or thousands of edits considered in practice. We also evaluate different batch sizes, including the single-edit setting, as discussed in Lines 504–510 of the manuscript.
  • We agree that the relationship between the single-edit formulation and the batch-based implementation required clarification. We have added an explicit explanation in Lines 353–357.

Scope of the alignment-preservation claim

  • We use “alignment” broadly to encompass the adaptation of pretrained models for human use through post-training. Our evaluation focuses on the preservation of capabilities developed or refined during this process, including reasoning, instruction following, code generation, and tool use. This perspective motivated both the name AlignEdit and our selection of evaluation benchmarks.
  • We recognize that this intended scope was insufficiently explicit in the original manuscript. We have clarified our usage of “alignment” and the scope of our preservation evaluation in Lines 44–47 of the revision.

Validation of the LLM judge

  • To assess the reliability of the LLM judge, we asked two human annotators to evaluate 200 sampled responses to Reliability and Generality prompts from CounterFact and ZsRE and compared their judgments with those of the LLM.
  • The resulting Krippendorff’s α was 0.929, indicating high agreement on the sampled responses. This supports the reliability of the LLM judge for our response-level evaluation of Reliability and Generality.
  • The detailed evaluation setup is described in Lines 987–998.

Evaluation beyond token-level Locality

  • We added a response-level Locality evaluation using an LLM judge, extending the generation-based evaluation used for Reliability and Generality to unrelated questions. The results are shown below and are also reported in Table 10 of the revised manuscript.

Table 10. LLM-Locality scores with reasoning mode disabled and enabled.

ModelReasoning modeFT-MICELocFTAlphaEditWISERLEditAlignEdit
Qwen3-4BOff10.642.870.056.338.565.1624.41
Qwen3-4BOn0.000.000.002.09–0.0024.72
SmolLM3-3BOff22.6924.8117.590.3710.1074.0243.04
SmolLM3-3BOn31.1837.591.000.00–59.5238.98
  • AlignEdit achieves the highest LLM-judged Locality on Qwen3-4B and the second-highest on SmolLM3-3B, with reasoning both disabled and enabled. On SmolLM3-3B, RLEdit achieves the highest LLM-judged Locality, but its Reliability and Generality are close to zero. These results suggest that AlignEdit achieves a more favorable balance between incorporating target knowledge and preserving unrelated knowledge under the response-level evaluation as well.

We thank the reviewer again for the suggestions, which helped us strengthen the experimental analysis and clarify the manuscript. We welcome further discussion.

AuthorsOpenReview ↗

Follow-up on Our Responses and Revisions

Dear Reviewer WvfB,

Thank you for your thoughtful feedback. We have addressed the points you raised and revised the manuscript accordingly. The revisions cover:

  • AlignEdit’s computational cost (Appendix H and Figure 4).
  • Ablations of its main components (Section 7.1).
  • The batch editing setup used in our experiments (Lines 504–510).
  • The scope of “alignment” in our paper (Lines 44–47).
  • The suitability of the LLM judge (Lines 987–998).
  • Locality evaluation using LLM-as-a-Judge (Table 10 and Lines 1158–1174).

We would sincerely appreciate any further feedback you may have on our responses and revisions. If you feel that your concerns have been adequately addressed, we would be grateful if you would consider revisiting your assessment. Should any points require further clarification or discussion, we would be happy to elaborate. Thank you again for your time and thoughtful consideration.

Best regards,
The authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

Thank you for your time and careful consideration of our work. With approximately one day remaining before the discussion period closes on November 18 at 23:59 AoE, we would welcome your feedback on whether our responses have addressed your main concerns. If so, we would be grateful if you could consider updating your assessment in light of the clarifications and additional results. We would be happy to clarify any remaining points.

For ease of reference, we summarize the key points of our rebuttal below:

  • Ablations of the main components: In response to the concerns raised by all reviewers, we conducted additional ablations of AlignEdit’s main components. The results support our key design choices: replacing independent teacher construction with sequential teacher construction reduces Locality, while replacing on-policy distillation with off-policy distillation degrades all three metrics. Restricting student updates to a single layer’s MLP reduces Reliability and Generality despite improving Locality, supporting the importance of full-parameter updates for balancing edit success and preservation. The λ ablation further shows that removing direct target supervision can compromise this balance.

  • Computational cost: As requested by Reviewers WvfB and m1Bb, we measured the computational cost of each editing method. Under the main experimental settings, AlignEdit uses approximately 4.5× the FLOPs of FT-M, the least computationally expensive baseline. To account for differences in computational requirements across methods, we also conducted runtime- and FLOPs-matched comparisons. AlignEdit achieves the highest overall Score in both settings, although some baselines perform better on individual metrics. Its Score advantage also grows as the editing-time budget increases. These results show that AlignEdit’s overall performance advantage persists under matched computational budgets.

  • Locality evaluation: As clarified in response to Reviewers WvfB and 7rdz, we use teacher forcing and token-level exact-match rates to check the behavior of the model before and after editing. To complement this token-level evaluation, we also assessed response-level Locality using an LLM-as-a-Judge protocol. AlignEdit achieves the highest LLM-judged Locality on Qwen3-4B and the second-highest on SmolLM3-3B, both with and without reasoning. Although RLEdit achieves the highest Locality on SmolLM3-3B, its Reliability and Generality remain close to zero, suggesting that AlignEdit offers a more favorable balance between incorporating target knowledge and preserving unrelated knowledge under LLM-judged evaluation as well.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.