acceptodds
Under review as a conference paper at ICLR 2027

Enhancing Knowledge Injection with Surrounding Backgrounds in Continual Training LLMs

Abstract

Updating the parametric knowledge of Large Language Models (LLMs) post-training remains a significant challenge, a scenario we term Epistemic Fluidity. Both model editing and Continual Pre-training are ill-suited: the former suffers from progressive parameter interference under sequential edits, while the latter is cost-prohibitive for sparse factual updates. While Supervised Fine-tuning (SFT) is promising and computationally efficient, our controlled experiments across a six-task evaluation protocol spanning from direct recall to scenario simulation reveal a critical gap: naive or semantically-flat rewrites yield only spurious memorization of co-occurring tokens, falling short of genuine internalization for retrieval and future reasoning. To close this gap, we introduce \ours, a framework of 12 structured rewriting strategies organized into an atom-to-interactive four-level taxonomy, spanning counterfactual data from seven domains. Our experiments with different sets of rewriting strategies on three LLMs reveal three key insights: 1) sufficient knowledge exposure is a necessary condition for internalization; 2) under sufficient exposure, data diversity dominates over quantity; and 3) explicit contrastive reasoning combined with anchor-knowledge bridging drives the deepest internalization. These findings provide a systematic, data-centric foundation for transforming static LLMs into continuously updatable knowledge bases.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Data Efficiency, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_rMAWOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper proposes KnowContext, a set of twelve structured rewriting strategies for training language models on updated or counterfactual facts. It evaluates direct and inverse recall and several downstream tasks across seven domains and three models. KnowContext improves the reported six-task average over DeepKI by roughly 20–25 points.

Critical strengths

The task-by-task evaluation shows why direct fact recall alone is an incomplete measure of a knowledge update. Matching the main rewriting baselines on training-sample count makes the large observed gains worth investigating.

Critical weaknesses

The paper frames the method as a targeted knowledge update but does not measure preservation of unrelated facts or general capabilities, as its limitations section acknowledges. Without that measurement, high scores on updated facts do not establish whether the overall update is beneficial.

Both training and evaluation material are synthetic. The paper checks exact duplication, reserved-anchor overlap, and some leave-one-format-out settings, but the latter cover only selected models and domains. Three of the six task families also depend on an LLM judge. These checks do not yet establish that the large average gains transfer to independently written questions or reliably judged answers.

Questions for authors

What happens to unrelated factual recall and general-task performance after KnowContext training? Can the authors also report results on independently authored evaluation questions, with human agreement for the open-ended answer judgments? Either result could change my recommendation.

6 comments
AuthorsOpenReview ↗

Ans to W1 and Q1

We appreciate the reviewer's constructive suggestions on preserving pretrained knowledge and independently validating generalization and evaluation reliability.

Due to limited time and GPU resources, we focused our additional experiments on GEO and BIO.

W1 & Q1 – Preservation of unrelated facts and general capabilities

Ans to W1 & Q1

Our main experiments isolate knowledge-injection effectiveness without interventions against catastrophic forgetting [2]. Fine-tuning can bias models toward the update distribution and reduce pretrained capabilities [7]; conflicting supervision can further contribute to forgetting [8].

In practice, forgetting can be mitigated by mixing a common corpus into the update data [1] or replaying data from earlier training stages [6]. We additionally mix WikiText-2 into the training data with a 10% token budget on both 8B backbones and report the results below.

We evaluate preservation of:

  1. Unrelated facts: 645 held-out questions probing pretrained knowledge from Brand, Creative, Game, Materials, and History, scored by GPT-5.5.
  2. General capabilities: MMLU [3] (1,140 questions, five-shot), ARC-Challenge [4] (1,172 test questions), and HellaSwag [5] (2,000 validation questions).

1) Retention of unrelated facts

Parentheses give Δ from Base, except for GEO AVG, which uses the same method without mixing as its reference.

CheckpointReplay (%)Pretrained-knowledge accuracy (%)GEO AVG (%)
Qwen3-8B
Base—71.32—
DeepKI055.81 (−15.50)67.21 (0.00)
DeepKI1060.78 (−10.54)65.72 (−1.48)
KnowContext048.84 (−22.48)89.05 (0.00)
KnowContext1061.86 (−9.46)88.45 (−0.60)
Llama-3.1-8B
Base—70.54—
DeepKI051.94 (−18.60)67.67 (0.00)
DeepKI1051.78 (−18.76)67.71 (+0.04)
KnowContext041.24 (−29.30)86.54 (0.00)
KnowContext1067.91 (−2.64)85.11 (−1.43)

2) General capabilities

CheckpointReplay (%)MMLU accuracy (%)ARC-C normalized accuracy (%)HellaSwag normalized accuracy (%)
Qwen3-8B
Base—74.9156.5775.05
DeepKI071.05 (−3.86)51.54 (−5.03)71.20 (−3.85)
DeepKI1072.19 (−2.72)52.73 (−3.84)75.65 (+0.60)
KnowContext068.33 (−6.58)51.62 (−4.95)69.95 (−5.10)
KnowContext1070.96 (−3.95)50.85 (−5.72)75.05 (0.00)
Llama-3.1-8B
Base—67.3755.4679.85
DeepKI059.21 (−8.16)46.16 (−9.30)74.75 (−5.10)
DeepKI1061.67 (−5.70)50.43 (−5.03)78.40 (−1.45)
KnowContext059.39 (−7.98)46.50 (−8.96)73.45 (−6.40)
KnowContext1058.42 (−8.95)50.77 (−4.69)76.70 (−3.15)

We make the following observations:

  1. Common-corpus mixing mitigates forgetting of both unrelated facts and general capabilities. After mixing, KnowContext's gaps from Base are 2.64–9.46 points on unrelated facts and 3.22–5.60 points averaged across the three general benchmarks. The 10% mixture is exploratory and not tuned for the best results.

  2. Adding a common corpus causes only small decreases in targeted knowledge-update accuracy. KnowContext's GEO AVG decreases by only 0.60–1.43 points, while remaining 17.40–22.73 points above DeepKI.

AuthorsOpenReview ↗

Ans to W2a, W2b and Q2

W2a & Q2 – Transfer to independently sourced evaluation questions

Ans to W2a & Q2

We evaluate DeepKI and KnowContext on the public MQuAKE-Remastered dataset. After filtering for single-edit cases with valid reasoning chains and removing duplicate edits, we sample 60 atomic facts and retain their 180 official questions for evaluation only. We rewrite each fact with both methods using the same budget of 150 rewrites, then train following Sec. 4.1 and Appendix B.1. The results are summarized below.

Official QA uses the public MQuAKE questions. AVG averages only our six evaluation tasks (DQA–TOL), excluding Official QA. Parentheses give KnowContext − DeepKI in points for Official QA and AVG. Scores are percentages.

MethodOfficial QADQAIQAMCQMHPSCETOLAVG
Qwen3-8B
DeepKI15.5696.6747.7823.3338.3372.7841.1153.33
KnowContext31.11 (+15.56)98.8960.0069.4462.7890.5681.1177.13 (+23.80)
Llama-3.1-8B
DeepKI12.7899.4448.3343.0628.8966.677.7849.03
KnowContext34.44 (+21.67)97.7863.3367.5066.6782.2278.3375.97 (+26.94)
Qwen3-14B
DeepKI18.8998.3356.6735.8343.3373.8944.4458.75
KnowContext48.33 (+29.44)100.0055.0080.8369.4492.7879.4479.58 (+20.83)

KnowContext shows clear improvements on both the official questions (+15.56–29.44 points) and our evaluation questions (+20.83–26.94 points in AVG) across all three backbones. Its advantage therefore extends to evaluation questions sourced independently of our pipeline.


W2b – Broader leave-one-format-out controls

Ans to W2b

For the concern about overlap between training and evaluation formats, Table 7 already reports Qwen3-8B on GEO/BIO and Llama-3.1-8B on GEO. We expanded the controls to all seven domains on Qwen3-8B and GEO on all three backbones, including −Inverse conditions absent from the original BIO and Llama experiments. We removed Inverse or Discrimination rewrites, matched Full's training exposure, and kept the evaluation questions and scoring fixed.

Parentheses give leave-out − Full in points for IQA, MCQ, and AVG, calculated before rounding.

DomainIQA Full → −Inverse (%)MCQ Full → −Discrimination (%)AVG Full / −Inverse / −Discrimination (%)
Qwen3-8B
GEO84.92 → 85.47 (+0.56)78.61 → 70.00 (−8.61)88.82 / 84.79 (−4.03) / 86.37 (−2.45)
BIO92.18 → 89.94 (−2.23)86.42 → 76.59 (−9.83)84.71 / 78.14 (−6.57) / 74.42 (−10.29)
Brand59.44 → 39.44 (−20.00)72.33 → 76.42 (+4.09)65.78 / 60.05 (−5.73) / 70.93 (+5.15)
Creative92.78 → 92.78 (0.00)35.75 → 82.12 (+46.37)76.89 / 81.26 (+4.37) / 83.86 (+6.97)
Game85.00 → 82.78 (−2.22)50.00 → 42.35 (−7.65)75.38 / 71.25 (−4.14) / 75.52 (+0.14)
Materials93.89 → 95.56 (+1.67)84.44 → 73.89 (−10.56)80.93 / 79.07 (−1.85) / 79.81 (−1.11)
History80.45 → 74.86 (−5.59)73.28 → 34.77 (−38.51)77.28 / 71.45 (−5.83) / 68.91 (−8.37)
Llama-3.1-8B
GEO79.89 → 74.86 (−5.03)64.72 → 73.89 (+9.17)79.36 / 79.77 (+0.41) / 77.36 (−2.00)
Qwen3-14B
GEO83.24 → 84.92 (+1.68)83.33 → 68.33 (−15.00)85.63 / 82.98 (−2.64) / 82.38 (−3.24)

All 18 leave-out conditions retain 60.05–86.37% AVG and remain 5.87–32.96 points above DeepKI in Table 5. This supports the paper's conclusion that direct format matching does not explain KnowContext's overall advantage. Individual task effects remain domain-dependent, including a 20.00-point IQA drop on Brand and a 38.51-point MCQ drop on History.

AuthorsOpenReview ↗

Ans to W2c

W2c – Human agreement and judge sensitivity

Ans to W2c

For the concern about reliable judgments of open-ended answers, our main evaluation uses GPT-5.5 at temperature 0 for MHP, SCE, and TOL. The scoring uses simple binary criteria that check whether the answer applies the updated fact and completes the requested task.

1) Agreement among human judges and with GPT-5.5

We sampled 90 GEO questions per task from MHP, SCE, and TOL and kept the DeepKI and KnowContext answers to each question, for 540 answers in total. Three human judges independently scored each answer using the same task rubrics as GPT-5.5.

ComparisonAgreement (%)Fleiss’ κ
Human majority vote vs. GPT-5.591.300.771
Unanimous agreement among all three human judges95.740.921
Unanimous agreement among all three human judges and GPT-5.590.000.852

Agreement is the match rate for human majority vote versus GPT-5.5, and the unanimity rate for the three- and four-judge comparisons.

The strong agreement with the human majority supports GPT-5.5's reliability under our simple binary rubrics, while the high agreement among human judges shows that the criteria can be applied consistently.

2) Sensitivity to the automated judge

We rescored saved MHP, SCE, and TOL answers from the Qwen3-8B matched-token DeepKI and KnowContext runs on GEO and BIO with DeepSeek V4 Pro at temperature 0 with thinking disabled. We compared its decisions with those of GPT-5.5, using the same questions, answers, references, and task rubrics. The accuracy columns below use the DeepSeek judge and average the three task accuracies.

DomainDeepKI (%)KnowContext (%)Δ under DeepSeek (points)Δ under GPT-5.5 (points)Decision agreement (%)
GEO57.1580.34+23.19+24.3091.47
BIO35.1181.15+46.04+46.0092.16

KnowContext retains its lead under both supplementary judges. Its margin changes by at most 1.11 points across GEO and BIO.

References

[1] Ibrahim et al. (2024). Simple and Scalable Strategies to Continually Pre-train Large Language Models. Transactions on Machine Learning Research.

[2] Kirkpatrick et al. (2017). Overcoming catastrophic forgetting in neural networks. PNAS.

[3] Hendrycks et al. (2021). Measuring Massive Multitask Language Understanding. ICLR’21.

[4] Clark et al. (2018). Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge.

[5] Zellers et al. (2019). HellaSwag: Can a Machine Really Finish Your Sentence? ACL’19.

[6] Rolnick et al. (2019). Experience Replay for Continual Learning. NeurIPS’19.

[7] Kotha et al. (2024). Understanding Catastrophic Forgetting in Language Models via Implicit Inference. ICLR’24.

[8] Diao et al. (2026). Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting. Findings of EMNLP’26.

Reviewer_RRySOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how training-data construction affects factual knowledge updates in LLMs. It introduces KnowContext, which expands target facts using 12 rewriting strategies covering bidirectional associations, logical reasoning, knowledge anchoring, and interactive tasks. It also proposes KnowDepth, a six-task evaluation protocol assessing factual recall and downstream application. Experiments across three models and seven counterfactual domains show improvements over existing baselines, particularly DeepKI.

Critical strengths
  1. KnowContext achieves substantial improvements over DeepKI, increasing average accuracy by approximately 20–25 percentage points across three models. The gains extend beyond direct recall to inverse retrieval and downstream application.

  2. The evaluation covers multiple aspects of knowledge application, and the strategy-level analyses and leave-one-format-out controls provide useful evidence about the effects of different training-data designs.

Critical weaknesses
  1. Limited novelty relative to DeepKI. Both methods follow the same general pipeline of LLM-based data augmentation followed by LoRA training. KnowContext primarily extends existing semantic rewriting and reasoning augmentation with additional contextual associations and interaction formats. The paper does not sufficiently establish a substantive methodological advance beyond expanding and organizing the augmentation strategies.

  2. The central comparison does not isolate the contribution of the proposed strategies. DeepKI uses Gemini-2.5-Flash for data generation, whereas KnowContext uses DeepSeek-V4-Flash. Although both use 150 samples per fact, this does not control for generator quality, text length, or training-token budget. The existing ablations examine components within KnowContext but do not resolve this confound in its comparison with DeepKI. Consequently, the reported improvements cannot be confidently attributed to the proposed augmentation strategies. This affects the evidence supporting the central contribution, rather than merely representing a missing supplementary experiment.

Questions for authors
  1. Can the authors compare DeepKI and KnowContext using the same data generator and matched training-token budgets? How much of the reported improvement remains under these controls?

  2. Which specific additions account for the improvement over DeepKI? Can the authors progressively add anchor bridging and interaction-based augmentation to DeepKI to establish their incremental contributions?

5 comments
AuthorsOpenReview ↗

Ans to W1 and Q2

We sincerely thank the reviewer for their careful reading and detailed feedback, particularly on the relationship to DeepKI and the fairness of the experimental comparison. These comments helped us clarify KnowContext's methodological contribution and strengthen the comparison with additional controlled experiments.

Due to limited time and GPU resources, we focused our additional experiments on GEO and BIO.

W1 & Q2 – Contribution beyond DeepKI and the role of added contexts

Ans to W1 & Q2

  1. A comprehensive rewriting framework for conflicting updates. DeepKI uses single-step reasoning supervision to help models combine injected and pre-existing knowledge at inference time. KnowContext organizes training supervision around the updated fact, progressing through four levels: the fact itself, its logical implications, connections to pretrained knowledge, and interactions that apply the update despite conflicting priors.

  2. A systematic analysis of effective rewriting (acknowledged by Reviewers #zCSn and #rMAW). We compare One-Step and Two-Step (Sec. 5.1) and all 12 strategies individually (Sec. 5.2), identifying explicit contrasts between obsolete and updated facts as an effective training signal for Epistemic Fluidity.

Following Q2, we ran progressive ablations on GEO and BIO: replace 20 of 150 DeepKI rewrites per fact with Network, then another 20 with Interaction, retaining Network. Within each domain, all settings share 60 facts, 9,000 examples, DeepSeek-V4-Flash, the LoRA configuration, and 10 epochs. This fixes training exposures per fact; loss-token counts vary. Scores are percentages; parentheses give Δ from DeepKI in points.

Training dataDQAIQAMCQMHPSCETOLAVG
GEO; Qwen3-8B
DeepKI98.3360.3454.7280.0075.9846.1169.25
+ Network97.78 (−0.56)76.54 (+16.20)51.11 (−3.61)85.00 (+5.00)80.45 (+4.47)70.56 (+24.44)76.90 (+7.66)
+ Network + Interaction98.33 (0.00)82.12 (+21.79)55.56 (+0.83)86.11 (+6.11)89.94 (+13.97)91.67 (+45.56)83.96 (+14.71)
Llama-3.1-8B
DeepKI94.4456.9842.7876.1162.0151.1163.91
+ Network97.22 (+2.78)70.95 (+13.97)38.89 (−3.89)82.78 (+6.67)75.42 (+13.41)95.56 (+44.44)76.80 (+12.90)
+ Network + Interaction98.33 (+3.89)69.83 (+12.85)70.56 (+27.78)83.89 (+7.78)78.21 (+16.20)92.78 (+41.67)82.27 (+18.36)
Qwen3-14B
DeepKI97.7860.8941.1178.3378.2147.2267.26
+ Network98.33 (+0.56)82.68 (+21.79)40.00 (−1.11)84.44 (+6.11)88.27 (+10.06)77.78 (+30.56)78.58 (+11.33)
+ Network + Interaction98.33 (+0.56)79.89 (+18.99)63.89 (+22.78)89.44 (+11.11)89.94 (+11.73)80.00 (+32.78)83.58 (+16.32)
BIO; Qwen3-8B
DeepKI88.3365.9237.2834.0067.786.1149.90
+ Network95.00 (+6.67)77.09 (+11.17)61.27 (+23.99)50.67 (+16.67)85.56 (+17.78)36.67 (+30.56)67.71 (+17.80)
+ Network + Interaction97.78 (+9.44)86.03 (+20.11)85.84 (+48.55)50.67 (+16.67)92.78 (+25.00)87.78 (+81.67)83.48 (+33.57)
Llama-3.1-8B
DeepKI95.5666.4844.8035.3352.7823.3353.05
+ Network96.67 (+1.11)74.86 (+8.38)65.03 (+20.23)66.00 (+30.67)72.22 (+19.44)62.78 (+39.44)72.93 (+19.88)
+ Network + Interaction98.33 (+2.78)81.01 (+14.53)73.41 (+28.61)72.67 (+37.33)82.22 (+29.44)57.78 (+34.44)77.57 (+24.52)
Qwen3-14B
DeepKI86.6765.9228.3226.0067.2220.0049.02
+ Network93.89 (+7.22)69.83 (+3.91)46.24 (+17.92)61.33 (+35.33)90.56 (+23.33)92.22 (+72.22)75.68 (+26.66)
+ Network + Interaction98.89 (+12.22)83.24 (+17.32)95.66 (+67.34)67.33 (+41.33)94.44 (+27.22)98.89 (+78.89)89.74 (+40.72)
  1. Network increases AVG by 7.66–12.90 points on GEO and 17.80–26.66 points on BIO.
  2. Interaction adds a further 5.00–7.05 points on GEO and 4.64–15.77 points on BIO.

Both replacements improve AVG across all three backbones and both domains at fixed knowledge exposure, demonstrating a substantive contribution beyond DeepKI's style-enhanced rewriting.

AuthorsOpenReview ↗

Ans to W2 and Q1

W2 & Q1 – Common generator and matched training-token budgets

Ans to W2 & Q1

For the reviewer's first question about the training-data generator, both DeepKI and KnowContext training corpora were generated with DeepSeek-V4-Flash. We thank the reviewer for pointing out the Gemini-2.5-Flash entry carried over from an earlier draft. (Our conclusions are robust to the choice of generator LLM, as shown in our response to Reviewer #zCSn, Q1.) We have corrected it in Appendix B.1.1.

For the second question about training-token budgets, our main comparison controls knowledge exposure, defined as the number of times an atomic fact or one of its rewritten variants is seen during training. We target 150 exposures per atomic fact per epoch and train each method for 10 epochs, retaining each method's own rewrite formats and lengths. Under a fixed token budget, shorter rewrites allow more exposures to the same atomic facts.

To answer the reviewer's question directly, we additionally matched the actual loss-token budgets. We repeated each corpus to match the loss tokens of ten KnowContext epochs. The final loss-token counts differ by less than 0.022% in each pair. Both methods use the same LoRA settings, evaluation questions, and scoring protocol.

ModelDeepKI loss tokensKnowContext loss tokensDeepKI AVG (%)KnowContext AVG (%)Δ (points)
GEO
Qwen3-8B2,639,2132,639,53664.0685.40+21.34
Llama-3.1-8B2,635,5272,635,67666.1381.03+14.90
Qwen3-14B2,639,2132,639,53667.7283.59+15.87
BIO
Qwen3-8B3,354,3653,353,65252.3085.41+33.11
Llama-3.1-8B3,349,4173,349,00255.8187.31+31.50
Qwen3-14B3,354,3653,353,65251.0590.03+38.99

With a shared generator and matched loss tokens, KnowContext still maintains a substantial accuracy lead over DeepKI across all three LLMs in both domains. These results demonstrate more effective training supervision per token from KnowContext's rewrites and further address our research question of which rewriting strategies enable reliable retrieval and cross-task application of updated facts in Epistemic Fluidity.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

As the discussion period draws to a close, we would be grateful if you could review our responses and additional experimental results. We would be happy to address any remaining questions or provide further clarification. Thank you again for your time and consideration.

Best regards,

The authors of Paper 32380

Reviewer_zCSnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to update the parametric knowledge of LLMs with counterfactual facts, a setting it terms Epistemic Fluidity. KnowContext rewrites each SRO triple with a generator LLM using 12 strategies grouped into Intrinsic, Chain, Network, and Interaction categories, and KnowDepth evaluates injection with six tasks: DQA, IQA, MCQ, MHP, SCE, and TOL. LoRA adapters are trained on Qwen3-8B, Qwen3-14B, and Llama-3.1-8B-Instruct over seven domains of 60 facts each, against SFT-Seed, SFT-Rewrite, DeepKI, MEMIT, and AlphaEdit, with category ablations, leave-one-format-out controls, and Logit-Lens probing.

Critical strengths

S1: KnowDepth is a careful protocol. It separates Direct Recall from Anchored Reasoning, reserves evaluation anchors before data generation, and scores MCQ under both option orders, with zero exact duplicates in Table 12 and a 93.2% human pass rate in Table 17.

S2: The gains are large and consistent. KnowContext improves AVG over DeepKI by 20.3 to 24.8 points in Table 3 and is best in every model and domain cell of Table 5, with the margin driven by IQA, MCQ, and TOL rather than saturated DQA.

S3: The analysis is informative. Table 6 and Figure 3 identify contrastive strategies such as Trap Correction and Contrastive Negation as the strongest, and the leave-one-format-out runs in Table 7 stay 2.88 to 34.33 points above DeepKI after aligned families are removed.

Critical weaknesses

W1: All results appear to come from single runs. A retrained Full model in Table 7 reaches 80.07 on Qwen3-8B BIO against 87.21 in Table 5, so run-to-run spread is noticeable. Reporting means and standard deviations over three seeds for Table 3 would make the 20.3 to 24.8 point margins easier to read.

W2: Among rewriting methods only DeepKI is run. PORE and Zhao et al. are discussed in Table 18 but not evaluated, although both target inverse retrieval or fact recall. Adding one of them on a subset of domains would place KnowContext more precisely among data-centric approaches.

W3: The cost of the pipeline is not reported. Generating 150 passages per fact with DeepSeek-V4-Flash and training 10 epochs per domain has a real budget. Reporting generation tokens and GPU hours next to DeepKI, MEMIT, and AlphaEdit would support the efficiency argument made in Section 1.

Questions for authors

Q1: Training data and probes are both generated by DeepSeek-V4-Flash. Would probes from another generator, or DeepKI data regenerated with the same model, change the margins in Table 3?

Q2: Section 6 notes that unrelated capabilities are not measured. Could you add a neighborhood or general benchmark, and explain why MEMIT and AlphaEdit fall below Base LLM in Table 3?

Q3: Table 11 tunes settings on GEO evaluation scores; were baselines tuned separately? Minor: Section 5.2 writes Task Correction for Trap Correction.

2 comments

Thank you for the authors' detailed responses, and I will keep my score.

We thank the reviewer again for their valuable suggestions and positive assessment of our work. We would be happy to address any further questions or provide additional clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Additional Experiments

Dear Area Chair,

Thank you for the update and for following up with the reviewers. We have addressed all reviewers’ questions and incorporated the additional results and clarifications into the revised manuscript.

  1. Fair comparisons and baselines. We added matched-token comparisons and rewriting baselines, corrected and reran the model-editing baselines, and reported generation and training costs.
  2. Component ablations. We conducted progressive Network/Interaction ablations and expanded leave-one-format-out controls.
  3. Robustness and generalization. We added multi-seed and cross-generator experiments and evaluated transfer on MQuAKE.
  4. Retention and evaluation reliability. We tested retention of unrelated knowledge and general capabilities with common-corpus mixing, and validated scoring with three human annotators and an alternative LLM judge.

We sincerely appreciate your timely response and the time and effort you have devoted to handling our submission.

Best regards, The authors of Paper 32380

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.