- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper argues that the dominant knowledge-editing paradigm is fundamentally flawed because LLMs store knowledge as interdependent belief networks rather than key-value tables. Injecting an isolated fact without reconciling the surrounding priors triggers Epistemic Dissonance. The paper introduce a new metric Self-Refutation Rate (SRR) judged by an LLM and the pilot experiments show the failure is a paradigm-level property rather than an optimization artifact. Motivated by this, the authors propose CODE. It first employ an open-book Teacher to produce dissonance-free reasoning and bootstrap the Student via SFT on the Teacher's best-scored trajectories. Experiments on LLaMA-3.1-8B and Qwen-2.5-7B (MQuAKE-CF-v2, MQuAKE-T, KnowEdit) show CODE suppresses SRR to 1.8–11.9% while roughly doubling multi-hop accuracy over the best baselines. Meanwhile, CODE scales to 90 simultaneous edits, preserves general capabilities, and faithfully internalizes the trained rationale.
Critical strengths
The method is well-motivated by the diagnosis, and each component is ablated. Also, the experiments contains multi-hop counterfactual editing, temporal updates, batch scaling to 90 edits, a 14B model, single-hop locality on KnowEdit, general-capability retention, efficiency amortization, and a comparison to inference-time scaffolding, which makes the methods very solid and strong.
Critical weaknesses
-
The methodological contribution is primarily a competent instantiation of an existing recipe. The "privileged-context oracle teacher + SFT cold start + on-policy KL distillation" pattern is the emerging on-policy self-distillation paradigm. The genuine novelty here is the diagnosis and the choice of privileged context. Overall, I think this paper is well-structured and findings is important for the model editing field so I decide to give it weak accept.
-
Table 5 demonstrates that CODE internalizes narratives faithfully , which also means every fabricated detail of a synthetic scaffold (invented agreements, archival discoveries, etc.) becomes a parametric belief the model will assert as fact. The locality evaluations (KnowEdit) test whether other attributes of the edited entity are corrupted, but not whether narrative-invented events leak into unrelated contexts. In model editing setting, this is acceptable, but I think the author should discuss this properly.
Questions for authors
- For batch edits, are narratives generated independently per edit, and is there any check for cross-narrative consistency (e.g., two narratives inventing mutually contradictory events)?
Thank you for the thorough responses and the new experiments. The provided results strengthen the paper. I decide to maintain my score. The work is solid and the rebuttal is responsive, but the contribution is primarily within the knowledge-editing subfield. I prefer more discussion for the potential of the proposed method.