- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
KGHEdit augments a continual parameter-editing method with support examples selected from a local knowledge graph. It treats the current edit and prior edits as heat sources, scores nearby entities by damped graph diffusion, and filters candidate examples using compatibility and conflict-risk rules before passing a weighted batch to alphaedit or NAS. The paper reports matched editing results on wikibigedit, constructed medical streams, and a 10,000-edit counterfact evaluation. It also compares heat diffusion with alternative support selectors and presents ablations of its balance component. The central claim is that graph-guided scope selection and historical balancing improve continual editing while preserving unrelated knowledge.
Critical strengths
- The method improves several matched editing outcomes. Table 1 compares KGHEdit with the same base editor under matched backbone, request order, and stream length. The NAS gains on Llama, Qwen, and GPT-J make the empirical result more consequential than a single-editor demonstration.
- Support selection is tested against meaningful alternatives. Table 5 holds the editor and support budget fixed while comparing diffusion with k-hop, entity retrieval, PPR, and a no-heat variant. This addresses the immediate concern that any larger support batch would yield the same result.
Critical weaknesses
- The method improves several matched editing outcomes. Table 1 compares KGHEdit with the same base editor under matched backbone, request order, and stream length. The NAS gains on Llama, Qwen, and GPT-J make the empirical result more consequential than a single-editor demonstration.
- Support selection is tested against meaningful alternatives. Table 5 holds the editor and support budget fixed while comparing diffusion with k-hop, entity retrieval, PPR, and a no-heat variant. This addresses the immediate concern that any larger support batch would yield the same result.
Questions for authors
- What is the substantive continual-editing advance over GLAME beyond using a different graph-based support score? Which part of the reported gain do the authors attribute to historical coupling?
- How do the authors reconcile the no-balance result in table 5 with the identical and scores in appendix table 12? In the main streams, when does memory change the selected support or the edited model’s answers?
- What graph source and snapshot were available at edit time for each evaluation? Could graph-derived support contain facts from later requests or answers to propagation and locality probes?
3 comments
Follow-up on Continual Editing and Historical Support
Dear Reviewer uGjG,
Thank you again for your constructive feedback and recognition of our experimental results. In response to your questions, we added a matched comparison with GLAME, examined when historical support improves editing through repeated-fact experiments, and clarified the distinction between memory-based diffusion and Balance. We also explained the information available at edit time and its separation from evaluation inputs.
As the ICLR discussion period is approaching its end, we would sincerely appreciate your thoughts on these responses and welcome any further discussion of your remaining questions.
Thank you very much for your time and consideration.
KGHEdit (#17368): Request for Response from Reviewer uGjG
Dear Reviewer uGjG,
As the discussion period is nearing its end, we sincerely hope to hear your thoughts on our rebuttal.
We have added a direct comparison with GLAME, examined the contribution of historical support, and clarified the information available during editing.
We would greatly appreciate it if you could review these results and let us know whether your main concerns have been addressed.
Thank you very much for your time and consideration.
Response to Reviewer uGjG
Thank you for recognizing the matched improvements across base editors and the controlled support-selection comparisons. We appreciate your questions about the continual-editing advance, the role of historical information, and the information available during editing.
KGHEdit maintains a decayed graph footprint of previous edits and combines this footprint with the current edit's local heat source (Section 3.3). Before each parameter update, it uses compatibility and collision-risk checks to construct a bounded support batch (Section 3.4). This stateful support-planning process allows earlier edits to influence later support selection when their graph neighborhoods interact.
We added a three-seed, same-stream Fixed-4 comparison with GLAME. The methods use matched request streams and the same underlying update interface; the comparison tracks behavior as edits accumulate:
GLAME performs better at the early checkpoints, while KGHEdit retains higher performance at later checkpoints; the GLAME runs also show substantial variation across seeds at longer horizons. This comparison illustrates the long-stream behavior of the two support-planning approaches.
To examine when historical interactions matter, we separately evaluated the pre-edit gate on three-seed natural temporal-repeat streams. Enabling the gate improves later repeated-fact performance by +0.0166 (95% CI [+0.0039, +0.0283]). The GLAME comparison and the gate experiment thus provide complementary evidence about long-horizon behavior and conditional support activation, respectively.
The two experiments intervene on different operations. In Section 3.3, α controls the contribution of previous-edit memory to the diffusion source. In Section 3.4, Balance applies compatibility and collision-risk checks when the heat-ranked candidates are compiled into the edit batch.
Table 5 shows that disabling Balance reduces Propagation from 0.2792 to 0.2375 on its matched proxy stream. Appendix Table 12 varies α on a separate 96-edit stress stream, where α=0 and α=0.2 both yield 0.3356 Propagation. The former measures the support compiler; the latter measures source mixing under that stress-flow configuration. We will make this distinction explicit beside both results.
We also measured the interaction-dependent effect of activating auxiliary support in natural temporal-repeat streams:
Both gains are positive on the later-repeat subset. In the separate frozen single-seed Heat × History-source intervention, the source toggle does not change the final score and changes selected support in one Heat round. These observations identify the repeated-slot activation regime where support affects measured outcomes, while keeping the effects of the α source and the Balance compiler distinct. We will add this distinction to the mechanism discussion and align the terminology across the main text and appendix.
KGHEdit uses two information sources at edit time: the external KG neighborhood linked to the current request, and planner memory containing previously submitted edits. Algorithm 1 updates planner memory only after the current parameter write. Appendix Table 8 separately reserves Rephrase, Persona, Locality, and Propagation prompts and answers for evaluation, so those evaluator fields are not supplied directly to the support selector.
The external KG is a distinct source of factual information. To establish its temporal and factual boundary, the relevant checks are the KG source and snapshot for each stream, together with fact-level comparisons between selected support, later edit targets, and evaluation answers. We will specify the KG provenance and the available overlap checks explicitly in the experimental details. The current protocol documents the field-level separation; a completed triple-level overlap audit is not included in the reported evidence.
We appreciate your recognition of the matched editing results and your focus on the mechanism behind continual gains. We hope the new GLAME comparison, repeated-fact analysis, and clearer information-boundary description help resolve your questions, and we would be grateful if you would reconsider your rating in light of the additional evidence.