- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes HebbianEdit, which implements knowledge editing through FactMLP, an associative memory external to a frozen model. The method constructs fact keys from shallow hidden states and values from target-token unembeddings. A similarity gate identifies the relevant edit, after which a bilinear feature map and a least-squares readout reconstruct its value for hidden-state injection after the final normalization layer. The authors evaluate up to 10,000 edits on CounterFact and zsRE across three models, reporting high editing efficacy and lower construction costs than the compared methods. The paper also analyzes bilinear feature capacity, sharded construction, and projection refinement.
Critical strengths
-
The method provides an external editing mechanism that requires no backpropagation updates and achieves strong large-scale results. In Table 1, CounterFact efficacy at 10,000 edits is at least 98.2% across all three models, while zsRE efficacy is at least 96.5%. Table 6 compares HebbianEdit with GRACE on the same hardware and shows a reduction in construction time from hours to minutes. Keeping the base model frozen while supplying edits through an external module has practical value, and these results are the main evidence supporting acceptance.
-
The authors provide substantial comparisons examining the role of the nonlinear readout. Appendix D compares linear and bilinear memory recovery at equal width and reports end-to-end results. On CounterFact, the bilinear readout improves efficacy by approximately 3.3–4.5 percentage points. The comparison between sharded and unsharded construction also demonstrates the role of sharding in the production configuration. These experiments support specific design choices rather than only reporting final performance.
Critical weaknesses
-
The contribution of FactMLP relative to directly retrieving stored values has not been established. Section 3.3 states that the gate first selects an entry and then passes that entry’s stored key, rather than the query key, to FactMLP. Query generalization is therefore primarily handled by the gate, while FactMLP reconstructs a known value from a known entry. A baseline using the same gate, directly returning the corresponding stored value, and applying the same hidden-state injection would test the necessity of the core design, but this comparison is missing. The linear-versus-bilinear readout comparison does not address this question. Without this baseline, it is difficult to assess what the feature construction and projection solves provide over simple key–value retrieval. This directly affects my assessment of the method’s novelty and practical value.
-
The experiments do not adequately establish sequential editing and per-entry update capabilities. The paper repeatedly emphasizes 10,000 sequential edits, but the construction procedure solves over the full edit set in shards of 5,000 entries. Each shard’s projections and readout depend jointly on its facts, and value centering also depends on the edit set. The paper does not fully explain how arriving facts are inserted, how an existing fact is revised, or whether these operations require re-solving a shard. Table 6 compares one-time construction with GRACE’s per-edit online updates without matching the update schedule. If each insertion requires rebuilding the memory, the results establish efficient batch construction rather than the claimed low-cost continual maintenance. This is a key gap in the practical claims.
Questions for authors
-
With exactly the same keys, gate, and injection strength, how does directly returning the matched entry’s stored value compare in editing performance, memory usage, and inference cost? What advantages does FactMLP provide over this baseline?
-
Are the 10,000 edits applied sequentially, with existing edits evaluated after each insertion, or is the complete memory constructed before evaluation? Does adding a fact to a partially filled shard require re-solving that shard? Please report cumulative construction costs under a matched online update schedule.
-
How are individual facts revised and revoked? Does revocation only remove the key from the gate, or does it also update the projections, readout, and centering terms? Do these operations affect other entries’ outputs, and has retention of the remaining edits been evaluated?
-
Appendix C acknowledges that edits sharing the same subject and the full prefix up to the end of its mention have identical keys. How does the method handle queries in which the relation follows the subject, such as “France’s president” and “France’s capital”? How common are these cases in the reported evaluations?
-
The gate thresholds in Table 2 vary across models, datasets, and edit counts. How are these thresholds selected, and is an independent validation set used? How do generality and specificity change as the memory grows when the threshold is held fixed?
-
I could not find the AI use statement required by ICLR 2027. Please include this statement and specify whether generative AI tools were used and, if so, for which tasks.