- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work focuses on two aspects of multi-layer locate-then-edit knowledge editing: residual allocation across editing layers and reconstruction of layer-wise parameter updates. Particularly, the authors propose FastGate, a predictor trained with an analytic teacher signal to estimate request-specific layer importance for residual allocation. They also introduce UAR, which decomposes parameter updates into low-rank components using SVD and optimizes their reconstruction coefficients through a regularized objective.
Critical strengths
-
The motivation is natural. The paper focuses on the coarse-grained nature of traditional knowledge editing methods and investigates heterogeneity across editing layers and within parameter updates. The proposed FastGate predictor avoids the need to perform full editing independently at different layers to estimate their importance. By designing an analytic teacher score, the predictor can directly estimate the importance weights of multiple editing layers.
-
The paper reports strong performance across multiple models, datasets, and editing batch sizes.
Critical weaknesses
- Insufficient justification of the proposed methods.
- Several previous works have investigated residual distribution in knowledge editing, such as BLUE. Although BLUE does not directly focus on learning layer-importance weights, it addresses the broader problem of residual allocation in multi-layer editing. However, this paper does not provide a direct comparison with BLUE.
- The design of the UAR loss requires further justification. The loss function is relatively complex and contains multiple terms. However, the paper does not sufficiently justify the necessity of each term. For example, an ablation that completely removes would be valuable, especially considering that the reported results are relatively insensitive to changes in and .
- Some experimental results may require further clarification.
- In Figure 4, RUR-Edit shows approximately a 50% reduction in runtime compared with AlphaEdit. However, if I understand correctly, RUR-Edit introduces two additional steps (FastGate weight prediction and UAR reconstruction) compared with AlphaEdit. The paper does not appear to introduce any specific optimization to accelerate the original AlphaEdit solver. Therefore, the source of this runtime improvement is unclear.
- The specificity scores of AlphaEdit on CounterFact (1,000 edits) appear substantially lower than those reported in the original AlphaEdit paper (2,000 edits) under the same batch size. Did the authors change the specificity evaluation protocol, or are there other differences in the experimental settings that explain this discrepancy?
Questions for authors
- Could the authors compare RUR-Edit with related methods, such as BLUE?
- Please refer to Weakness 2.
- Could the authors show the distribution of the optimized coefficients ? I am curious whether they exhibit meaningful component-wise differences or just correspond to uniform shrinkage.
- How long does it take to train FastGate? Could the authors report the total wall-clock time?