- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies covariance-based knowledge editors, which choose the update direction by minimizing the mean squared response on unrelated inputs. It asks when this direction also minimizes the expected maximum absolute response over a budget of M probes. Theoretically, it shows this holds for every target and budget exactly when the symmetrized whitened activation law is spherical (Theorem 1), and gives a dimension-free bound linking measured directional anisotropy to the worst-target risk ratio (Theorem 2). Empirically, across five architectures, activations are anisotropic: covariance placement is about 2.2x worse than a fitted alternative at searched worst-case targets, but only about 4.8% worse at real edit targets.
Critical strengths
This work provides a strong theoretical analysis of when covariance-based placement minimizes the expected maximum edit response. However, the logical flow of the paper makes it difficult to follow as a whole.
Critical weaknesses
Beyond the presentation weakness, my main concern is the practical significance of the method. The reported gains are confined to a layer-level metric and are modest. I see no clear evidence of a downstream behavioral benefit: rewrite and paraphrase accuracy are essentially unchanged, and neighborhood accuracy improves by only 7–11 prompts out of 1,500. The caption of Table 21 notes that the two ROME baseline runs themselves differ on 12 neighborhood prompts, so this change appears to be of the same order as run-to-run variation.
Questions for authors
Can the authors show that the reduced layer-level response leads to a measurable improvement in specificity?