Rethinking Covariance in Knowledge Editing: Estimation, Update Geometry, and Mechanisms

Many locate-then-edit knowledge editing (KE) methods like ROME and AlphaEdit promise efficient factual updates by modifying a small set of model weights, and they rely on precomputed covariance statistics to preserve existing knowledge during parameter updates. However, the cost of estimating these statistics is generally overlooked and the mechanisms through which they influence editing remain poorly understood. In this paper, we present a systematic investigation connecting covariance estimation, update geometry, and editor-specific sensitivity. We find that extending calibration contexts increases estimation costs but yields no consistent gains in editing performance or general ability, revealing a mismatch between computational overhead and downstream benefits. We further examine how editing performance varies across calibration sequence length distributions. Beyond estimation, we show that covariance defines an anisotropic geometry whose effects on editing depend on how each method translates it into parameter updates. Mechanism analysis identifies update magnitude and directional structure as distinct factors in covariance sensitivity, with their relative importance varying across editors: MEMIT's sensitivity to calibration length is largely attributable to update magnitude, whereas AlphaEdit's behavior depends on the subspace it selects. Together, these findings show that covariance's role in knowledge editing is governed by the interaction between its geometry and the editor's update rule.