Efficient Closed-Form Knowledge Editing for Mixture-of-Experts LLMs via Tucker-Structured Updates

Knowledge editing (KE) offers a cheap substitute for repeatedly fine-tuning LLMs. Most KE methods are built for dense feed-forward layers, while modern LLMs increasingly use Mixture-of-Experts (MoE) architectures which cost less memory and run faster at inference. As a result, a growing family of models has no editing tools designed for them. We introduce a MEMIT-like framework for knowledge editing in MoE-based LLMs. The method builds on the tensor structure of MoE layers to state the editing objective directly at the level of individual experts and uses the Woodbury matrix identity so that the full stacked matrix of expert weights never has to be formed or inverted. The final update reduces to inverting fixed low-rank matrices and needs no extra backward passes. In experiments, our approach stays within a small margin of strong baselines on the main KE metrics while speeding up the editing procedure by up to $3$x. The gain comes from the batched MEMIT-style formulation together with the low-dimensional inversions made possible by the Woodbury identity. Closed-form, parameter-modifying KE therefore extends beyond dense layers at low cost and can be applied to modern sparse LLM architectures at scale.