acceptodds
Under review as a conference paper at ICLR 2027

Edit the Model You Deploy: Quantization Weakens Knowledge Edits Through Their Conditioning

Abstract

Knowledge edits and unlearning updates are computed on full-precision weights, but the served model is quantized, which can weaken or undo them. For unlearning, the explanation given is that an update smaller than the quantization step is rounded away. We show that this explanation does not carry over to closed-form (locate-and-edit) edits. By a classical dither argument, rounding on a fixed, evenly spaced grid without clipping is unbiased on an update of any size when the weights' bin positions are uniform and independent of it, and on four models single edits below the bin width keep at least 99% of their margin, on average, when only the edited matrix is quantized. The damage comes instead from the model the edit was solved against: quantized layers below the edit move the key it fires at, and quantized layers above change the read-out its value was fitted to. Quantizing everything except the edited matrix reproduces 90–101% of the damage to a single MEMIT edit of one layer (33–91% of it generic at int3), on seven models (124M–8B), two datasets and weight-only round-to-nearest, NF4, AWQ and served GPTQ and AWQ at 3 and 4 bits. At the served int4, an edit loses 4–16% of its margin, and at 3 bits 20–56%. Solving the same closed form inside the quantized model returns 90–100% of the 3-bit single-edit loss on four of five models (72–96% for released MEMIT), 77–104% under 3-bit GPTQ and AWQ, and 72–95% on held-out paraphrases; on Qwen2.5-3B, whose symmetric int3 model is itself broken, it buys efficacy at the cost of locality. Where the dither's assumptions fail, the explanation holds: gradient unlearning at a small learning rate, stored in bf16, changes mainly near-zero weights, whose bin positions are not uniform, and int4 rounding delivers 5–29% of such an update for seven of eight methods on a 1B model. All code and result files will be public at https://github.com/xxx/xxx upon acceptance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
knowledge editing, model editing, post-training quantization, memit, key vectors, unlearning, llm deployment

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jpvyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why quantization weakens closed-form knowledge edits. Using a classical dither argument and controlled quantization experiments, it attributes most single-edit degradation to changes in upstream keys and downstream readout rather than rounding of the edited weights. It proposes quantization-aware editing (QAE), which recomputes the key and target value through the quantized network.

Critical strengths

The main contribution is a carefully qualified mechanistic explanation supported by theoretical analysis and extensive channel-isolation experiments. The evaluation includes multiple model families, quantizers, released-editor configurations, and held-out paraphrases. Explicit reporting of failed predictions, generic quantization effects, and QAE’s limitations makes the conclusions more credible.

Critical weaknesses

The practical benefit is less established than the margin recovery suggests. Under the tested 4-bit GPTQ/AWQ configurations, only 3 of 800 edit–quantizer pairs lose efficacy, and all eight paraphrase-recovery intervals include zero. At 3 bits, QAE improves free-generation performance on GPT-2-XL but shows no significant recovery on Llama-3.2-3B despite substantial margin gains. These results leave uncertainty about when restoring margins meaningfully improves model behavior.

The deployment conclusions need validation with an actual serving pipeline. The GPTQ/AWQ experiments use custom simulated-quantization implementations rather than checkpoints produced and executed by the released toolchains. Downstream calibration statistics are also retained from the unedited model, and instruction-tuned models are evaluated without chat templates. These choices may affect the conditioning central to the proposed explanation.

Questions for authors

The same as weaknesses

Reviewer_GBFxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper says quantization hurts knowledge edits (like MEMIT), but not because small weight changes get rounded away. Instead, quantizing the other layers changes the input the edit sees and how its output is used. The fix, QAE, computes the edit directly in the quantized model.

Critical strengths
  1. The math on rounding is correct, and the assumptions are tested on real models.
  2. The finding about bf16-saved unlearning checkpoints (App. J) is interesting.
Critical weaknesses
  1. The problem is very small at 4 bits, which is what people actually deploy. Only 3 of 800 edits fail (Table 1). Real damage only shows up at 3 bits, where the models are already badly broken.
  2. The main finding is not surprising. If the rest of the model is quantized, it is a different model, so the edit works less well. Facts the model already knew also lose 33-69% as much (App. H).
  3. The edits are trained much harder than usual (no early stopping), which makes the margins very large. The setup also uses a single-layer version of MEMIT, not the standard one.
  4. QAE is tested on the same thing it is optimized for, so its good numbers are expected. It also only works at the exact bit width it was built for.
  5. The paper is very hard to read. Almost every sentence is packed with numbers, and the main text relies on a 77-page appendix.
Questions for authors
  1. With standard early stopping, do edits fail often at 4 bits?
  2. Why does quantization push the key in the opposite direction of the edit?
Reviewer_nWGeOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies quantization in the context of unlearning and "knowledge edits".

Critical strengths
  • The paper seems to have extensive numerical experiments.
Critical weaknesses
  • The paper is extremely hard to read, and the contribution is extremely hard to figure out.
Questions for authors
  • What does this sentence mean? "For closed-form edits, whose mechanism Kolbeinsson et al. (2025) left open, this is the only explanation on offer, and we show that it does not apply to them unless the weights sit on or near the grid." Seems critical to the motivation of the paper.
  • The entire introduction was hard to read.
  • The bullet point "The damage is in the conditioning" makes a fairly obvious point (for a contribution). It seems very obvious that quantization should precede editing, or be done very carefully after. Ditto for the next point.
  • The related works section (as with many other sections) is very dense writing, and hard to parse.
  • Equation 1, what is k⋆k^\star, what is v⋆v^\star? It is good to define notation.
  • Lemma 1: Assumption (ii) is not realistic at low bit widths. For example if you have a binary or ternary alphabet, the weights (hence the "phase") are not likely to be uniformly distributed in the bins.
  • Corollary 1 is not written properly as a mathematical statement! Almost certainly AI generated.
  • The paper continues like this, and is essentially unreadable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.