acceptodds
Under review as a conference paper at ICLR 2027

Quantinue: Quantization for Unlearning operator

Abstract

Machine unlearning aims to make a trained model behave like a model retrained without designated forget data, and the resulting model is typically quantized for efficient deployment. Low-bit quantization, however, can undo unlearning by discarding parameter changes introduced during unlearning and recovering knowledge that was previously forgotten. Existing methods address this by making the unlearning stage more robust to quantization, but they still treat unlearning and quantization as two separate steps. We instead replace the two stages, unlearning and quantization, with quantization itself the unlearning operator. Quantization maps continuous weights to discrete codes while seeking to retain the original model behavior. We guide this mapping not only to retain the original model behavior on the retain data, but also to induce forgetting behavior on the forget data, directly constructing an unlearned low-bit model without a preceding continuous-space unlearning update. Activation statistics from forget and retain calibration data define an element-wise score that determines where and how strongly the assigned code deviates from the reconstruction-driven code. The resulting process separates ordinary quantization error from code-displacement error and compensates them separately, reducing compensation along forget-sensitive directions for the code displacement error. Both code assignment and error compensation use only forward calibration statistics, without gradient computation or fine-tuning. On the MUSE and TOFU benchmarks, our method achieves the lowest aggregate distance to the retrained reference among the compared methods in 4 bit precision. These results suggest that quantization, previously a threat to unlearning, can instead serve as an effective means of achieving it.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Unlearning, Quantization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PrMzOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper concerns the topic of machine unlearning where by a model is trained on a dataset and then one wants to remove knowledge obtained by the model on some subset of the dataset, known as the forget set. The natural question that develops from this motivation is what is a good way to remove that knowledge from the model without affecting the models ability outside the forget set. In this paper, the authors approach this problem through the lens of quantization. Their approach is to view quantization as an actual unlearning operator and in this way they develop a quantization map that enables the model to forget a subset of the dataset while keeping capability outside the forget set. They then validate their approach on standard unlearning benchmarks MUSE and TOFU.

Critical strengths
  1. The paper is written clearly with the authors taking the time to clearly explain the motivation of the problem and their methodology. One thing I really liked about the paper was the fact that the authors, in section 4, clearly explained how their quantization approach works and how it affects the forget and retain statistics (see 4.2).

  2. There is some novelty in this approach in that while it is natural to apply quantization after applying an unlearning operator it is very interesting to consider whether quantization can be used as an unlearning operator itself.

Critical weaknesses
  1. My main weakness with the paper was that the authors didn't really give any insight into why quantization can behave as an unlearning operator. They explain their method nicely in section 4 but why does this lead to decent unlearning results as shown in section 5? In some sense it is not surprising that such a method would work as unlearning itself can be thought of as trying to compress knowledge on the forget set, as perfect removal is probably impossible without high degradation on the retain set. However, understanding how their quantization as an unlearning operator does this via some analysis would have been much more insightful.

  2. The authors predominately focused on 4-bit quantization but it would have been good if they considered other bit quantization as a form of comparison and giving more granularity to their approach.

  3. I would have liked to see some limitations of the method especially since the paper doesn't explain the key driver of why this method seems to work.

Questions for authors
  1. Could you give some insight into why your method works?

  2. What happens when you try other post-training quantization methods possibly with other bit quantization?

13 comments

We thank the reviewer for the careful and constructive comments. Below we provide a point-by-point response. Where the reviewer requested additional evidence, we include new experiments: results of our method at 3-, 4-, 6-, and 8-bit precision, further quantization of our 8-bit model to 4-bit precision, and an implementation of our method on top of AWQ. Throughout, the retrained model (trained from scratch with the forget data removed) is the reference that unlearning aims to approximate, and NRD is our aggregate distance to this reference across the four metrics (KnowMem, VerbMem, PrivLeak, MU).

W1 / Q1: Why quantization can act as an unlearning operator

Quantization is a form of knowledge compression. It aims to compress the knowledge a model acquires during training into fewer bits while retaining it with minimal loss. This compression inevitably introduces quantization error. Prior quantization methods have accordingly focused on how to handle this error.

AWQ identifies salient weights from activation magnitudes and protects them by reducing their quantization error through scaling. GPTQ applies error compensation, which propagates the error from quantizing each weight to the remaining unquantized weights. This compensation keeps the layer output close to that of the original model. These two methods yield two insights. First, weights differ in their importance for retaining knowledge, and this importance can be identified from activation statistics. Second, controlling where quantization error remains and how it is compensated is central to knowledge compression.

We adopt both insights and extend each by distinguishing forget data from retain data.

Extending weight importance to forget data. We compute activation statistics separately on forget and retain data to identify forget-sensitive weights. We displace the codes of these weights away from their nearest codes, intentionally increasing their quantization error (code displacement).

Extending quantization error control to forget and retain data. The existing GPTQ compensation can also offset the code displacement. We therefore separate the error into the quantization error of the nearest code and the code displacement error, and control the two separately. Only the code displacement error is propagated along a direction whose forget-related component is attenuated. The layer output on retain data is therefore preserved, whereas the error on forget data remains.

Our method thus builds directly on the two insights by which conventional quantization handles error, but amplifies quantization error only on the forget side. Applying the compression of quantization asymmetrically to forget and retain data in effect implements the reviewer's view of unlearning as trying to “compress knowledge on the forget set.”

In conventional unlearning, forgetting is induced by small continuous parameter updates, which can be removed by rounding. In our method, forgetting is encoded as integer changes in the assigned codes and therefore persists after rounding. The ablation study in Table 3 of the paper also shows that both mechanisms are necessary. Replacing our compensation with the standard GPTQ compensation offsets the code displacement and thereby undoes the forgetting. Applying the same amount of code displacement to randomly selected weights severely degrades model utility.

This insight extends beyond GPTQ-based quantization. We applied it to AWQ and confirmed that AWQ-based quantization is also sufficient to achieve unlearning. We describe these results in detail in our response to Q2.

W2: Results at other bit widths

At 3-, 4-, 6-, and 8-bit precision, our method lowers the memorization metrics on the forget data to near the level of the retrained model. The full-precision target model has KnowMem and VerbMem of 57.89 and 64.43 on News and 47.12 and 99.70 on Books. In contrast, across all bit widths, our method yields KnowMem of 30.88–34.66 and VerbMem of 20.37–22.57 on News and 26.10–30.42 and 13.23–21.19 on Books. These values are close to those of the retrained model, which has KnowMem and VerbMem of 32.42 and 20.50 on News and 30.33 and 14.53 on Books. The NRD of our method also ranges from 0.07 to 0.17 at every bit width, lower than that of the target model (0.29 on News and 0.33 on Books).

At 3-bit precision, the quantization error itself is large, so even the retrained model reaches an NRD of 0.09 on News and 0.11 on Books from quantization alone, higher than at the other bit widths. Our method obtains the results above even at 3-bit precision, where the quantization error grows. We attribute this to its separate control of the ordinary quantization error through reconstruction-based compensation on retain data. We leave extending this principle to quantization methods that support extremely low precision of 3 bits or fewer for future work.

Results of our method on MUSE at 3-, 4-, 6-, and 8-bit precision.

News KnowMemNews VerbMemNews PrivLeakNews MUNews NRDBooks KnowMemBooks VerbMemBooks PrivLeakBooks MUBooks NRD
Retrain (FP)32.4220.50−0.0655.69–30.3314.53−0.1468.98–
Target (FP)57.8964.43−99.8155.520.2947.1299.70−57.3469.130.33
3bit retrain21.1520.32−10.4737.350.0918.5314.30−6.3241.910.11
4bit retrain34.6517.47−0.8243.370.0527.8514.14−4.6459.200.04
6bit retrain33.9420.50−0.5753.650.0129.3414.08−0.4869.880.01
8bit retrain31.7020.56−0.1755.250.0029.0214.12−0.3368.750.01
3bit ours30.8822.57−94.2735.320.1730.4221.19−57.3640.920.16
4bit ours32.9720.68−67.2037.300.1326.1016.034.1147.300.07
6bit ours34.6620.37−51.9142.660.1027.1313.23−59.1947.170.15
8bit ours31.2520.74−46.1838.290.1028.9317.18−55.3848.820.14

W3: Limitation under re-quantization of the unlearned model to a lower bit width

Our method constructs the unlearned model directly at a given bit width and thus removes the post-unlearning quantization stage. Quantizing this model again to a lower bit width, however, can reduce its agreement with the retrained model. The table below reports the results of further quantizing the 8-bit model obtained by our method to 4-bit precision.

Results on MUSE when the 8-bit model of our method is further quantized to 4-bit precision.

News KnowMemNews VerbMemNews PrivLeakNews MUBooks KnowMemBooks VerbMemBooks PrivLeakBooks MU
Ours 4-bit32.9720.68−67.2337.3026.1016.034.1147.30
Ours 8-bit → 4-bit12.1618.72−50.0031.6820.3216.23−58.3937.29

After the additional quantization, the memorization metrics stay at or below their 8-bit values, but the decrease in KnowMem and MU moves the model farther from the retrained model, raising NRD from 0.10 to 0.17 on News and from 0.14 to 0.19 on Books. Our method, however, quantizes a model to its final bit width in a single stage, so applying it directly at the desired bit width can avoid this degradation.

Q2: Other post-training quantization methods (AWQ)

To examine whether our method is limited to the sequential quantization of GPTQ-family methods, we implemented it on top of AWQ (Lin et al., 2024). Our method consists of two components.

  • (1) Code assignment: we displace the codes of forget-sensitive weights according to the intervention score.
  • (2) Error compensation: we compensate the error introduced by this displacement so that the layer output on retain data is preserved, while reducing compensation along forget-sensitive directions.

AWQ absorbs per-input-channel scales determined from activation statistics into the weights and then quantizes them with group-wise round-to-nearest (RTN), and unlike GPTQ, it has no per-weight error compensation step. We therefore implement (1) by applying the intervention score and the code displacement directly in the RTN step. For (2), we move the compensation to the group scales, which are the quantization parameters of AWQ. Specifically, with the codes fixed, we refit the group scales by least squares with respect to the retain statistic HR\mathbf{H}_R, and attenuate the forget-related component of this correction with the same attenuation factors as in the GPTQ version. We keep the scale and clipping search of AWQ unchanged. The table below reports the results on MUSE News.

Results on MUSE News with our method implemented on AWQ and GPTQ.

MethodKnowMemVerbMemPrivLeakMU
Retrain (FP)32.4220.50−0.0655.69
Target (4bit)55.9549.23−99.8149.49
Ours (AWQ 4bit)31.1025.89−95.4934.17
Ours (AWQ 8bit)32.1422.07−30.8129.95
Ours (GPTQ 4bit)32.9720.68−67.2337.30
Ours (GPTQ 8bit)31.2520.74−46.1838.29

Our method also removes knowledge of the forget data when built on AWQ. At 4-bit precision, KnowMem and VerbMem decrease from 55.95 and 49.23 under AWQ alone to 31.10 and 25.89, respectively, approaching the retrained reference, and NRD falls from 0.26 to 0.18. At 8-bit precision, the AWQ version attains an NRD of 0.11, similar to the 0.10 of the GPTQ version, and a PrivLeak of −30.81, closer to the retrained reference than the −46.18 of the GPTQ version. In contrast, MU is lower than that of the GPTQ version at both bit widths, and at 4-bit precision PrivLeak remains at the level of the target model (−95.49). We attribute this to the AWQ version performing compensation only by refitting the group scales. This compensation has fewer degrees of freedom than that of GPTQ, which updates all remaining unquantized weights. We leave refining the compensation scheme for the structure of AWQ to future work.

References

  • Lin et al. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. MLSys 2024.
Reviewer_5JivOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes performing machine unlearning directly during quantization. Starting from the original model, forget and retain activation statistics determine which integer codes to displace and the amount. The method separates ordinary rounding error from deliberate displacement error, applying modified GPTQ compensation to avoid undoing the intended forgetting. It requires forward calibration statistics without gradient-based unlearning or fine-tuning. On MUSE and TOFU, the resulting 4-bit models achieve the lowest normalized retrain distance compared to prior methods.

Critical strengths
  1. The formulation is interesting and practically motivated. Directly constructing an unlearned low-bit model removes the intermediate deployment transition that can erase continuous-space unlearning updates.
  2. Separating rounding and intervention errors is a coherent algorithmic contribution. The compensation ablation in Table 3 supports the importance of this distinction, particularly on NEWS.
  3. The experiments cover various benchmarks, including real-text and synthetic ones, and demonstrate promising results.
Critical weaknesses
  1. The comparison does not establish the best forgetting–utility trade-off of the baselines. The authors select MUSE baseline hyperparameters and checkpoints using VerbMem and KnowMem distance to retraining, explicitly without a utility constraint. The headline NRD also includes utility and privacy leakage.
  2. The remaining gaps are substantial in some cases. On NEWS, PrivLeak is -67.20, compared with -0.82 for the quantized retrained reference, so matching memorization scores does not imply comparable privacy behavior. Utility is also below the quantized retrained reference, particularly on NEWS and BOOKS.
  3. The forget specificity of the intervention needs stronger support. Equations 8–10 use activation-energy contrasts, while Equation 13 chooses displacement direction from the rounding residual. These choices do not directly establish that the impact is specific to the knowledge designated for removal.
Questions for authors
  1. How is intervention strength selected, and can the MUSE baselines be selected using the same validation criterion, or compared at matched retain utility?
  2. Could you report a intervention-strength sweep showing the individual forgetting, privacy, and utility metrics, rather than only the selected aggregate result?
  3. Would the advantage remain if forget/retain calibration labels were shuffled, keeping the calibration budget and intervention budget comparable?
6 comments

We thank the reviewer for the careful and constructive comments. Below we provide a point-by-point response. Where the reviewer requested additional evidence, we include new experiments: an intervention-strength (κ\kappa) sweep of our method on both datasets, a 35-configuration hyperparameter sweep of the representative baseline NPO+SURE, and a forget/retain label-shuffling control. Throughout, the retrained model (trained from scratch with the forget data removed) is the reference that unlearning aims to approximate, and NRD is our aggregate distance to this reference across the four metrics (KnowMem, VerbMem, PrivLeak, MU).

W1 / Q1 / Q2: Baseline forget–utility trade-off and selection of the intervention strength

The intervention strength κ\kappa is selected on the κ\kappa sweep using the same criterion as the baseline hyperparameters, namely the smallest normalized RMSE of KnowMem and VerbMem relative to the full-precision (FP) retrained reference; MU and PrivLeak are not used for selection. We will also add this κ\kappa-selection criterion to the appendix. The selection criterion is measured at full precision for GA, NPO, and SimNPO, which are designed for full-precision unlearning, and at 4-bit precision for the SURE variants and our method, which target 4-bit unlearning.

As κ\kappa increases, the memorization metrics decrease and PrivLeak moves toward the retrained reference, but MU decreases as well; the selected κ\kappa (in bold) is the point at which the memorization metrics are closest to those of the retrained model while NRD is also lowest.

To examine whether the comparison holds even when the baseline attains its best forgetting–utility trade-off, we also swept the representative baseline NPO+SURE over 35 hyperparameter configurations. On Books, all 35 configurations have PrivLeak and MU farther from the retrained model than those of our method. On News, for a comparison at matched retain utility (Wang et al., 2025), we examine the configurations whose MU is equal to or higher than that of our method. Of these nine configurations, eight have PrivLeak farther from the retrained model than that of our method. The remaining one (α=10\alpha=10, β=0.01\beta=0.01, in bold) has MU and PrivLeak similar to those of our method, but its memorization metrics are farther from the retrained model. Even when the baseline is selected with utility taken into account, no configuration is closer to the retrained model than our method on all four metrics.

News (κ\kappa sweep, ours)KnowMemVerbMemPrivLeakMUNRD
retrain (FP)32.4220.50−0.0655.69–
retrain 4-bit34.6517.47−0.8243.370.04
target (FP)57.8964.43−99.8155.520.29
κ=0\kappa=056.8046.90−99.8050.700.26
κ=1\kappa=152.3040.00−97.8048.500.23
κ=2\kappa=244.9039.50−95.0043.900.22
κ=4\kappa=436.4024.60−85.6038.800.16
κ=5\kappa=525.9020.70−75.1028.700.17
κ=6\kappa=629.7021.40−70.6026.100.17
κ=6.0156\kappa=6.015632.9720.68−67.2037.300.13
κ=8\kappa=827.1017.00−56.1030.000.15
κ=10\kappa=1023.4016.50−64.5024.200.19
κ=12\kappa=121.1010.40−24.903.900.26
κ=15\kappa=150.003.001.800.000.27
Books (κ\kappa sweep, ours)KnowMemVerbMemPrivLeakMUNRD
retrain (FP)30.3314.53−0.1468.98–
retrain 4-bit27.8514.14−4.6459.200.04
target (FP)47.1099.70−57.3069.100.33
κ=0\kappa=041.8091.50−58.2059.400.32
κ=2\kappa=238.4075.20−53.1056.000.28
κ=4\kappa=436.8054.00−69.1059.500.23
κ=8\kappa=831.5043.10−58.2058.800.18
κ=12\kappa=1224.1031.60−42.2052.600.16
κ=15\kappa=1532.5049.60−11.7051.400.15
κ=20\kappa=2025.6041.3016.2048.500.15
κ=25\kappa=2524.5018.9013.5047.600.10
κ=26.5625\kappa=26.562526.1016.034.1147.300.07
κ=30\kappa=3017.3019.50−3.0029.500.15
κ=40\kappa=4012.2010.4021.8028.400.19

References

  • Wang et al. Towards Effective Evaluations and Comparisons for LLM Unlearning Methods. ICLR 2025.

W1 / Q1 / Q2 (continued): NPO+SURE hyperparameter sweep

News (NPO+SURE sweep)

α\alphaβ\betaKnowMemVerbMemPrivLeakMU
retrain (FP)32.4220.50−0.0655.69
retrain (4-bit)34.6517.47−0.8243.37
Ours (4-bit)32.9720.68−67.2037.30
1.000.0127.5816.61−54.3432.66
0.0238.7020.27−80.0933.45
0.0513.5814.60−87.8517.20
0.1021.5215.76−96.9023.22
0.5020.8817.05−97.8621.39
2.000.0125.3317.56−61.4225.33
0.0238.8321.00−85.2938.03
0.0517.0915.57−87.8724.73
0.1025.0417.21−95.8728.96
0.5025.3717.34−97.7427.93
5.000.0123.0717.69−63.2028.94
0.0234.9719.62−84.4941.23
0.0517.1616.32−89.5023.89
0.1026.0415.63−95.7024.40
0.5028.7916.38−98.0926.66
10.000.0135.8817.26−61.6538.13
0.0234.6920.85−93.6136.95
0.0519.1416.13−89.4624.33
0.1016.1516.73−92.2023.29
0.5024.5017.64−97.4922.70
50.000.0136.3018.89−78.4239.01
0.0238.0221.11−94.1337.99
0.0518.1015.45−91.1621.15
0.1016.5116.07−93.1523.88
0.5030.8317.64−98.1127.26
100.000.0136.9419.85−80.6237.25
0.0230.0822.06−92.7740.51
0.0542.0323.68−99.4840.16
0.1017.8516.58−91.7924.66
0.5021.7717.24−98.0326.90
300.000.0142.3519.02−88.9837.24
0.0240.6622.29−94.8245.48
0.0537.7024.69−99.5043.96
0.1014.7315.29−92.0825.93
0.5020.1317.43−97.3028.40

Books (NPO+SURE sweep)

α\alphaβ\betaKnowMemVerbMemPrivLeakMU
retrain (FP)30.3314.53−0.1468.98
retrain (4-bit)27.8514.14−4.6459.20
Ours (4-bit)26.1016.034.1147.30
1.000.015.3911.98−60.8217.19
0.0220.9310.11−45.5631.81
0.0522.9214.50−51.2134.50
0.1013.9314.25−46.6022.60
0.5023.8314.51−56.9836.50
2.000.0127.8614.36−52.2037.01
0.0220.1913.14−51.5738.02
0.0520.797.61−49.7139.05
0.1015.9514.09−46.5834.63
0.5014.5314.60−57.3237.16
5.000.0113.7712.55−48.0432.35
0.0216.0513.68−53.3324.21
0.0518.4813.82−39.3121.66
0.1019.2614.65−54.0317.88
0.5021.3714.25−56.8545.43
10.000.0118.4513.70−47.0439.72
0.0214.938.41−40.0532.16
0.0521.0013.65−58.1126.28
0.1015.8314.47−45.2831.51
0.5025.2514.29−50.7128.66
50.000.0111.842.91−27.7622.35
0.029.8313.74−48.6015.90
0.0523.0914.17−58.2232.55
0.106.3914.35−42.933.25
0.5023.1014.07−61.5630.42
100.000.0131.1616.84−44.3544.95
0.0210.3913.21−26.6117.03
0.0520.1213.86−46.0226.11
0.105.9213.73−46.3610.02
0.5024.1614.09−58.5944.40
300.000.0131.1117.74−54.2945.26
0.0210.0113.49−62.6213.75
0.0518.4814.37−60.1923.93
0.1016.2614.56−50.0321.13
0.5015.5914.47−54.2432.88

W2: Remaining gaps to the retrained reference (PrivLeak and utility)

The retrained model is trained from scratch excluding the forget data and is the reference that unlearning ultimately aims to approximate. As the reviewer notes, our method brings the memorization metrics (KnowMem and VerbMem) close to those of the retrained model, but gaps to the retrained model remain in PrivLeak on News and in MU on both datasets. We also agree that closeness in memorization metrics does not by itself imply identical privacy behavior.

In a realistic unlearning setting, however, the goal is to avoid the cost of retraining, so the retrained model is unavailable and unlearning methods approximate it. We therefore take as our main criterion how much closer to the retrained model a method gets relative to baselines under the same 4-bit condition, where the baselines are the existing unlearning methods (GA, NPO, SimNPO) and the quantization-robust SURE variants.

Under this criterion, the comparison is as follows. The existing unlearning methods recover the forgotten knowledge after 4-bit quantization. On Books, all three methods show a large increase in VerbMem (e.g., the VerbMem of SimNPO grows from 16.35 to 91.08), and on News the VerbMem of GA and SimNPO also increases, from 18.91 to 30.77 and from 21.48 to 26.34, respectively. Only NPO on News shows small recovery; its PrivLeak of −61.54 is comparable to ours, but its MU of 33.09 is lower than our 37.30. The SURE variants suppress this recovery and keep the memorization metrics close to those of the retrained model, but their PrivLeak is far from that of the retrained model (−95.66 to −99.17 on News and −56.68 to −68.51 on Books).

Our method keeps the memorization metrics close to those of the retrained model, while its PrivLeak is also closer to that of the retrained model than those of the SURE variants on both datasets (−67.20 on News and 4.11 on Books). Its MU is higher than those of the SURE variants on News and lies in a similar range on Books (45.21–53.95). Our method therefore attains the lowest NRD among the 4-bit unlearning methods (0.13 on News and 0.07 on Books). Reducing the remaining gap to the retrained model, in particular in PrivLeak and MU on News, is a direction we will pursue in future work.

Reviewer_oQMBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Quantinue, a single-stage method that performs machine unlearning directly during post-training quantization. Motivated by prior observations that low-bit quantization can recover knowledge removed by continuous-space unlearning, the authors treat quantization itself as the unlearning operator. Building on GPTQ's sequential reconstruction framework, they introduce two modifications: (1) a score-guided code assignment that uses forget/retain activation statistics to displace codes of forget-sensitive weights away from their nearest reconstruction-driven codes, and (2) an unlearning-aware error compensation that separates ordinary quantization error from intentional code-displacement error and attenuates the propagation of the latter along forget-correlated directions. Both steps rely only on forward calibration statistics, without gradients or fine-tuning. On the MUSE and TOFU benchmarks at 4-bit precision, the method achieves the lowest normalized retrain distance (NRD) to the full-precision retrained reference among the compared approaches, including quantization-robust baselines such as SURE variants.

Critical strengths
  1. Novel and clean problem reformulation. Treating quantization itself as the unlearning operator elegantly removes the post-unlearning quantization stage where prior work has reported knowledge recovery. By decomposing the sequential quantization error and projecting the compensation direction away from forget-correlated subspaces, this single-stage view is both conceptually interesting and practically appealing for deployment.
  2. Strong efficiency and competitive empirical results. The method requires only forward calibration statistics (no gradients or fine-tuning) and produces a 4-bit model directly within two minutes. On both MUSE and TOFU, it achieves the lowest NRD to the full-precision retrained reference among the compared 4-bit methods, outperforming strong quantization-robust baselines such as SURE. The coarse-grained ablation study further confirms that score-guided selection and modified error compensation are mutually indispensable.
Critical weaknesses
  1. High degree of heuristic design without fine-grained justification. The element-wise intervention score and the modified compensation direction rely on multiple ad-hoc combinations (separate vs. joint statistics, min aggregation, four different normalization schemes in Table 4, and three attenuation factors). While coarse-grained ablations show that the overall scoring and compensation modules matter, these specific sub-component designs lack fine-grained ablation justification, making the method feel over-engineered and raising questions about generalization to other quantization schemes or model architectures.
  2. Limited experimental scope, remaining metric gaps, and sensitivity to the intervention strength κ\kappa. Results are reported only for 4-bit weight-only quantization on Llama-2-7B and Llama-3.2-1B. On MUSE, individual metrics (especially PrivLeak on NEWS at -67.20 and Model Utility on BOOKS dropping to 47.30 vs. 59.20 for 4-bit Retrain) still deviate noticeably from the retrained reference, which is partially masked by the aggregate NRD metric. Furthermore, the global intervention strength κ\kappa varies by over 60x across settings (from 0.4375 to 26.5625 in Table 7, tuned to four decimal places), making it unclear how κ\kappa can be reliably selected in realistic unlearning scenarios where a retrained reference model is unavailable. The paper also does not explore other bit-widths, activation quantization, sequential unlearning requests, or larger models.
Questions for authors
  1. How sensitive is the final NRD to the specific design choices in the intervention score (e.g., taking the minimum of the two input scores, the four distinct normalizations in Table 4) and to the three attenuation factors in the compensation direction? Have the authors explored simpler score or compensation variants that achieve comparable performance?
  2. Can the approach be extended beyond GPTQ-style sequential quantization (for example to AWQ, rotation-based methods, or joint weight-and-activation quantization), or to other bit-widths (such as 3-bit or 8-bit)? What is the sensitivity to block size and damping?
  3. In Table 7, the intervention strength κ\kappa varies drastically across benchmarks and models (from 0.4375 to 26.5625, specified to four decimal places). In practical unlearning scenarios where a retrained reference model is unavailable to compute NRD, how should practitioners select κ\kappa? Additionally, how sensitive is the performance to variations in κ\kappa and to the number of calibration sequences (currently fixed at 64)?
6 comments

We thank the reviewer for the excellent review. The major weaknesses and questions raised are grouped and addressed in the following order.

  1. Ablation study across different architectures
  2. Simpler variants of the score
  3. Various bit widths, a different architecture, a larger model, and sequential unlearning
  4. Extension to another quantization method (AWQ)
  5. Concern that NRD obscures PrivLeak and MU
  6. Hyperparameter sensitivity (κ\kappa, calibration set size, block size, and damping)
  7. Realistic unlearning scenarios

Ablation study across different architectures

To examine the effect of the detailed design choices, we conducted an additional fine-grained ablation study with ten variants, each of which removes or simplifies a single component of the intervention score or the error compensation. The experiments use News and Books with Llama-2-7B and News with Qwen-1.5-7B (see the tables below).

Several variants attain an NRD comparable to that of our method in individual settings, but no variant attains an NRD equal to or lower than that of our method in all three settings. Removing any component increases NRD in at least one setting, and the full configuration yields the most consistent results across all settings.

Ablation variants.

GroupVariantChange
Selectionsel_no_rowRemove the output score sˉout\bar{s}^{\mathrm{out}} (use the input score only)
sel_no_colRemove the input score sˉin\bar{s}^{\mathrm{in}} (use the output score only)
col_jointUse only sˉjoint\bar{s}^{\mathrm{joint}} instead of the min combination in the input score
col_balancedUse only sˉsep\bar{s}^{\mathrm{sep}} instead of the min combination in the input score
row_no_massRemove log⁡(1+uˉF,i)\log(1+\bar{u}_{F,i}) from the output score
Compensationrec_offRemove the entire pjp_j component (ℓjF=bj\ell^F_j = b_j)
no_rowgateRemove the row factor (1−sˉiout)(1-\bar{s}^{\mathrm{out}}_i)
no_omegaRemove the direction factor cos⁡2θj\cos^2\theta_j
no_piRemove the first term of ηt\eta_t, ∣ℓj,t∣/(∣pj,t∣+∣ℓj,t∣)\vert \ell_{j,t}\vert /(\vert p_{j,t}\vert +\vert \ell_{j,t}\vert )
no_domRemove the second term of ηt\eta_t, EˉtR/(EˉtF+EˉtR)\bar{E}^R_t/(\bar{E}^F_t+\bar{E}^R_t)

Ablation results on MUSE News with Llama-2-7B.

Ablation studyKnowMemVerbMemPrivLeakMUNRD
Retrain (FP)32.4220.50−0.0655.69–
ours32.9720.68−67.2037.300.13
rec_off29.4919.44−75.0636.640.15
no_omega34.1421.13−65.6236.720.13
no_rowgate36.0220.72−68.6035.390.14
no_pi31.0419.76−72.2333.220.15
no_dom33.5620.35−75.0832.850.15
sel_no_row33.5119.03−51.8936.550.12
sel_no_col31.3620.11−66.1631.260.14
col_joint28.3921.86−57.3339.400.12
col_balanced29.7020.76−71.9425.790.17
row_no_mass30.7121.21−80.3328.210.17

Ablation results on MUSE Books with Llama-2-7B.

Ablation studyKnowMemVerbMemPrivLeakMUNRD
Retrain (FP)30.3314.53−0.1468.98–
ours26.1016.034.1147.300.07
rec_off33.6615.0122.7146.000.10
no_omega29.9821.7212.3054.070.07
no_rowgate26.9613.18−19.3032.030.13
no_pi24.9919.667.2745.330.10
no_dom28.2421.09−71.7149.220.17
sel_no_row35.5419.35−9.3455.130.07
sel_no_col28.3417.91−80.2747.180.18
col_joint24.8122.53−65.9050.640.17
col_balanced31.7124.97−74.3953.460.17
row_no_mass27.3222.05−55.4447.730.15

Ablation results on MUSE News with Qwen-1.5-7B.

Ablation studyKnowMemVerbMemPrivLeakMUNRD
Retrain (FP)25.2219.380.0030.59–
ours25.3316.19−61.0624.830.10
rec_off25.2815.80−63.8427.990.09
no_omega21.0915.17−65.3326.380.11
no_rowgate29.8116.06−57.1624.890.10
no_pi25.1416.07−50.3019.950.09
no_dom22.9915.56−52.0518.600.11
sel_no_row22.9114.62−52.9920.080.11
sel_no_col25.3515.67−63.8125.420.10
col_joint23.9714.99−74.9322.910.12
col_balanced25.7917.89−66.2624.450.10
row_no_mass27.4218.33−77.5728.930.10

Simpler variants of the score

On MUSE Books (Llama-2-7B, 4-bit), we evaluated additional simplified variants that remove several components at once (see the table below). All variants use the same calibration data and the same compensation step, and κ\kappa is selected for each variant according to the same criterion as in the paper.

The core of our score is the ratio of the activation on the forget data to the cost incurred on the retain data. For the input-side retain cost, we use the second moment of the part of input coordinate jj that the remaining coordinates cannot linearly predict and that the GPTQ compensation therefore cannot recover (1/Ujj21/U^2_{jj}, Equation 28). The rows of the table are defined as follows.

  • A: Core ratio. This variant keeps only the core ratio and removes the magnitude weighting, the min combination with sseps^{\mathrm{sep}}, and the robust standardization.
  • B: Energy ratio. This variant replaces the input-side retain cost of A with the raw retain activation HR,jjH_{R,jj}, and constitutes the simplest form of a contrastive score.
  • Magnitude only. This variant constructs the score from the magnitude of the forget activation, without comparison to the retain data.
  • No intervention. This corresponds to κ=0\kappa = 0, i.e., quantization without any code displacement.
  • Retrain and Target. These denote the retrained reference and the model before unlearning, respectively.

Moved denotes the fraction of weights assigned to a code other than the nearest one, and quantifies the amount of intervention.

Simplified variants of the score on MUSE Books (Llama-2-7B, 4-bit).

Variantκ\kappaMovedKnowMemVerbMemPrivLeakMUNRD ↓\downarrow
Ours270.06728.9917.76−26.6744.120.120
A: Core ratio13.50.11911.8416.20−49.6543.700.192
B: Energy ratio270.06427.0323.03−59.8946.700.177
Magnitude only270.4741.911.98−20.460.670.312
No intervention0036.5095.75−58.2858.570.334
Retrain (reference)––30.3314.538.0168.98–
Target (before unlearning)––47.1299.70−57.3469.130.344

Various bit widths, a different architecture, a larger model, and sequential unlearning

Bit-width experiments

At 3-, 4-, 6-, and 8-bit precision, our method lowers the memorization metrics on the forget data to near the level of the retrained model. The full-precision target model has KnowMem and VerbMem of 57.89 and 64.43 on News and 47.12 and 99.70 on Books. In contrast, across all bit widths, our method yields KnowMem of 30.88–34.66 and VerbMem of 20.37–22.57 on News and 26.10–30.42 and 13.23–21.19 on Books. These values are close to those of the retrained model, which has KnowMem and VerbMem of 32.42 and 20.50 on News and 30.33 and 14.53 on Books. The NRD of our method also ranges from 0.07 to 0.17 at every bit width, lower than that of the target model (0.29 on News and 0.33 on Books).

At 3-bit precision, the quantization error itself is large, so even the retrained model reaches an NRD of 0.09 on News and 0.11 on Books from quantization alone, higher than at the other bit widths. Our method obtains the results above even at 3-bit precision, where the quantization error grows. We attribute this to its separate control of the ordinary quantization error through reconstruction-based compensation on retain data. We leave extending this principle to quantization methods that support extremely low precision of 3 bits or fewer for future work.

Results at 3, 4, 6, and 8-bit precision on MUSE.

News KnowMemNews VerbMemNews PrivLeakNews MUNews NRDBooks KnowMemBooks VerbMemBooks PrivLeakBooks MUBooks NRD
Retrain (FP)32.4220.50−0.0655.69–30.3314.53−0.1468.98–
Target (FP)57.8964.43−99.8155.520.2947.1299.70−57.3469.130.33
3bit retrain21.1520.32−10.4737.350.0918.5314.30−6.3241.910.11
4bit retrain34.6517.47−0.8243.370.0527.8514.14−4.6459.200.04
6bit retrain33.9420.50−0.5753.650.0129.3414.08−0.4869.880.01
8bit retrain31.7020.56−0.1755.250.0029.0214.12−0.3368.750.01
3bit ours30.8822.57−94.2735.320.1730.4221.19−57.3640.920.16
4bit ours32.9720.68−67.2037.300.1326.1016.034.1147.300.07
6bit ours34.6620.37−51.9142.660.1027.1313.23−59.1947.170.15
8bit ours31.2520.74−46.1838.290.1028.9317.18−55.3848.820.14

Generalization to other architectures

To examine whether our method works on a larger model and on a different model family, we conducted additional experiments on Llama-2-13B (MUSE News) and Qwen-1.5-7B (MUSE News and Books).

Results on MUSE News with Llama-2-13B.

ModelMethodNews KnowMemNews VerbMemNews PrivLeakNews MUNews NRD
Llama-2-13BRetrain32.7720.560.0039.49–
Target47.4693.41−100.0043.840.35
NPO35.2223.29−98.1435.810.14
Retrain (4bit)25.4320.83−1.5936.330.03
NPO + SURE (4bit)34.2425.62−100.0039.330.14
Ours (4bit)32.7821.27−84.1430.250.13

On Llama-2-13B at 4-bit precision, our method reduces KnowMem and VerbMem to 32.78 and 21.27, at the level of the 32.77 and 20.56 of the retrained model. Its PrivLeak of −84.14 is also closer to that of the retrained model than the −100.00 of NPO+SURE. Although its MU of 30.25 is lower than the 39.33 of NPO+SURE, its NRD of 0.13 is lower than the 0.14 of NPO+SURE.

Results on MUSE News and Books with Qwen-1.5-7B.

ModelMethodNews KnowMemNews VerbMemNews PrivLeakNews MUNews NRDBooks KnowMemBooks VerbMemBooks PrivLeakBooks MUBooks NRD
Qwen-1.5-7BRetrain25.2219.380.0030.59–31.2915.480.0057.66–
Target37.0877.19−100.0029.040.3039.9759.77−100.0061.320.28
Ours (4bit)25.3316.19−61.0624.830.1027.2813.09−69.4745.440.14

On Qwen-1.5-7B, our method also reduces the memorization metrics to values close to those of the retrained model, and lowers NRD from the 0.30 of the target model to 0.10 on News and from 0.28 to 0.14 on Books. These results show that our method can perform unlearning through quantization even when the model size and the model family differ.

Sequential unlearning

Sequential unlearning lies outside the scope of this work. We focus on the problem of performing unlearning within the quantization process itself for a single unlearning request. As the reviewer notes, however, unlearning requests can arrive sequentially in practical scenarios. We therefore regard the extension of our method to this setting as a meaningful direction for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.