acceptodds
Under review as a conference paper at ICLR 2027

NormSub: Normalization-Affine Subspace Adaptation for Quantization Recovery

Abstract

Low-bit quantization can substantially degrade model accuracy. Beyond improving quantizers and weight reconstruction, recovery requires understanding which parameters are useful to adapt under a limited budget. We investigate normalization-affine parameters as a structured, low-dimensional update subspace. In a fixed-graph study with matched parameter counts, data, and update budgets, affine updates achieve lower perplexity and higher mean accuracy than the tested weight-coordinate controls, with a small advantage over gradient-selected coordinates. Functional diagnostics further show that task-level recovery need not reduce every local reconstruction error. These findings motivate NormSub, a two-stage affine adaptation framework around a target post-training quantization (PTQ) backend. The pre-stage adapts only normalization-affine parameters on a disposable backend-aligned quantized proxy, then discards the proxy, restores the original floating-point weights, and transfers only the backend-compatible affine state for fresh PTQ reconstruction. The post-stage refines the affine coordinates exposed by the reconstructed graph while freezing its non-affine persistent state. Across the evaluated dense language models and PTQ backends, NormSub improves perplexity and mean zero-shot accuracy. On Qwen3-8B W2A4KV4 with QuaRot+ResComp, the unweighted six-task mean accuracy increases from 49.6% to 58.1%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Model Quantization, Post-Training Quantization, Large Language Models, Low-Bit Quantization, Model Compression, Normalization-Affine Adaptation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_cdBFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

NormSub studies normalization-affine parameters as a small update subspace for quantization recovery. Its full pipeline adapts a quantized proxy, transfers the learned affine state to a restored floating-point model under the backend's coordinate convention, reconstructs the quantized model, and adapts its affine parameters again. Both stages use task loss and distillation from the original floating-point teacher. Matched-coordinate controls and local-response diagnostics accompany experiments on dense language models and vision tasks; MoE and post-QAT experiments evaluate post-only adaptation. The reported gains are strongest at very low precision.

Critical strengths
  1. The dense-model results show useful recovery across several backends and model families. For Qwen3-8B with QuaRot+ResComp at W2A4KV4, the six-task mean increases from 49.6% to 58.1%, while both reported perplexities improve substantially (Table 6). The fixed learned-rotation extension in Table 10 adds evidence beyond the Hadamard-rotation setting.
  2. The coordinate controls are more informative than an unstructured comparison with an untuned model. Figure 3 matches trainable parameter count, data, objective and update budget across normalization and weight-coordinate subsets. Table 9 provides a separate vision control and includes relative timing. These results support normalization-affine adaptation as a useful finite-budget choice, without establishing universal superiority.
  3. The functional diagnostics distinguish local reconstruction error from downstream prediction quality. Figure 4 shows that useful output recovery need not improve every local error measure. The paper appropriately presents its Jacobian projection as a local interpretation rather than the exact objective solved by finite-step adaptation.
  4. Important qualifications are explicit: the ImageNet observer-recalibration exception, post-only MoE/QAT scope, a harmful post-QAT case, the extra stage's computational budget, and the absence of native low-bit deployment measurements. The final model retains no learned auxiliary inference branch, although backend fusion still requires checking.
Critical weaknesses
  1. The stage ablation does not isolate the benefit of reconstruction and transfer from additional optimization. Table 3 gives post-only adaptation 200 updates, whereas the full method receives 200 updates per stage plus a second PTQ construction. On Qwen3-8B W2A4KV4, the full method improves the mean from 56.47% to 58.05% over post-only; on Llama3-8B W3A16, post-only is slightly higher, 70.08% versus 70.05%. A longer post-only control and end-to-end construction time/peak-memory measurements are needed to judge the additional stage. The vision subset timing in Table 9 does not answer this dense-pipeline cost question.
  2. The experimental distinction from existing normalization-only recovery remains incomplete. Norm Tweaking is explicitly discussed, but no matched direct comparison is provided. The strongest dense-LLM coordinate control uses one model and setting, with a small advantage over TopGradW. Its candidate-pool size is unspecified. The local projection analysis does not itself establish that normalization coordinates outperform alternatives or explain the extra reconstruction stage's benefit.
  3. All experiments use one deterministic seed, with no reported uncertainty. This particularly limits interpretation of small differences over strong W3A16 baselines and TopGradW, even though it is less likely to explain the large W2A4KV4 recovery. The manuscript acknowledges this limitation; the smaller gains still need repeatability evidence.
  4. No reproduction code is supplied or linked. The protocol gives useful optimizer and library details, but backend-specific affine transfer remains difficult to reproduce, the ImageNet affine-stage epoch count is unclear, and the differing source runs behind Tables 3 and 6 lack an explicit run-to-table mapping. The acknowledged run reuse is not evidence of a numerical contradiction, but it prevents treating these tables as interchangeable ablations.
Questions for authors
  1. On the four Table 3 settings, how does 400-update post-only adaptation compare with the complete pipeline? Please report construction-inclusive time and peak memory alongside accuracy and perplexity, including the second PTQ pass.
  2. Can you compare against Norm Tweaking on one representative dense setting using the same backend, calibration data and stated adaptation budget? What candidate-pool size and selection procedure were used for TopGradW?
  3. How stable are the Figure 3 normalization-versus-TopGradW difference and the incremental Table 3 stage gains across a small number of calibration/adaptation seeds?
  4. Can you supply the affine-transfer implementation, the ImageNet/Table 9 adaptation budgets, and a run mapping that explains the different Llama3-8B W2A4KV4 baselines in Tables 3 and 6?
2 comments
AuthorsOpenReview ↗

Response to Reviewer cdBF (Part 1/2)

Thank you for asking whether the Pre stage earns its additional PTQ pass. On Qwen3-8B at W3A16, the answer changes with the backend. We give the paired comparison and its measured construction costs first, then distinguish the Norm Tweaking comparison from the experiment that isolates the training objective.

1. Post-400 versus Full: perplexity, accuracy, and construction cost (Q1 / W1)

At 400 affine updates, Post-only gives lower PPL on both corpora with GPTQ, while Full gives lower PPL on both with ResComp/GPTAQ. Table R2 shows the W3 comparisons for which model, data and evaluation are aligned (Qwen3-8B, group size 128, seed 0). Pre-only is measured after transferring the affine state and running fresh PTQ; the disposable proxy is not evaluated.

Table R2. WikiText-2 / C4-256 PPL (lower is better).

MethodW3A16 GPTQW3A16 ResComp/GPTAQ
Pre-200 + fresh PTQ10.74 / 16.4210.43 / 15.76
Pre-400 + fresh PTQ10.65 / 16.3310.36 / 15.80
Post-4009.74 / 15.3710.64 / 16.28
Full: Pre-200 + Post-20010.51 / 16.159.92 / 15.34

These are equal-update, not equal-compute, comparisons: Full adds a PTQ pass relative to Post-only. We exclude W2 from this stage ranking because its Pre/Full runs have not been matched to its Post runs. Our new PPL results cover four Qwen backend–precision settings, not every Qwen and Llama configuration requested from Table 3. Llama Post-400 and matched-budget AVG6 (mean accuracy over six zero-shot tasks) are still missing.

Table R3. Measured offline construction cost, Qwen3-8B, group size 128, one MetaX C500.

SettingPTQ per pass (min)Each 200-update affine phase (min)Full(200+200) total (min)Full construction peak GPU memory (GiB)
GPTQ W3A1655.01.6113.217.0
QuaRot+ResComp W2A4KV485.01.6173.225.0

Full starts from the floating-point checkpoint and includes proxy PTQ, Pre-200, fresh PTQ, and Post-200. The two affine phases take 3.2 minutes together; the additional PTQ pass, not the affine updates, dominates the extra cost. In the recorded runs, Full and its corresponding Post-200 had the same construction peak. We did not separately measure the full-process peak for Post-400. These are construction measurements for the two listed configurations, not inference benchmarks.

2. Norm Tweaking, the objective control, and TopGradW selection (Q2 / W2)

We added a paper-based sequential Norm Tweaking adapter. A separate comparison—Moment-400 versus Post-400—isolates the objective. Table R1 reports both on Qwen3-8B (g128, seed 0).

Table R1. WikiText-2 / C4-256 PPL (lower is better).

MethodW3A16 GPTQW3A16 ResComp/GPTAQW2A4KV4 QuaRot+GPTQW2A4KV4 QuaRot+ResComp/GPTAQ
PTQ baseline11.13 / 16.9611.49 / 17.2524.11 / 32.5223.21 / 31.21
NT-adapter11.04 / 16.9511.26 / 17.2923.48 / 32.0423.05 / 31.02
Moment-40010.84 / 16.9311.25 / 17.3019.86 / 27.4922.15 / 29.59
Post-2009.92 / 15.5810.13 / 15.5915.80 / 22.8116.12 / 22.69
Post-4009.74 / 15.3710.64 / 16.2814.99 / 21.9315.45 / 21.96

What the NT comparison controls. Following Algorithm 1, NT-adapter alternates block quantization with local normalization updates; the adapted prefix can affect later blocks. We use the common C4 calibration text instead of generated text and adapt only block-residual normalizations. Post/Moment also include final normalization and, for W3 only, Q/K normalization.

What the objective comparison controls. In each of the four columns, including W2, Moment and Post use the same quantized weights, teacher, examples and order, affine coordinates, optimizer settings, step count and evaluation. Moment matches channel moments at block outputs and final normalization; Post uses CE + teacher-logit KL. Post-400 has lower PPL on both corpora in all four settings. This supports the functional objective in these runs, not a claim that moment matching generally fails.

How TopGradW is selected. From the NN eligible attention/MLP projection coordinates we sample min⁡(8P,N)\min(8P,N) uniformly without replacement. Here P=299,008P=299{,}008, giving 2,392,064 candidates. On the first calibration block, we rank the candidate floating-point increments by absolute CE + teacher-KL gradient, train the top PP and keep that mask fixed. The 8P8P pool is for screening, not the number of trained scalars. Under the shared 200-update recipe, normalization-affine tuning is competitive with TopGradW. Figure 3 does not establish a ranking after independently tuning each subspace, and the local-Jacobian argument does not prove one.

AuthorsOpenReview ↗

Response to Reviewer cdBF (Part 2/2)

3. Are the small differences stable across seeds? (Q3 / W3)

All current results use seed 0, so we do not yet report across-seed standard deviations or paired significance for the smaller coordinate or stage differences. Post-200 and Post-400 are cumulative checkpoints along a single trajectory rather than independent replications (for W3 ResComp/GPTAQ, PPL shifts from 10.13/15.59 at 200 steps to 10.64/16.28 at 400 steps). Accordingly, we scope our current claim to empirical recovery under the stated protocol, and we commit to running multi-seed replications (3 independent seeds) for the Figure 3 coordinate comparisons and Table 3 stage ablations in the camera-ready revision to report mean and variance across seeds.

4. Vision budgets, and table provenance (Q4 / W4)

What counts as a vision update? On ImageNet, Pre and Post each use 200 optimizer steps (400 total), not epochs. “Full training set” names the data source rather than exhaustive dataset traversal. BN running statistics stay frozen, while observers are recalibrated as an explicit exception to the frozen non-affine state in LLMs. We will replace the epoch-based phrasing in the revision, and we commit to reconciling and documenting the exact observer recalibration frequency and per-branch Table 9 update budgets (alongside a small-calibration ImageNet PTQ control) in the camera-ready appendix.

Which table entries change? We apologize for two reporting errors and commit to completing the full provenance reconciliation in the camera-ready revision:

  • The Table 3 Llama W2 QuaRot+GPTAQ baseline is 23.0/62.8, not 18.07/50.56. It agrees with Table 6, and the appendix label “QuaRot+ResComp” will be corrected accordingly. Until the reconciled Llama W2 stage rows and matched-budget AVG6 evaluations are completed for the camera-ready revision, we do not derive stage gains from the unverified rows.
  • In Table 4, reevaluation of the complete Qwen W3 ResComp/GPTAQ Pre→PTQ→Post checkpoint gives 9.92/15.34 for Full. The duplicated 10.51/16.15 belongs to GPTQ Full, while ResComp/GPTAQ Post-400 is 10.64/16.28. We will update all affected tables, verify all derived averages, and include an explicit run-to-table provenance mapping in the camera-ready version.
Reviewer_xvEtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NormSub which is a parameter-efficient two-stage framework that recovers model performance after low-bit quantization by adapting normalization-affine parameters before and after PTQ reconstruction. The approach is motivated by a matched-budget comparison in which normalization-affine parameters recover more than equally sized weight-coordinate subsets. The method demonstrates consistent improvements across multiple models and quantization backends including an increase in mean zero-shot accuracy from 49.6% to 58.1% on Qwen3-8B W2A4KV4 with QuaRot+ResComp. Overall this work provides valuable insights into structured adaptation for quantization recovery although its computational overhead and the limited comparison with alternative parameter-efficient adaptation methods warrant further investigation.

Critical strengths
  1. The paper compares normalization-affine coordinates with equally sized weight-coordinate subsets under a matched adaptation budget (Figure 3). This experimentally shows that normalization-affine parameters are an effective subspace for quantization recovery.
  2. NormSub is applied on top of several PTQ backends such as AWQ/GPTQ/GPTAQ/ResComp. The complete pipeline consistently improves perplexity and mean zero-shot accuracy across these backends. This suggests that the method can serve as a general add-on to existing quantization methods.
  3. The paper is well organized and generally easy to follow.
Critical weaknesses
  1. Table 3 shows that most of the gain comes from PTQ + NormSub-Post. On Qwen3-8B W2A4KV4 the Avg6 increases from 49.62 to 56.47 with Post alone and only to 58.05 with the full pipeline. On Llama3-8B W3A16 Post alone is even slightly better (70.08 versus 70.05).
  2. The full pipeline requires two PTQ reconstructions and two affine adaptation stages. This likely adds substantial computational cost. However the paper reports no wall-clock time comparison for the full pipeline. It is therefore difficult to judge whether the pre-stage justifies its cost.
  3. The usefulness of normalization parameters for quantization recovery has already been demonstrated in prior work such as Norm Tweaking (Li et al. 2024). It is unclear what new insight this paper adds beyond that finding.
  4. (Minor) The figures are too small to read. Figure 4 in particular is nearly illegible.
Questions for authors
  1. Do the update dynamics of the normalization-affine parameters differ between the pre-stage and the post-stage? For example does initializing from the transferred pre-stage state help the post-stage converge to a better local minimum?
  2. Each stage uses 200 updates. How does the full pipeline compare with running 400 updates in the pre-stage or the post-stage alone? This would clarify whether the gain comes from the two-stage design or simply from additional updates.
  3. The dense LLM results are reported only for W3A16 and W2A4KV4. Are results available for other weight and activation settings such as W4A4?
  4. Could the authors report the end-to-end wall-clock time of the full pipeline compared with the PTQ backend alone?
2 comments
AuthorsOpenReview ↗

Response to Reviewer xvEt (Part 1/2)

Thank you for asking what Pre adds beyond Post, and at what cost. Our clearest result is a contrast: with 400 affine updates on Qwen3-8B W3A16, GPTQ favors Post alone, whereas ResComp/GPTAQ favors splitting the updates between Pre and Post. Below we distinguish this endpoint result from claims about how the optimizer gets there.

Q1. Does Pre lead Post to a better local minimum?

The experiments do not establish that. Pre learns affine parameters on a disposable quantized proxy. We discard its quantized weights and quantizer state, restore the original floating-point weights, transfer only compatible affine parameters, and run the target PTQ afresh. Post then starts a new optimizer on the resulting quantized model. Both stages use next-token cross-entropy (CE) and KL divergence to the frozen teacher’s predictions, but they do not form a continuous trajectory on a single loss surface. Fusion or rotation can also change which affine coordinates are exposed.

We measured final perplexity, not paired gradients, curvature, or stepwise optimization trajectories. Figure 4 addresses a different question in Post-only adaptation: output predictions can recover without every intermediate reconstruction error falling. It is not evidence for a better local minimum.

Q2. Does Full outperform Pre-only or Post-only at the same update budget?

Not on both backends. Table R2 reports aligned Qwen3-8B results (group size 128, seed 0; WikiText-2 / C4-256 perplexity, lower is better). Each Pre-only row evaluates the model after affine transfer and fresh PTQ, not the discarded proxy.

MethodW3A16 GPTQW3A16 ResComp/GPTAQ
Pre-200 + fresh PTQ10.74 / 16.4210.43 / 15.76
Pre-400 + fresh PTQ10.65 / 16.3310.36 / 15.80
Post-4009.74 / 15.3710.64 / 16.28
Full: Pre-200 + Post-20010.51 / 16.159.92 / 15.34

Pre-400, Post-400 and Full (200+200) all use 400 affine updates. Full improves on Pre-400 on both corpora for both backends. Compared with spending all 400 updates on Post, however, Full improves both PPLs for ResComp/GPTAQ and worsens both for GPTQ.

Q3. Do the additions cover other dense-LLM settings such as W4A4?

We do not have new dense-LLM W4A4 results completed within the rebuttal window, and we commit to adding dense W4A4 benchmarks in the camera-ready revision.

Our current dense-LLM evaluation was designed to bracket two contrasting precision boundaries: mild weight-only quantization (W3A16, where initial grids remain largely intact) and extreme joint weight-activation-KV quantization (W2A4KV4, where quantization degradation and recovery headroom are largest). We agree that vision W4A4, post-QAT, and MoE experiments operate under different task protocols and cannot substitute for dense-LLM W4A4. For the camera-ready version, we will evaluate NormSub-Post and NormSub-Full on dense W4A4 (covering Qwen3-8B and Llama3-8B under our unified C4-calibrated protocol with WikiText-2/C4 PPL and AVG6) to fill in this intermediate activation-quantized regime.

Q4. What are the time and memory costs of the complete procedure?

Table R3 gives the recorded offline construction time and peak GPU memory on one MetaX C500, Qwen3-8B, group size 128. Full starts from the original floating-point checkpoint and includes proxy PTQ, Pre-200, fresh PTQ, and Post-200.

SettingPTQ per pass (min)Each 200-update affine phase (min)Full(200+200) total (min)Full construction peak GPU memory (GiB)
GPTQ W3A1655.01.6113.217.0
QuaRot+ResComp W2A4KV485.01.6173.225.0

The two affine phases together take 3.2 minutes. The second PTQ pass—not the affine updates—accounts for most of Full’s additional time relative to a single PTQ pass. Full and the corresponding Post-200 had the same recorded construction peak in these runs. These measurements concern construction, not inference latency.

AuthorsOpenReview ↗

Response to Reviewer xvEt (Part 2/2)

W3. What is new relative to Norm Tweaking?

We credit Li et al., Algorithm 1 for using normalization updates for quantization recovery. Their procedure quantizes and adapts successive blocks using local channel means and variances; an adapted prefix can affect later PTQ. Our question is what can be recovered by optimizing a fixed set of affine parameters against the model’s final predictions, and when transferring those parameters before fresh PTQ helps beyond Post alone. Figure 3 shows that affine tuning is competitive with gradient-selected weight updates under the shared training recipe; it does not establish a best subspace after independent tuning.

Table R1 uses the same model, group size and seed as R2. W3 = W3A16; W2 = W2A4KV4. Entries are WikiText-2 / C4-256 PPL.

MethodW3 GPTQW3 ResComp/GPTAQW2 QuaRot+GPTQW2 QuaRot+ResComp/GPTAQ
PTQ baseline11.13 / 16.9611.49 / 17.2524.11 / 32.5223.21 / 31.21
NT-adapter11.04 / 16.9511.26 / 17.2923.48 / 32.0423.05 / 31.02
Moment-40010.84 / 16.9311.25 / 17.3019.86 / 27.4922.15 / 29.59
Post-2009.92 / 15.5810.13 / 15.5915.80 / 22.8116.12 / 22.69
Post-4009.74 / 15.3710.64 / 16.2814.99 / 21.9315.45 / 21.96

NT-adapter compares workflows. It replaces generated text with C4 calibration text and updates block-residual normalization only. Post/Moment also update final normalization and, in W3 only, Q/K normalization. Layer-local passes are not end-to-end updates; we have not verified an equivalent total budget. The NT-to-Post difference is therefore not an objective-only effect.

Moment-400 versus Post-400 isolates the objective. Within each column they share starting quantized weights, teacher, calibration data and order, trainable affine coordinates, optimizer settings, update count, and evaluation. Moment matches block-output and final-normalization moments; Post minimizes CE + teacher KL. Post-400 yields lower PPL on both corpora in all four columns. More Post updates do not always help: W3 ResComp/GPTAQ is better at 200 than at 400.

Corrections and figure readability (W4)

We will correct Table 4: the reevaluated Qwen W3 ResComp/GPTAQ Full checkpoint gives 9.92/15.34, not the GPTQ Full value (10.51/16.15) or ResComp/GPTAQ Post-400 (10.64/16.28). The Table 3 Llama W2 QuaRot+GPTAQ baseline is 23.0/62.8, not 18.07/50.56; we will correct the backend labeling. We will enlarge Figure 4’s labels and separate its crowded panels.

Reviewer_CmGEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper NormSub: Normalization-Affine Subspace Adaptation for Quantization Recovery examines the recovery of accuracy in low-bit quantized models under a limited adaptation budget. The central argument is that normalization-affine parameters, like RMSNorm scales, BatchNorm scales and biases, constitute an effective, structured, low-dimensional correction subspace. This argument hinges on the evidence that on Qwen3-8B at W2A4KV4, training only these parameters beats training the same number of randomly chosen, gradient-selected, attention-only or MLP-only weights, under the same data and update budget. Building on this, the authors propose NormSub as a two-stage method around an existing PTQ backend. In the Pre-stage, the model is quantized normally with a PTQ method (for instance, GPTQ) and normalization-affine parameters are trained on this quantized model so that they learn to compensate for the quantization error. Subsequently, so that PTQ starts from quantization-aware affine parameters, PTQ is re-run from the original weights with the learned affine values. In the Post-stage, the normalization-affine parameters are retrained on this newly quantized model to correct remaining quantization errors.

Experiments are conducted on 7–8B Llama and Qwen models at W3A16 and W2A4KV4 across several PTQ backends. Additional experiments cover MoE models, QAT checkpoints and two vision tasks. The authors highlight Qwen3-8B at W2A4KV4, where mean accuracy rises from 49.6% to 58.1% (abstract; Tables 3 and 6).

Critical strengths
  1. Simple and practical method: Only a small set of parameters is trained, no inference overhead is added, and the proposed method works on top of existing PTQ backends.
  2. Significant gains at very low bit widths: The performance improvements are big as well as consistent across models and backends. At W2A4KV4, NormSub substantially improves both models in Table 3. For Llama3-8B and Qwen3-8B, WikiText-2 perplexity drops from 18.07 to 12.31 and from 18.71 to 12.03, respectively, whereas C4 perplexity drops from 50.56 to 28.86 and from 38.27 to 25.37. Moreover, from Table 3, mean accuracy rises by 6.6 points on Llama3-8B and by 8.4 points on Qwen3-8B. Furthermore, Tables 6 and 10 in the appendix demonstrate that gains at W2A4KV4 hold across all the tested model-backend pairs, covering both QuaRot and SpinQuant rotations with GPTAQ and ResComp.
  3. Broad and transparent evaluation: NormSub is evaluated on seven models (three dense LLMs, two MoE models, MobileNetV2 and U-Net), six PTQ backends, and four task types (language modelling, zero-shot reasoning, classification and segmentation). In Sec. 3.4 of the paper, the authors clearly state the scope of the method and also acknowledge its extra offline cost, a second PTQ pass and two adaptation stages, with no added inference overhead. The appendix gives enough detail to reproduce the work.
Critical weaknesses
  1. The existing work Norm Tweaking (Li et al., 2024) for LLM PTQ recovers accuracy in extreme low-bit regimes. The paper under review discusses it in Related Work (Sec. 2) but never compares against it, so the added value over this closest prior work is not clear to the reader.
  2. From the stage ablation reported in Table 3, the value of the Pre-stage (NormSub-Pre) is unclear. The results in Table 3 show that post-adaptation alone gives most of the gain: for Qwen3-8B at W2A4KV4, it reaches 56.47% mean accuracy against 58.05% for the full pipeline, and for Llama3-8B at W3A16 it is slightly better (70.08% vs. 70.05%). However, this comparison is uneven because the full pipeline uses 400 adaptation updates (200 per stage) plus an extra PTQ pass, while post-only uses 200 updates. As stated in Sec. 4.1, budgets are matched per stage, not in total. A post-only run with 400 updates is therefore needed to tell whether the remaining gain comes from the pre-stage itself or simply from more training.
  3. Sec. 3.4 acknowledges the extra cost of NormSub (two PTQ passes, two adaptation stages, etc.), but no runtime or memory figures are given.
  4. One result reported in the main table appears to be inconsistent. In Table 4, NormSub(GPTQ) and NormSub(ResComp*) for Qwen3-8B report identical perplexities (10.51 / 16.15) despite different backends.
  5. The main paper shows W3A16 (Table 4), where gains are smallest, while the full W2A4KV4 table with the headline result is in the appendix. Figures 2b and 4 are too small to read, and the relative quality metric in Figure 3a is not defined.
Questions for authors

NormSub is a practical method with strong results at very low bit widths. However, the novelty over Norm Tweaking (Li et al., 2024) is not demonstrated, and the Pre-stage is not shown to be worth its extra cost. I currently rate it marginally below the acceptance threshold. That said, I would raise my score if the rebuttal addresses the following questions:

  1. How does NormSub compare with Norm Tweaking (Li et al., 2024) under the same settings?
  2. How does post-only adaptation with 400 updates (matching the total number of updates) compare with the full pipeline? What are the runtime and memory costs (see Weakness 3)?
  3. Can you explain or correct the inconsistency in Table 4, where NormSub(GPTQ) and NormSub(ResComp*) for Qwen3-8B report identical perplexities (10.51 / 16.15) despite different backends?
4 comments
AuthorsOpenReview ↗

Response to Reviewer CmGE (Part 1/2)

Thank you for asking what NormSUb adds to Norm Tweaking, whether Pre is worth its cost once Post has the same update budget, and which entries in the original tables are correct. We address these in turn.

Q1. What does NormSub add beyond Norm Tweaking? (W1)

Updating normalization parameters is not itself our contribution. Norm Tweaking uses sequential blockwise quantization and local normalization updates. NormSub-Post instead trains the permitted affine parameters jointly, on the final quantized graph, using next-token CE and KL to a frozen teacher's output. NormSub-Full also transfers the Pre affine state back to a compatible graph with the original floating-point weights and runs fresh PTQ before Post. Figure 3 asks whether these affine coordinates are an effective choice under a common training recipe, not whether they are an optimal subspace after independent tuning.

Table R1. Qwen3-8B, group size 128, seed 0. Entries are WikiText-2 / C4-256 perplexity (lower is better).

MethodW3A16 GPTQW3A16 ResComp/GPTAQW2A4KV4 QuaRot+GPTQW2A4KV4 QuaRot+ResComp/GPTAQ
PTQ baseline11.13 / 16.9611.49 / 17.2524.11 / 32.5223.21 / 31.21
NT-adapter11.04 / 16.9511.26 / 17.2923.48 / 32.0423.05 / 31.02
Moment-40010.84 / 16.9311.25 / 17.3019.86 / 27.4922.15 / 29.59
Post-2009.92 / 15.5810.13 / 15.5915.80 / 22.8116.12 / 22.69
Post-4009.74 / 15.3710.64 / 16.2814.99 / 21.9315.45 / 21.96

There are two distinct comparisons in this table:

  • Workflow (NT-adapter vs. Post). Our adapter follows Norm Tweaking's Algorithm 1; an adapted prefix can affect later blocks during PTQ. We use the same C4 calibration text rather than generated text, and the adapter updates block-residual normalizations only. W3 Post/Moment also update Q/K and final normalization; W2 freezes Q/K but includes final normalization.
  • Objective (Moment-400 vs. Post-400). Within each column, including W2, these runs use the same initial quantized backbone, teacher, calibration examples and order, affine coordinates, optimizer settings, update count, and evaluation. Only the loss changes: block-output and final-normalization channel moments versus CE + teacher KL. Post-400 has lower PPL on both corpora in every column. On W2 QuaRot+ResComp/GPTAQ, for example, WikiText-2 / C4 drops from 22.15 / 29.59 to 15.45 / 21.96. This supports the functional objective under this protocol; it does not establish that moment matching necessarily overfits.

Q2. With 400 affine updates available, when is Pre worth the extra PTQ pass? (W2/W3)

It depends on the backend. On W3 GPTQ, Post-400 is better on both corpora than splitting the budget 200+200. On W3 ResComp/GPTAQ, the split is better on both. Pre is therefore an option whose benefit should justify its cost, not a necessary component of recovery.

Table R2. Aligned W3 stage comparison; metrics as in R1. Pre-only is evaluated after affine transfer and fresh PTQ, not on the disposable proxy.

MethodW3A16 GPTQW3A16 ResComp/GPTAQ
Pre-200 + fresh PTQ10.74 / 16.4210.43 / 15.76
Pre-400 + fresh PTQ10.65 / 16.3310.36 / 15.80
Post-4009.74 / 15.3710.64 / 16.28
Full: Pre-200 + Post-20010.51 / 16.159.92 / 15.34

Pre-400, Post-400, and Full (200+200) use 400 affine updates each, not equal compute. Full beats Pre-400 for both W3 backends, but does not consistently beat Post-400.

These are Qwen PPL results from seed 0. Post-200 and Post-400 are checkpoints on one trajectory, not independent runs; more steps do not always help. The original Llama W3 Post-only/Full accuracy values (70.08% / 70.05%) likewise do not favor Full. We lack Llama Post-400 and matched-budget AVG6 and do not extrapolate to them.

Table R3. Measured offline construction cost. Qwen3-8B, group size 128, one MetaX C500. Full includes proxy PTQ, Pre-200, fresh PTQ, and Post-200.

SettingPTQ per pass (min)Each 200-update affine phase (min)Full(200+200) total (min)Full construction peak GPU memory (GiB)
GPTQ W3A1655.01.6113.217.0
QuaRot+ResComp W2A4KV485.01.6173.225.0

The two affine phases together take 3.2 minutes; omitting the second PTQ pass would substantially understate Full's cost. The measured Full peak equaled that of the corresponding Post-200 run in each setting. We did not separately measure the full-process peak for Post-400. These are construction measurements for the listed settings, not inference latency.

AuthorsOpenReview ↗

Response to Reviewer CmGE (Part 2/2)

Q3. Which table entries and figure definitions will change? (W4/W5)

  • Table 4 (Qwen W3). We apologize for duplicating the GPTQ Full value in the ResComp Full row. Reevaluation of the complete Pre → PTQ → Post checkpoint gives 9.92 / 15.34 for NormSUb(ResComp*); GPTQ Full remains 10.51 / 16.15. ResComp Post-400 is 10.64 / 16.28, a different checkpoint. This reevaluation verifies PPL, not accuracy.
  • Tables 3 and 6 (Llama W2). The baseline in Table 3 is 23.0 / 62.8, not 18.07 / 50.56. Both tables use QuaRot+GPTAQ; the appendix label “QuaRot+ResComp” will be corrected. Until the complete stage rows and AVG6 are reconciled, we do not use the old rows to infer stage gains or claim that all derived conclusions are unaffected.
  • Presentation. We will move representative W2 results into the main text and enlarge Figures 2b and 4 and their labels. In Figure 3a, relative PPL = 100 × PPL(Frozen) / PPL(method), whereas relative accuracy = 100 × AVG6(method) / AVG6(Frozen). WikiText-2, C4, and AVG6 are normalized separately; Frozen = 100 and higher is better for each. They are not averaged into a new score. We will clarify the caption and retain absolute metrics.

The detailed rebuttal and new experiments have addressed some of my main concerns. In particular, the clarified distinction between Norm Tweaking’s blockwise, moment-matching updates and NormSub-Post’s joint optimisation of the final quantised graph with CE + teacher KL seems convincing to me. The Moment-400 vs. Post-400 ablation is useful; it holds the backbone, data and update count fixed and varies only the loss. Post-400 clearly yields lower perplexity in every setting. This addresses my main concern, so I am raising my score to 3.

However, after reviewing the authors' rebuttal, I would like to note the following:

(i) The new comparisons report perplexity only, on Qwen3-8B with a single seed; zero-shot accuracy and a second model family would strengthen the claim.

(ii) Table R2 shows that the Pre-stage does not consistently outperform Post-400 for the same number of updates, while it roughly doubles the construction time reported in Table R3. I suggest that the authors clearly state this in the main paper and present Pre as optional.

AuthorsOpenReview ↗

Response to Reviewer CmGE

Thank you very much for carefully reviewing our rebuttal and for raising your score. We sincerely appreciate your constructive feedback and the time you have invested in evaluating our work.

We agree with both of your remaining suggestions and will incorporate them into the camera-ready revision:

  1. Expanded Evaluation: We will extend the controlled stage comparisons to include zero-shot accuracy (AVG6), a second model family (Llama3-8B), and multi-seed replications. We will report the corresponding mean and standard deviation to provide a more complete assessment of the observed gains.

  2. Clarified Method Positioning: We will explicitly discuss the backend-dependent trade-off between Full and Post-only adaptation, together with their offline construction costs, in the main paper. In particular, we will present the Pre-stage as an optional refinement whose benefit depends on the PTQ backend and computational budget, rather than as a universally necessary component.

Thank you again for your constructive suggestions. They have helped us improve both the experimental rigor and the clarity of our paper.

Reviewer_1xGhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that normalization-affine parameters form an effective low-dimensional subspace for quantization recovery. Affine updates beat random, gradient-selected, attention-only, and MLP-only weight coordinates. NormSub is a two-stage method: First tune affine parameters on a disposable quantized proxy and transfer only the affine state into a fresh PTQ run. Then refine it post-quantization. It improves perplexity and accuracy across several backends and models, as well as vision and MoE/QAT settings.

Critical strengths
  1. The method is simple, adds no inference overhead, and is backend-agnostic.
  2. Broad experimentation over multiple domains with matched-budget comparisons.
  3. The limitations section is clear and honest.
Critical weaknesses
  1. The full method needs two PTQ passes plus 2×200 optimization steps with teacher distillation. Post-only gets most of the gain on some settings. ImageNet adaptation uses the full training set, which is a very different regime from the PTQ baselines which are often only given a small calibration set which may not even be in-domain.
  2. The local Jacobian argument applies to any parameter subset and does not explain why affine is special. Also, in rotated backends the original norm scales are folded into projections, so the "normalization" coordinates are effectively new per-channel scales, which blurs the distinction from learned diagonal transforms.
  3. The idea is close to Norm Tweaking and PTQ4VM. There is no direct empirical comparison against Norm Tweaking despite citing the method in the related works.
Questions for authors
  1. How do results change if each subspace gets its own tuned learning rate?
  2. Can you report multiple seeds and out-of-domain perplexity (e.g., calibrate on C4, evaluate on WikiText-2)?
  3. Can you compare against Norm Tweaking and an equal-budget LoRA?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 1xGh

Thank you for asking what Figure 3 actually establishes and how much of the recovery transfers beyond the calibration data. We distinguish three questions below.

Q1. Would the coordinate ranking hold with a separately tuned learning rate for each subspace?

Figure 3 asks a practical question: on the same quantized backbone, with the same calibration data, CE + teacher-logit KL objective, optimizer settings, 299,008 trainable scalars, and 200 updates, how much recovery do we obtain by changing which coordinates are trained? Under this shared recipe, normalization-affine tuning is competitive with TopGradW. We did not tune each subspace’s learning rate separately. The small observed gap therefore does not establish a ranking at independently tuned optima. We will revise the text and caption to describe an effective recovery subspace under the tested conditions, not an optimal one.

For TopGradW, we sample 2,392,064 eligible attention/MLP projection coordinates uniformly without replacement (8P8P). On the first calibration block, we rank candidate floating-point increments by absolute loss gradient and retain P=299,008P=299{,}008 coordinates. The mask stays fixed. Thus 8P8P is the screening pool, not the trainable budget or an exhaustive search over weights.

Q2. What evidence is there across corpora and seeds?

We have a cross-corpus result, but not independent seeds. Using C4 training text for calibration, W3A16 Post-400 reduces WikiText-2 test PPL from 11.13 to 9.74 on GPTQ and from 11.49 to 10.64 on ResComp/GPTAQ. This establishes transfer in this particular setting, not general out-of-distribution robustness. All results use seed 0. Post-200 and Post-400 are snapshots of one trajectory: on W3 ResComp/GPTAQ, PPL changes from 10.13/15.59 to 10.64/16.28, so more updates do not always help.

Where should the updates be spent? Table R2 reports WikiText-2 / C4-256 PPL (lower is better). Pre-only is evaluated after transferring the learned affine parameters back and running fresh PTQ, not on the proxy.

MethodW3A16 GPTQW3A16 ResComp/GPTAQ
Pre-200 + fresh PTQ10.74 / 16.4210.43 / 15.76
Pre-400 + fresh PTQ10.65 / 16.3310.36 / 15.80
Post-4009.74 / 15.3710.64 / 16.28
Full: Pre-200 + Post-20010.51 / 16.159.92 / 15.34

Among the three 400-update rows, GPTQ favors Post-only and ResComp/GPTAQ favors Full on both corpora.

What did Full cost? Table R3 reports measured offline construction time and peak GPU memory for Qwen3-8B, group size 128, on one MetaX C500. Full includes proxy PTQ, Pre-200, fresh PTQ, and Post-200.

SettingPTQ per pass (min)Each 200-update affine phase (min)Full(200+200) total (min)Full construction peak GPU memory (GiB)
GPTQ W3A1655.01.6113.217.0
QuaRot+ResComp W2A4KV485.01.6173.225.0

The two affine phases together take 3.2 minutes; the second PTQ pass accounts for most of Full’s additional time.

Q3. How does this compare with Norm Tweaking and equal-budget LoRA?

We evaluated a paper-based Norm Tweaking adapter. Norm Tweaking (Alg. 1) quantizes blocks sequentially and locally adjusts normalization using channel means and variances. An adapted prefix can affect the quantization of later blocks. Our NT-adapter uses our C4 calibration text instead of generated text and updates block-residual normalization only. W3 Post/Moment additionally include Q/K and final normalization; W2 freezes Q/K but includes final normalization.

Table R1. Qwen3-8B, g128, seed 0. WikiText-2 / C4-256 PPL ↓. W3 = W3A16; W2 = W2A4KV4.

MethodW3 GPTQW3 ResComp/GPTAQW2 QuaRot+GPTQW2 QuaRot+ResComp/GPTAQ
PTQ baseline11.13 / 16.9611.49 / 17.2524.11 / 32.5223.21 / 31.21
NT-adapter11.04 / 16.9511.26 / 17.2923.48 / 32.0423.05 / 31.02
Moment-40010.84 / 16.9311.25 / 17.3019.86 / 27.4922.15 / 29.59
Post-2009.92 / 15.5810.13 / 15.5915.80 / 22.8116.12 / 22.69
Post-4009.74 / 15.3710.64 / 16.2814.99 / 21.9315.45 / 21.96

The objective-only comparison is Moment-400 vs. Post-400. Within each column the two runs share the quantized weights, teacher, calibration examples and order, trainable coordinates, optimizer settings, update count, and evaluation. Moment matches block-output and final-normalization moments; Post uses CE + teacher-logit KL. Post yields lower PPL on both corpora in all four configurations.

We credit Norm Tweaking and PTQ4VM for earlier affine calibration. Our contribution is to test which coordinates recover accuracy under a fixed recipe, how local error relates to output recovery, and when affine transfer before fresh PTQ adds value beyond Post alone.

Comments on the paper

1 comment
Area_Chair_fMCwOpenReview ↗

Engage in discussion

The authors and reviewers are requested to engage in the discussion phase.

Authors: please respond to the review comments.

Reviewers: please review the authors’ rebuttal and consider revising your assessment/score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.