- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
NormSub studies normalization-affine parameters as a small update subspace for quantization recovery. Its full pipeline adapts a quantized proxy, transfers the learned affine state to a restored floating-point model under the backend's coordinate convention, reconstructs the quantized model, and adapts its affine parameters again. Both stages use task loss and distillation from the original floating-point teacher. Matched-coordinate controls and local-response diagnostics accompany experiments on dense language models and vision tasks; MoE and post-QAT experiments evaluate post-only adaptation. The reported gains are strongest at very low precision.
Critical strengths
- The dense-model results show useful recovery across several backends and model families. For Qwen3-8B with QuaRot+ResComp at W2A4KV4, the six-task mean increases from 49.6% to 58.1%, while both reported perplexities improve substantially (Table 6). The fixed learned-rotation extension in Table 10 adds evidence beyond the Hadamard-rotation setting.
- The coordinate controls are more informative than an unstructured comparison with an untuned model. Figure 3 matches trainable parameter count, data, objective and update budget across normalization and weight-coordinate subsets. Table 9 provides a separate vision control and includes relative timing. These results support normalization-affine adaptation as a useful finite-budget choice, without establishing universal superiority.
- The functional diagnostics distinguish local reconstruction error from downstream prediction quality. Figure 4 shows that useful output recovery need not improve every local error measure. The paper appropriately presents its Jacobian projection as a local interpretation rather than the exact objective solved by finite-step adaptation.
- Important qualifications are explicit: the ImageNet observer-recalibration exception, post-only MoE/QAT scope, a harmful post-QAT case, the extra stage's computational budget, and the absence of native low-bit deployment measurements. The final model retains no learned auxiliary inference branch, although backend fusion still requires checking.
Critical weaknesses
- The stage ablation does not isolate the benefit of reconstruction and transfer from additional optimization. Table 3 gives post-only adaptation 200 updates, whereas the full method receives 200 updates per stage plus a second PTQ construction. On Qwen3-8B W2A4KV4, the full method improves the mean from 56.47% to 58.05% over post-only; on Llama3-8B W3A16, post-only is slightly higher, 70.08% versus 70.05%. A longer post-only control and end-to-end construction time/peak-memory measurements are needed to judge the additional stage. The vision subset timing in Table 9 does not answer this dense-pipeline cost question.
- The experimental distinction from existing normalization-only recovery remains incomplete. Norm Tweaking is explicitly discussed, but no matched direct comparison is provided. The strongest dense-LLM coordinate control uses one model and setting, with a small advantage over TopGradW. Its candidate-pool size is unspecified. The local projection analysis does not itself establish that normalization coordinates outperform alternatives or explain the extra reconstruction stage's benefit.
- All experiments use one deterministic seed, with no reported uncertainty. This particularly limits interpretation of small differences over strong W3A16 baselines and TopGradW, even though it is less likely to explain the large W2A4KV4 recovery. The manuscript acknowledges this limitation; the smaller gains still need repeatability evidence.
- No reproduction code is supplied or linked. The protocol gives useful optimizer and library details, but backend-specific affine transfer remains difficult to reproduce, the ImageNet affine-stage epoch count is unclear, and the differing source runs behind Tables 3 and 6 lack an explicit run-to-table mapping. The acknowledged run reuse is not evidence of a numerical contradiction, but it prevents treating these tables as interchangeable ablations.
Questions for authors
- On the four Table 3 settings, how does 400-update post-only adaptation compare with the complete pipeline? Please report construction-inclusive time and peak memory alongside accuracy and perplexity, including the second PTQ pass.
- Can you compare against Norm Tweaking on one representative dense setting using the same backend, calibration data and stated adaptation budget? What candidate-pool size and selection procedure were used for TopGradW?
- How stable are the Figure 3 normalization-versus-TopGradW difference and the incremental Table 3 stage gains across a small number of calibration/adaptation seeds?
- Can you supply the affine-transfer implementation, the ImageNet/Table 9 adaptation budgets, and a run mapping that explains the different Llama3-8B W2A4KV4 baselines in Tables 3 and 6?
2 comments
Response to Reviewer cdBF (Part 2/2)
3. Are the small differences stable across seeds? (Q3 / W3)
All current results use seed 0, so we do not yet report across-seed standard deviations or paired significance for the smaller coordinate or stage differences. Post-200 and Post-400 are cumulative checkpoints along a single trajectory rather than independent replications (for W3 ResComp/GPTAQ, PPL shifts from 10.13/15.59 at 200 steps to 10.64/16.28 at 400 steps). Accordingly, we scope our current claim to empirical recovery under the stated protocol, and we commit to running multi-seed replications (3 independent seeds) for the Figure 3 coordinate comparisons and Table 3 stage ablations in the camera-ready revision to report mean and variance across seeds.
4. Vision budgets, and table provenance (Q4 / W4)
What counts as a vision update? On ImageNet, Pre and Post each use 200 optimizer steps (400 total), not epochs. “Full training set” names the data source rather than exhaustive dataset traversal. BN running statistics stay frozen, while observers are recalibrated as an explicit exception to the frozen non-affine state in LLMs. We will replace the epoch-based phrasing in the revision, and we commit to reconciling and documenting the exact observer recalibration frequency and per-branch Table 9 update budgets (alongside a small-calibration ImageNet PTQ control) in the camera-ready appendix.
Which table entries change? We apologize for two reporting errors and commit to completing the full provenance reconciliation in the camera-ready revision:
- The Table 3 Llama W2 QuaRot+GPTAQ baseline is 23.0/62.8, not 18.07/50.56. It agrees with Table 6, and the appendix label “QuaRot+ResComp” will be corrected accordingly. Until the reconciled Llama W2 stage rows and matched-budget
AVG6evaluations are completed for the camera-ready revision, we do not derive stage gains from the unverified rows. - In Table 4, reevaluation of the complete Qwen W3 ResComp/GPTAQ Pre→PTQ→Post checkpoint gives 9.92/15.34 for Full. The duplicated 10.51/16.15 belongs to GPTQ Full, while ResComp/GPTAQ Post-400 is 10.64/16.28. We will update all affected tables, verify all derived averages, and include an explicit run-to-table provenance mapping in the camera-ready version.
Response to Reviewer cdBF (Part 1/2)
Thank you for asking whether the Pre stage earns its additional PTQ pass. On Qwen3-8B at W3A16, the answer changes with the backend. We give the paired comparison and its measured construction costs first, then distinguish the Norm Tweaking comparison from the experiment that isolates the training objective.
1. Post-400 versus Full: perplexity, accuracy, and construction cost (Q1 / W1)
At 400 affine updates, Post-only gives lower PPL on both corpora with GPTQ, while Full gives lower PPL on both with ResComp/GPTAQ. Table R2 shows the W3 comparisons for which model, data and evaluation are aligned (Qwen3-8B, group size 128, seed 0). Pre-only is measured after transferring the affine state and running fresh PTQ; the disposable proxy is not evaluated.
Table R2. WikiText-2 / C4-256 PPL (lower is better).
These are equal-update, not equal-compute, comparisons: Full adds a PTQ pass relative to Post-only. We exclude W2 from this stage ranking because its Pre/Full runs have not been matched to its Post runs. Our new PPL results cover four Qwen backend–precision settings, not every Qwen and Llama configuration requested from Table 3. Llama Post-400 and matched-budget AVG6 (mean accuracy over six zero-shot tasks) are still missing.
Table R3. Measured offline construction cost, Qwen3-8B, group size 128, one MetaX C500.
Full starts from the floating-point checkpoint and includes proxy PTQ, Pre-200, fresh PTQ, and Post-200. The two affine phases take 3.2 minutes together; the additional PTQ pass, not the affine updates, dominates the extra cost. In the recorded runs, Full and its corresponding Post-200 had the same construction peak. We did not separately measure the full-process peak for Post-400. These are construction measurements for the two listed configurations, not inference benchmarks.
2. Norm Tweaking, the objective control, and TopGradW selection (Q2 / W2)
We added a paper-based sequential Norm Tweaking adapter. A separate comparison—Moment-400 versus Post-400—isolates the objective. Table R1 reports both on Qwen3-8B (g128, seed 0).
Table R1. WikiText-2 / C4-256 PPL (lower is better).
What the NT comparison controls. Following Algorithm 1, NT-adapter alternates block quantization with local normalization updates; the adapted prefix can affect later blocks. We use the common C4 calibration text instead of generated text and adapt only block-residual normalizations. Post/Moment also include final normalization and, for W3 only, Q/K normalization.
What the objective comparison controls. In each of the four columns, including W2, Moment and Post use the same quantized weights, teacher, examples and order, affine coordinates, optimizer settings, step count and evaluation. Moment matches channel moments at block outputs and final normalization; Post uses CE + teacher-logit KL. Post-400 has lower PPL on both corpora in all four settings. This supports the functional objective in these runs, not a claim that moment matching generally fails.
How TopGradW is selected. From the N eligible attention/MLP projection coordinates we sample min(8P,N) uniformly without replacement. Here P=299,008, giving 2,392,064 candidates. On the first calibration block, we rank the candidate floating-point increments by absolute CE + teacher-KL gradient, train the top P and keep that mask fixed. The 8P pool is for screening, not the number of trained scalars. Under the shared 200-update recipe, normalization-affine tuning is competitive with TopGradW. Figure 3 does not establish a ranking after independently tuning each subspace, and the local-Jacobian argument does not prove one.