acceptodds
Under review as a conference paper at ICLR 2027

The Misallocation of Precision in Quantized Language Models

Abstract

Mixed-precision quantization must decide where to spend a fixed budget of full-precision weights, and the criteria in use are derived from activation statistics: GPTQ minimises layer-wise error under the Hessian , ResQ keeps that covariance's highest-variance subspace at 8 bits. We test the premise underneath, that numerical sensitivity tracks functional importance, by taking its natural per-layer summary, , as a cross-layer allocation rule. In Llama-3.1-8B it is not merely uncorrelated with functional importance but *inverted*: Spearman (, ). The sub-layers it ranks highest sit in a functional dead zone; those it ranks lowest are load-bearing. The cost is measurable: at an identical 6/32 FP16 budget and identical attention/MLP composition, allocating by Hessian rank recovers less of the damage 4-bit quantization does to ARC-Challenge ( vs. , ), and leaves the model significantly worse than FP16 where the functionally-chosen budget does not. We establish this by treating 4-bit NF4 quantization as a controlled perturbation probe, analogous to lesion studies in neuroscience: quantize globally, restore full precision to one sub-layer, measure task recovery. A 64-sub-layer sweep localises the effect to *named sub-layers* rather than to components as classes. Against a do-nothing null measured on the same examples (ARC exactly 0/468) with BH-FDR correction, L13–14 attention clear the null on GSM8K and MATH but on neither measure of ARC, while L6–7 MLP clear it on all three; the component-level contrast is itself not significant (). Split-half selection shows the ranking beats a random equal-budget subset on ARC (pp, 95% CI [, ]) while carrying 4.6–12.2pp of selection optimism. The inversion replicates on Mistral-7B and under GPTQ, and the Llama-3.1-8B-derived layer set transferred to Mistral-7B unchanged recovers of its ARC damage against for Mistral-7B's own Hessian-top layers, though offset by collateral damage elsewhere, so it transfers as a diagnostic rather than an allocator. A surgical model keeping 6 of 32 sub-layers at full precision reaches FP16 equivalence at 42% memory, yet neither Hessian sensitivity nor weight-space SQNR (, ns) predicts which six.

Primary area:
interpretability and explainable AI
Keywords:
quantization, mixed-precision quantization, post-training quantization, mechanistic interpretability, hessian sensitivity, layer-wise sensitivity, large language models, model compression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LtD8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a method to identify and protect layers in mixed-precision quantization settings. The authors claim that the Hessian of the layer-wise optimization problem used in GPTQ and ResQ is uninformative for ranking the sensitivity of the layers. As a result they present a functional importance metric in order to rank which layers most affect model performance.

The main contributions that the authors claim are:

  1. The local hessian (XX^T), the signal used in GPTQ and ResQ, is uninformative about the importance of the a layer.
  2. They propose a method to find important layers: They take the quantized model and restore one sub-layer at a time (MLP or Attention), then they evaluate the model on a calibration subset of the benchmarks and rank the sub-layers according to the restored accuracy.
  3. They achieve iso-accuracy with FP16 model using less memory.
  4. Functional importance is positional and shared across independently trained models.

Validation:

  1. They show the negative correlation between local hessian (XX^T) and restoration rank. This result is presented in Figure 3.
  2. The method presented in section 3.
  3. In Table 2 they show that with REST-6 the accuracy of ARC can be improved and reach close to iso-accuracy.
  4. They showed that for Mistral-7B which shares the same number of layers, that putting the same transformer blocks in FP16 that made the Llama-8B model robust also makes the Mistal-7B model more robust.
Critical strengths

None

Critical weaknesses
  1. The method does not deliver: REST-6 puts six out of 32 transformer blocks in FP16. As a result, the accuracy would be expected to improve since less quantization error is added into the model. But the method improves only on one benchmark ARC and is still worse than GPTQ with all the layers in 4-bit. (Table1 and Table 2)

  2. There is a presentation issue regarding the Hessian. GPTQ and ResQ use the Hessian of a layer-wise optimization objective. This local Hessian (2XX^T) and depends only on the activations not the weights. In both works the local Hessian is used within a layer and not to rank layers across the model. Furthermore, this signal seems to be informative within a layer as GPTQ achieves iso-accuracy with the FP16 model (Table 1).

  3. The comparison of REST-6 and HESS-6 is unfair: REST-6 is task supervised where HESS-6 uses WikiText-2 for calibration.

Questions for authors

None

4 comments
AuthorsOpenReview ↗

What the paper is for, and why accuracy barely moves on GSM8K and MATH

"The method does not deliver: REST-6 puts six out of 32 transformer blocks in FP16. As a result, the accuracy would be expected to improve since less quantization error is added into the model. But the method improves only on one benchmark ARC and is still worse than GPTQ with all the layers in 4-bit. (Table1 and Table 2)"

We should have been clearer about the claim, and the revision is. REST-6 is a diagnostic. The object that carries the paper's argument is the ranking, and REST-6 is the construction that tests whether the ranking means anything at a fixed budget. It is not proposed as a deployable allocator, it is not tuned to beat GPTQ, and we agree that it does not. Removing that impression from the framing is one of the changes in the revision.

On the specific comparison with GPTQ-INT4, one measurement in our own Table 1 argues against reading that row as a precision effect at all: GPTQ-INT4 scores 85.4 on GSM8K against FP16's 80.2, 5.2pp above the unquantized model. Reducing precision cannot add capability. The GPTQ row is a different artifact: a separately released checkpoint (hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4), not our base model put through GPTQ. The gap most likely reflects how its calibration interacts with the chat template and answer extractor. Nothing in the paper's argument rests on that row, and the revision marks it as not budget-comparable to the NF4 arms. The internally controlled comparison is REST-6 against HESS-6: same base model, same quantizer, same 788.5M FP16 parameters, same 2-attention/4-MLP composition, differing only in which six positions are chosen.

On the expectation that accuracy should improve: it does not, and the reason is measurable rather than a matter of interpretation. Quantization damage is bidirectional. Restoring precision repairs forward flips, but it also removes the reverse flips, the examples the FP16 model gets wrong and the 4-bit model happens to get right.

DatasetFP16 → NF4 forward flips ↓Reverse flipsNetNet accuracy loss ↓
GSM8K4836122.4pp
MATH474071.4pp
ARC-Challenge3614224.4pp

On GSM8K and MATH the two channels nearly cancel, so there is almost no net accuracy for any allocation to recover, whatever it protects. On ARC the reverse channel is small, net damage is 4.4pp, and repairing forward flips shows up in accuracy. That is why the accuracy-level result appears on ARC only, and it is a property of where quantization damage is visible rather than of REST-6. We state it in the revision as a limit on what a 4-bit accuracy benchmark can resolve, not as a result in our favour.

AuthorsOpenReview ↗

Which Hessian, at what scope, and over which input

"There is a presentation issue regarding the Hessian. GPTQ and ResQ use the Hessian of a layer-wise optimization objective. This local Hessian (2XX^T) and depends only on the activations not the weights. In both works the local Hessian is used within a layer and not to rank layers across the model. Furthermore, this signal seems to be informative within a layer as GPTQ achieves iso-accuracy with the FP16 model (Table 1)."

We agree with you on every point. VnmZ raises the same objection, so the full answer, with the new HAWQ-V2 experiment, is in our comment there (W1). In short: 2XX⊤2XX^\top is the Hessian of GPTQ's layer-wise reconstruction objective, it depends on activations only, and it is used within a layer for error compensation at a uniform bit-width, not to rank layers. Using it as a cross-layer allocation criterion and attributing that use to GPTQ and ResQ was our error, and the claim has been withdrawn. We implemented HAWQ-V2's actual cross-layer metric and report there that it could not be estimated reliably enough at 8B to rank sub-layers (seed-to-seed rank agreement 0.832 against a required 0.90, an asymptote of 0.841, top-6 overlapping across probe seeds at 3 of 6).

On your last sentence we agree and want to make the agreement precise, because it also explains how two things that look contradictory can both be true. GPTQ's within-layer use of 2XX⊤2XX^\top is informative, since it does reach near-FP16 accuracy, and the restoration probe applied to the GPTQ model itself still finds the same sub-layers: under GPTQ quantization, L14 attention is the top attention sub-layer on GSM8K at 46.4% recovery, and L6 and L7 MLP are again among the top MLP sub-layers on ARC. Error compensation inside a layer and functional importance across layers are different questions, and a criterion can answer the first well while saying nothing about the second. That is the distinction the paper should have drawn explicitly, and the revision draws it.

One further precision, since it bears directly on scope and LTNt asks about it too (their Q4). Our trace is accumulated by a forward hook on each self_attn and mlp module over its input hidden_states, the post-RMSNorm residual stream entering the sub-layer (d = 4096 in both cases), pooled over all 262,144 token positions of the 128 × 2048 WikiText-2 calibration corpus, as Tr(H)/d=E[∥x∥2]/d\mathrm{Tr}(H)/d = \mathbb{E}[\lVert x \rVert^2]/d. That is one number per sub-layer, covering the input to q/k/v_proj and to gate_proj/up_proj. It does not cover o_proj, whose input is the attention output, or down_proj, whose input is the SwiGLU activation. GPTQ builds a separate H per projection matrix. Our per-sub-layer scalar is therefore a strictly coarser object than what GPTQ computes, which is your point stated in our own notation, and the revision says so where the metric is defined.

AuthorsOpenReview ↗

We ran the control. The asymmetry is real, and it is not what produces the gap.

"The comparison of REST-6 and HESS-6 is unfair: REST-6 is task supervised where HESS-6 uses WikiText-2 for calibration."

We agree with you on the facts, and we should have flagged the asymmetry ourselves. REST-6 is selected from recovery measured on the task benchmarks; HESS-6 is selected from Tr(H)/d\mathrm{Tr}(H)/d accumulated over WikiText-2 test, 128 × 2048, following the GPTQ calibration protocol. The two arms did not see the same information.

So we removed the asymmetry. We recomputed the trace from activations collected on the task prompts themselves (the same 500 GSM8K, 500 ARC and 500 MATH prompts the evaluation uses), re-selected the top six at the same composition and FP16 budget, and re-ran the head-to-head. The pre-registration, committed before any compute, names the outcome in which your objection holds and the paper's comparison does not survive. The code selected the outcome from the thresholds, not us.

The positive control first, because nothing below means anything without it. Recomputing the trace on WikiText-2 with the new code reproduces the published ranking at rank agreement 1.0000, Spearman ρ −0.4969, and re-selects exactly the published HESS-6. (This needed float16 rather than bfloat16: the published attention cut is decided by a 0.607% margin.)

Task calibration does not weaken the inversion. It slightly strengthens it.

Trace computed onρ vs restoration ↑ρ vs depth
WikiText-2 (published)−0.4969+0.663
GSM8K prompts−0.4746+0.642
ARC prompts−0.5182+0.697
MATH prompts−0.5465+0.757
Pooled over all three−0.5145+0.699

The re-selected set, HESS-6-task = attn:25, attn:27, mlp:27–30, differs from the published HESS-6 by a single sub-layer. Evaluated at 788,529,152 FP16 parameters in both arms, the identical budget and composition, on the primary endpoint:

ARCFP16NF4HESS-6-taskREST-6
accuracy0.7880.7440.7360.774
recovery of the 39 flipped——2/39 = 5.1% [0.6, 17.3]21/39 = 53.8% [37.2, 69.9]

Paired difference +48.7pp, bootstrap 95% CI [+30.8, +66.7], REST-6 ahead in all 10,000 resamples; McNemar rest_vs_hess p = 0.0017. HESS-6-task is significantly worse than FP16 (p = 0.0008), as the published HESS-6 was.

The second column is why no calibration corpus could have fixed this. The trace's correlation with depth goes up under task calibration, not down: +0.663 on WikiText-2, +0.699 pooled, +0.757 on MATH. Tr(H)/d\mathrm{Tr}(H)/d here is largely a function of where a sub-layer sits in the residual stream, so changing what you feed the model moves it by about 4% of a correlation unit. This is the same object LTNt's Q1 is about, measured from a second direction.

The design, the positive control and these limits are Appendix O of the revision.

Four things we will not claim from this run, and state in the paper.

  1. GSM8K separates nothing: rest_vs_hess p = 0.7794. We pre-registered ARC as primary precisely because GSM8K carries too little net quantization damage to resolve (forward/reverse flips 36/14 on ARC against 48/36 on GSM8K). We do not cite GSM8K as support.
  2. HESS-6-task's attention half is not uniquely determined. Its cut margin is 0.099%, and across corpora the attention picks permute among L22, L25 and L27 while the MLP half never leaves mlp:27–30. What holds is not those two layers but the cluster: every member of it has 0.0000 single-layer ARC recovery and a bottom-quartile restoration rank (1, 2, 6, 7, 12, 14 of 64) against REST-6's 55, 60, 61, 62, 63 and 64.
  3. Collateral breakage persists. On ARC, REST-6 repairs 21 and breaks 7 (net +14); HESS-6-task repairs 2 and breaks 6 (net −4). The mechanistic claim holds; the practical claim remains ARC-only.
  4. This equalises the corpus, not the objective. Restoration measures exact-match recovery while the trace measures mean squared activation; the symmetric comparison needs a Hessian supervised by the same correctness signal. That is the task-loss Hessian of our answer to your W2, which landed on "instrument not validated" (seed-to-seed rank agreement 0.832 against a pre-registered 0.90, three sub-layers with non-positive average trace). This run neither inherits nor repairs that, and it does not touch your granularity objection.

A calibration-free criterion is precisely what makes Hessian-guided allocation attractive: the appeal of Tr(H)\mathrm{Tr}(H) is that it needs no labels. Had the criterion ranked layers correctly only once given task supervision, that would itself be a result about the criterion. It does not.

Reviewer_LtD8OpenReview ↗

Response to Authors

We thank the authors for the responses. The authors explain that the method is a diagnostic tool and not a deployable allocator. This diagnostic tool identifies which layers are more sensitive to quantization, but when they restore these layers with FP16 precision the improvement is only on one task, even though they select these layers using the same benchmarks. As a results, this diagnostic tool has not been show to be useful. Furthermore, the method does not have a strong baseline comparison. For these reasons we maintain our original score.

Reviewer_LTNtOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates whether numerical sensitivity can identify which sub-layers are important for maintaining the accuracy of quantized language models. The authors first quantize an LLM to 4-bit NF4 and then restore each attention and MLP sub-layer to FP16, one at a time, measuring how many examples damaged by quantization (correct under FP16 but wrong under 4-bit) are recovered. They compare this restoration-based ranking with Hessian sensitivity, measured as the per-layer activation-covariance trace, and find the two are inversely correlated (ρ = −0.497 on Llama-3.1-8B). Based on these results, they select six sub-layers to retain at FP16 (REST-6) and compare them with six sub-layers selected by Hessian sensitivity (HESS-6) under the same precision budget. On ARC-Challenge, REST-6 recovers 53.8% of the quantization damage, against 7.7% for HESS-6. The authors also replicate the findings on Mistral-7B and under GPTQ quantization, and show that keeping the six selected sub-layers at FP16 matches FP16 accuracy at 42% of its memory.

Critical strengths

(1) The paper questions whether Hessian-based numerical sensitivity accurately reflects the functional importance of sub-layers.

(2) The approach is simple and intuitive: quantize the model, restore one sub-layer at a time to FP16, and measure how many incorrect answers are recovered.

(3) Under the same precision budget, restoration-guided selection recovers substantially more quantization-damaged examples than Hessian-guided selection on ARC-Challenge (53.8% vs. 7.7%).

(4) The layer-wise analysis reveals task-specific recovery patterns, with L13–L14 attention showing strong recovery on GSM8K and L6–L7 MLP across multiple tasks.

(5) The evaluation includes significance tests, confidence intervals, multiple-testing correction, and split-half validation.

(6) The findings are examined across different models and quantization methods, including Mistral-7B and GPTQ.

(7) The paper is clearly written and provides code and experimental results to support reproducibility.

Critical weaknesses

(1) The negative correlation is largely driven by layer depth and becomes statistically insignificant after controlling for it (ρ = −0.212, p = 0.096).

(2) The same examples are used to select and evaluate important sub-layers. Although split-half validation shows 4.6–12.2 percentage points of selection optimism, the main REST-6 versus HESS-6 comparison is not repeated with independently selected layers.

(3) REST-6 significantly improves overall accuracy over HESS-6 only on ARC-Challenge. No significant improvement is observed on GSM8K or MATH and the selected layers do not improve overall accuracy when transferred to Mistral-7B.

(4) The paper evaluates activation-based Hessian sensitivity and weight-space SQNR separately but does not test their combined effect through activation-weighted quantization error.

(5) Some reporting inconsistencies remain: §4.3 reports 25 significant MLP sub-layers on GSM8K, while Appendix B states that none passes the FDR threshold. The related work also omits sensitivity-guided mixed-precision methods such as HAWQ or HAWQ-V2.

Questions for authors

(1) Controlling for depth makes the inversion non-significant . How much of the result comes from activation size itself rather than layer depth?

(2) Does REST-6 still outperform HESS-6 when the six layers are chosen on one set of examples and evaluated on a different set?

(3) Activation size and weight error are tested separately. Did you test a measure that combines both and does it still rank layers in reverse? Why does REST-6 improve overall accuracy on ARC but not on GSM8K, MATH or Mistral?

(4) For an attention or MLP sub-layer, which input is used to compute the activation trace

6 comments
AuthorsOpenReview ↗

Depth and the trace are not separable in this model, and the test that survives

"Controlling for depth makes the inversion non-significant. How much of the result comes from activation size itself rather than layer depth?"

Thank you. This is the sharpest statistical question we received, and the honest answer is that the partial correlation the paper reports is not a well-posed test of what it looks like it tests. In Llama-3.1-8B the activation-covariance trace is very nearly a monotone function of depth:

RelationshipSpearman ρp
Trace vs restoration (pooled)−0.4973.0 × 10⁻⁵
Trace vs depth, all 64+0.6632.3 × 10⁻⁹
Trace vs depth, within MLP only+0.9955.8 × 10⁻³¹
Trace vs depth, within attention only+0.8554.9 × 10⁻¹⁰
Restoration vs depth−0.5502.5 × 10⁻⁶

Within the MLP sub-layers the trace and the layer index are the same ranking to within one swap. Partialling depth out of the trace therefore removes almost all of the trace's variance, not a confound in it, and the residual −0.212 (p = 0.096) is close to asking what is left of the trace after removing the trace. We will replace the partial correlation in the paper with this statement, because it is the more informative one: as a ranking criterion, Tr(H)/d\mathrm{Tr}(H)/d on this model is approximately "deeper is more sensitive", and a criterion that is a restatement of depth cannot be carrying layer-specific information.

The test that does separate the two is the one that holds depth exactly constant. At each layer index there are two sub-layers, an attention and an MLP, at identical depth. Ask, at each depth, whether the higher-trace member is the lower-restoration member:

Test, depth held exactly constantResult
Discordant attention/MLP pairs24 of 32
Binomial, one-sided ↓p = 0.0035

Three quarters of the matched pairs are locally inverted, with depth contributing nothing. The inversion also survives within each component class separately, where depth still varies but the attention/MLP contrast is removed: attention ρ = −0.526 (p = 0.0020, n = 32), MLP ρ = −0.561 (p = 0.00084, n = 32), Fisher-z mean −0.544. By depth band it is −0.013 (ns) early (L0–L10), −0.353 (p = 0.107) mid (L11–L21) and −0.439 (p = 0.053) deep (L22–L31), i.e. carried by the deeper half.

To answer your question in one sentence: on this model we cannot attribute the pooled −0.497 to activation size as distinct from depth, because the two are 85–99% collinear; what we can show is that with depth held exactly fixed the higher-trace sub-layer is the less restorable one in 24 of 32 cases.

AuthorsOpenReview ↗

Selection on one half, evaluation on the other

"Does REST-6 still outperform HESS-6 when the six layers are chosen on one set of examples and evaluated on a different set?"

Yes, and this is the right thing to ask given your weakness (2); we had run it and it belongs in the paper rather than only in the repository. We split the flipped examples in half at random, select the top six by recovery on one half, and score those six on the other half, over 1000 splits. HESS-6 is fixed by the Hessian so it does not move, and we score it on the same held-out halves. We also report a composition-matched HESS-6, so the comparison is not carried by 2-attention/4-MLP versus the Hessian's all-attention selection.

DatasetREST-6 ↑HESS-6 ↑HESS-6 (comp.-matched) ↑Difference ↑95% CIFavouring REST-6 ↑
ARC0.1420.0000.006+0.136[+0.065, +0.204]100%
GSM8K0.3370.0990.121+0.216[+0.115, +0.313]100%
MATH0.2820.1010.176+0.107[+0.032, +0.173]99.8%

This table is Appendix P of the revision. Selected-on-one-half, scored-on-the-other, REST-6 stays ahead of the composition-matched Hessian six on all three datasets with intervals excluding zero. The optimism the paper reports is the gap between in-sample and held-out columns: 12.2pp on GSM8K, 4.6pp on ARC, 9.3pp on MATH. It is real, but it does not consume the difference.

Two things we flag rather than let the table imply. First, which six are chosen is not stable across splits: half-split agreement on the top six is 0.341 on GSM8K, 0.557 on ARC and 0.276 on MATH, and the composition varies too (on GSM8K, 3-attention/3-MLP is the modal draw at 513 of 1000, with the paper's 2/4 at 337). The stable object is the ranking's signal, not the identity of the six. Second, these are mean single-sub-layer recovery scores for a set, the same convention as the paper's advantage_over_random, not the joint recovery of an assembled model. That would require one GPU evaluation per split, and we have not run it.

AuthorsOpenReview ↗

A combined curvature × weight-error criterion

"Activation size and weight error are tested separately. Did you test a measure that combines both and does it still rank layers in reverse?"

Yes. The measure that combines them is exactly HAWQ-V2's Ωi=Tr(Hi)⋅∥Q(Wi)−Wi∥2\Omega_i = \mathrm{Tr}(H_i)\cdot\lVert Q(W_i) - W_i \rVert^2, and this experiment was designed to answer your question and VnmZ's scope objection in one run. The full result is in our comment to VnmZ (W1); the direct answer to "does it still rank layers in reverse" is no, and we cannot claim it ranks them correctly either.

Criterion over all 64 sub-layersρ vs restoration ↑p
Activation-covariance trace, Tr(2XX⊤)/d\mathrm{Tr}(2XX^\top)/d−0.4973.0 × 10⁻⁵
Weight-space SQNR+0.032ns
Gold-continuation NLL (teacher-forced)+0.2080.099
Ω=Tr(H)⋅∥ΔW∥2\boldsymbol{\Omega = \mathrm{Tr}(H)\cdot\lVert \Delta W \rVert^2}, 256 probes+0.3520.004

Ω is positively associated with restoration, which is the direction our pre-registered falsifier named, so we report it, and we report that it cannot be read as evidence while the estimator is unvalidated. Seed-to-seed rank agreement is 0.832 against the pre-registered 0.90 with a fitted asymptote of 0.841, and the top six by Ω overlap across probe seeds at 3 of 6. The likeliest mechanism for the positive sign is size rather than curvature: Ω's perturbation term correlates 0.866 with parameter count across only two distinct values (41.9M for attention, 176.2M for MLP), and at 256 probes the size-normalised variant falls to +0.107 (ns) while the task-loss trace alone gives +0.109 (ns). So the combined measure's association with restoration is plausibly the attention/MLP size contrast re-entering through ∥ΔW∥2\lVert \Delta W \rVert^2, not curvature information.

We also tested a fourth criterion beyond the ones you named, because it is the obvious loss-level one: teacher-forced NLL of the gold continuation, measured for all 64 sub-layers under all five E6 backgrounds. It is a precise and stable instrument, with its own cross-background rank stability at 0.82–0.997, and it measures something that is not exact-match recovery. It fails to separate REST-6 from HESS-6 under four of five backgrounds (one-sided p = 0.41 under full NF4, where generation gives p = 0.0011), and 10 of 24 candidates move the two instruments in opposite directions. The clearest case: restoring L1 MLP repairs 25.5% of flipped generations while increasing the gold continuation's NLL by 0.012. On chain-of-thought tasks many derivations are correct and the reference is one of them, so a sub-layer can improve the decode while lowering the gold string's likelihood.

Reviewer_VnmZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies precision sensitivity by restoring individual sub-layers of quantized language models to FP16. It reports an inverse association between activation-covariance trace and restoration recovery, and compares restoration-guided REST-6 with trace-guided HESS-6.

Critical strengths
  • The sub-layer restoration sweep provides a useful empirical view of precision sensitivity, supported by measured rerun controls, multiple-testing correction, and an explicit assessment of selection optimism.

  • The main REST-6/HESS-6 comparison matches FP16 parameter count and attention/MLP composition, showing a clear ARC-Challenge advantage. The paper also distinguishes conditional recovery from overall accuracy and acknowledges collateral errors.

Critical weaknesses
  • The conclusion about existing quantizers rests on a cross-layer trace heuristic that neither GPTQ nor ResQ uses. GPTQ uses matrix-valued Hessian information for reconstruction, while ResQ preserves high-variance principal subspaces. Trace discards the directional information used by these methods, so HESS-6's failure does not demonstrate that their actual sensitivity criteria misallocate precision.

  • The restoration probe does not uniquely identify where quantization damage originates. All other sub-layers remain quantized, so successful restoration can compensate for errors elsewhere, while ineffective restoration can reflect interactions among damaged sub-layers. The observed repair effects therefore leave the stronger claims of functional redundancy and irreplaceable computation unestablished.

  • The four significantly inferior random configurations use fewer FP16 parameters, while the equal-parameter random configuration and EARLIEST-6-MLP are not significantly worse in overall accuracy. Although the ARC split-half analysis supports the ranking, these controls do not establish the claimed advantage of the fixed REST-6 allocation over simpler alternatives.

  • HellaSwag and MMLU losses fall below the stated detection floor, which does not establish that their underlying capabilities are unaffected by quantization. The reported comparisons therefore do not substantiate the categorical claim that algorithmic reasoning requires precision whereas commonsense and factual recall do not.

Questions for authors

See Weakness

4 comments
AuthorsOpenReview ↗

Testing the metric cross-layer allocators actually use

"The conclusion about existing quantizers rests on a cross-layer trace heuristic that neither GPTQ nor ResQ uses. GPTQ uses matrix-valued Hessian information for reconstruction, while ResQ preserves high-variance principal subspaces. Trace discards the directional information used by these methods, so HESS-6's failure does not demonstrate that their actual sensitivity criteria misallocate precision."

We agree with you, and we have changed the claim rather than defend the wording. H=2XX⊤H = 2XX^\top is the Hessian of GPTQ's layer-wise reconstruction objective; GPTQ uses its inverse Cholesky for error compensation inside a layer at a uniform bit-width, and ResQ preserves high-variance principal subspaces, which is directional information the trace throws away. Neither ranks layers against each other with Tr(H)/d\mathrm{Tr}(H)/d. HESS-6's failure is evidence about the trace, not about GPTQ's or ResQ's own criteria, and the paper should not have said otherwise.

So we ran the metric that is used across layers. HAWQ-V2 allocates by Ωi=Tr(Hi)⋅∥Q(Wi)−Wi∥2\Omega_i = \mathrm{Tr}(H_i)\cdot\lVert Q(W_i) - W_i \rVert^2 with HiH_i the task-loss Hessian, a different mathematical object from 2XX⊤2XX^\top, estimated by Hutchinson with Rademacher probes and double backward. We pre-registered that two independent probe seeds must agree on the ranking of the 64 sub-layers at ρ ≥ 0.90 before any correlation would be interpreted.

64 probes256 probesRequired
Seed-to-seed rank agreement ↑0.8040.832≥ 0.90
Convergence drift, median ↓0.0800.076≤ 0.05
Sub-layers with non-positive trace ↓230
Positive control: recomputed Tr(2XXᵀ) vs restoration−0.4969−0.4969−0.497 ✓

Quadrupling the probes bought 0.027 where a pure sampling-noise model fitted at 64 predicts 0.943 at 256. Fitting ρ(n) = S/(S + N/n + F) gives an asymptote of 0.841, so the gate is unreachable by adding compute and we stopped rather than hunt for a configuration that passes. The consequence for an allocator is direct: the top six by Ω overlap across the two probe seeds at only 3 of 6 at 256 probes (4 of 6 at 64). Which six sub-layers a HAWQ-V2 allocator would protect is close to a coin flip on the probe seed, which is why we did not run the matched HAWQ-6 vs REST-6 head-to-head: it would have produced an uninterpretable number.

One structural observation does not depend on the estimator's precision. Three sub-layers carry negative average trace, and L1 attention at −1.04 × 10⁻⁴ exceeds the largest positive trace (7.8 × 10⁻⁵) in magnitude. HAWQ-V2's derivation assumes H is evaluated at a converged minimum W∗W^{*}, where it is PSD; an instruction-tuned model scored on a held-out corpus is not at such a point. That is an objection to applying the method here, not a sampling artifact.

This is not our inference about someone else's method. Q-BERT makes the condition explicit ("our method expects that before performing quantization the trained model has converged to a local minima… The necessary optimality conditions are zero gradient, and positive curvature"), checks it, and finds it violated on one of its four tasks, where it observes "very large negative eigenvalues" and reports "the highest performance loss" of SST-2, MNLI, CoNLL-03 and SQuAD on exactly that task, which it attributes to the non-convergence. Our negative traces are the same phenomenon at a larger scale: a reason the premise does not transfer to this setting, rather than a reason to doubt either result.

We also report what we will not claim from it. Ω's correlation with restoration is positive, at +0.404 and +0.408 across two seeds at 64 probes and +0.352 (p = 0.004) at 256, which is the direction our pre-registered falsifier named. We report it and we report that it cannot be read either way while the instrument is unvalidated. The likeliest mechanism is size, not curvature: Ω's perturbation term correlates 0.866 with parameter count across only two distinct values, and at 256 probes the size-normalised variant falls to +0.107 (ns) while the task-loss trace alone gives +0.109 (ns).

All of this is Appendix M of the revision, which claims only this: among cheap numerical criteria, the activation-covariance trace is inverted against functional importance, SQNR is null, gold-continuation likelihood is not significant, and the task-loss Hessian could not be estimated reliably at 8B. We make no claim about GPTQ's or ResQ's own allocation behaviour.

[1] Dong, Yao, Arfeen, Gholami, Mahoney and Keutzer. HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). The formula we implement is its Eq. 10. (The arXiv preprint, 1911.03852, lists a seventh author, Yaohui Cai, who is not on the published version; we cite the published one.)

AuthorsOpenReview ↗

Is recovery a property of the sub-layer, or of the background it is measured against?

"The restoration probe does not uniquely identify where quantization damage originates. All other sub-layers remain quantized, so successful restoration can compensate for errors elsewhere, while ineffective restoration can reflect interactions among damaged sub-layers. The observed repair effects therefore leave the stronger claims of functional redundancy and irreplaceable computation unestablished."

This is the objection we took most seriously, because it is aimed at the instrument rather than the framing: if it holds, every per-sub-layer number in the paper measures an interaction artifact. We ran a new experiment to test it, and we pre-registered the falsifier before any compute: if the mean rank correlation across alternative backgrounds falls below 0.40 we withdraw every per-sub-layer claim in the paper. It did not fire.

We re-measured marginal recovery Δ(i∣B)=acc(B∪i)−acc(B)\Delta(i \mid B) = \mathrm{acc}(B \cup \\{i\\}) - \mathrm{acc}(B) for 24 candidate sub-layers under five backgrounds, holding the flipped example sets fixed.

BackgroundAlready restored before probingSpearman ρ vs full-NF4 ↑p ↓
B1L2/L5/L19 attn, L17 MLP (random, seed 101)+0.6417.4 × 10⁻⁴
B2L20/L25 attn, L14/L23 MLP (random, seed 102)+0.8393.0 × 10⁻⁷
B3L25/L26/L28/L30 attn (highest trace, GPTQ-style)+0.9231.4 × 10⁻¹⁰
B4L5/L8/L12/L13 MLP (highest-recovery, adversarial)+0.6466.5 × 10⁻⁴
Mean+0.762

B4 is the background your mechanism predicts should break the ranking: it has already repaired the sub-layers most able to compensate, so any sub-layer whose apparent value came from covering their damage should collapse. It reproduces the ranking at 0.646.

The selection control, which we pre-committed to reporting whatever it showed: if stability appeared only among the sub-layers we had chosen on our own data, that would be selection, not stability. RANDOM-12 was drawn uniformly from the 52 remaining sub-layers and carries no such bias.

StratumnMean ρ across B1–B4 ↑
RANDOM-12 (no selection bias)12+0.659
HESS-66+0.743
REST-66−0.071

REST-6's near-zero value is a resolution limit rather than instability. Under full NF4 its six values span 0.207–0.255, a spread of 2.05× the measured do-nothing noise band against 8.06× for RANDOM-12, so there is no ordering inside REST-6 for another background to reproduce.

StratumΔ range under full NF4Spread× noise band ↑
All 24 candidates0.017 – 0.2550.23910.15
RANDOM-120.030 – 0.2190.1898.06
HESS-60.017 – 0.1070.0903.83
REST-60.207 – 0.2550.0482.05

What this licenses and what it does not. The probe identifies which group a sub-layer belongs to, reproducibly and under an adversarial background; it does not resolve the ordering within the top six. The supported statement is "these six", never "L7 > L1 > L31", and the revision says so. Two further limitations we report rather than omit: top-6 overlap with the full-NF4 ranking is 4, 5, 4 and 3 of 6, worst under the adversarial background, with L31 MLP and L6 MLP in all five; and per dataset the mean correlation is 0.680 on GSM8K and 0.733 on ARC but only 0.405 on MATH.

The experiment and its three limitations are Appendix N of the revision.

Gates, all passed before the sweep was paid for: G0b (FP16 correct on 0.97–1.00 of each flipped set, the check that catches a dataset-source swap), G0 (full NF4 reproduces the published sweep to within 1 flipped example against a tolerance of 2), G1 (no background out of headroom, so nothing was excluded).

On the stronger claims. Here too we agree with you: this establishes reproducibility, not localisation. Restoring sub-layer i against a quantized background measures i's marginal contribution in that context; it does not prove the damage originated there, and we have removed "irreplaceable computation" from the paper. The independence assumption the objection targets is one the mixed-precision literature makes and does not check. HAWQ-V2 sums a per-layer sensitivity across layers (Ω=∑iTr(Hi)⋅∥Q(Wi)−Wi∥2\Omega = \sum_i \mathrm{Tr}(H_i)\cdot\lVert Q(W_i) - W_i \rVert^2, its Eq. 10) while Q-BERT sorts layers by its own per-layer Ωi=∣mean(λi)∣+std(λi)\Omega_i = |\mathrm{mean}(\lambda_i)| + \mathrm{std}(\lambda_i) (its Eq. 2), and in both cases each layer's sensitivity is measured once at the unquantized model. A negative result here would have been a finding about that literature as much as about this paper. We now state the assumption explicitly and point at this measurement of it.

AuthorsOpenReview ↗

The budget confound in the random controls, and a matched test with power

"The four significantly inferior random configurations use fewer FP16 parameters, while the equal-parameter random configuration and EARLIEST-6-MLP are not significantly worse in overall accuracy. Although the ARC split-half analysis supports the ranking, these controls do not establish the claimed advantage of the fixed REST-6 allocation over simpler alternatives."

We agree with your reading of the table, and the criticism lands. Four of the five random draws hold 520–654M FP16 parameters against REST-6's 788M, so their deficit on ARC is confounded with budget. The one draw that is exactly budget-matched is not significantly worse, and EARLIEST-6-MLP holds 1,057M, or 1.34× REST-6, and is also not significantly worse.

Condition on ARCFP16 paramsRecovery ↑McNemar vs REST-6 ↓
REST-6788.5M0.526—
RAND6 seed 105 (budget-matched)788.5M0.3950.664
EARLIEST-6-MLP (1.34× budget)1057.0M0.3680.473
RAND6 seed 101654.3M0.1050.0068
RAND6 seed 102654.3M0.2890.0259
RAND6 seed 103520.1M0.2110.0061
RAND6 seed 104654.3M0.1580.0033

One budget-matched draw is n = 1 and cannot settle the question in either direction, and we note that the draw that ties with REST-6 shares a member with it (L6 MLP, one of the two sub-layers that appears in all five E6 backgrounds). So we replaced the single draw with the whole matched distribution. Because an attention sub-layer holds 41.9M FP16 parameters and an MLP sub-layer 176.2M, a six-set's FP16 budget is fixed by its attention/MLP composition, which makes parameter-matching and composition-matching the same constraint: there are 17,836,160 sets with REST-6's 2-attention/4-MLP composition, and we sampled 100,000 of them.

DatasetPercentile among 100k draws ↑Held-out adv. ↑95% CISplits favouring ↑
ARC99.5+9.0pp[+1.9, +15.7]99.2%
GSM8K100.0+11.0pp[−0.03, +20.8]96.4%
MATH99.55+7.1pp[−1.3, +15.4]95.0%

This is Appendix P of the revision. The held-out columns select the six on one half of the flipped examples and score them on the other, over 1000 splits, so they are not contaminated by selection. Against EARLIEST-6-MLP at its larger budget, the held-out advantage is +7.0pp on ARC with 99.8% of splits favouring selection, and +5.4pp and +5.3pp on GSM8K and MATH with CIs that include zero.

What we conclude, and what we no longer conclude. ARC supports the ranking against a matched random baseline; GSM8K and MATH do not, at this sample size. We have correspondingly removed the claim that REST-6 is advantageous as an allocation and kept the claim that the ranking carries signal a matched random draw does not. One scope note we add rather than leave implicit: these held-out scores are mean single-sub-layer recovery for a set, which is the convention the paper's advantage_over_random already uses, not the joint recovery of an assembled six-sub-layer model, which would need one GPU evaluation per set.

AuthorsOpenReview ↗

The null result on HellaSwag and MMLU is not evidence of absence, and we withdraw the claim

"HellaSwag and MMLU losses fall below the stated detection floor, which does not establish that their underlying capabilities are unaffected by quantization. The reported comparisons therefore do not substantiate the categorical claim that algorithmic reasoning requires precision whereas commonsense and factual recall do not."

We agree with you, and this one we concede without qualification. Quantization costs HellaSwag 1.0pp (65.0 → 64.0) and MMLU 2.21pp (55.62 → 53.41) at n = 500 and n = 498. Those are the point estimates; the intervals are the problem.

BenchmarkFP16 ↑NF4 ↑Damage ↓95% CI on the damage
ARC-Challenge79.274.84.4pp[−0.8, +9.6]
MMLU (6 subjects)55.6253.412.21pp[−4.0, +8.4]
HellaSwag65.064.01.0pp[−4.9, +6.9]

Every one of those intervals contains both zero and ARC's measured 4.4pp. The design cannot distinguish "unaffected" from "damaged about as much as ARC". It is worse than that: we did not record per-example correctness arrays for HellaSwag or MMLU, so we cannot run the paired McNemar test that gives the other three benchmarks their power, and we cannot count flipped examples at all, which is the quantity the probe is defined on. A sub-layer sweep over a benchmark with no measurable flipped set has nothing to measure.

The categorical claim is therefore withdrawn. The revision states the scope as: the inversion and the task-selective pattern are established on GSM8K, ARC-Challenge and MATH, where quantization produces a flipped set large enough to measure against a replicated null; HellaSwag and MMLU were run and showed no detectable loss, and we report that as an under-powered observation rather than as a contrast. Recording per-example arrays for both and re-running the sweep is the experiment that would settle it, and we say so rather than imply the question is closed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.