- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Stout, a two-stage framework for efficient layerwise QAT. It introduces Oats, a curvature-aware score that estimates each layer's ability to recover from quantization errors and guides the training order. It also proposes ScaleKernel to reduce memory overhead during scale training. Experiments demonstrate improved quantization performance and memory efficiency.
Critical strengths
The paper is well-written and well-structured. The proposed method is supported by theoretical analysis and extensive experiments, including comparisons with different layer ordering strategies and quantization baselines.
Critical weaknesses
-
The theoretical analysis does not fully justify the proposed ordering strategy. The derived recoverable loss measures the potential benefit of adapting an individual layer, but it does not necessarily guarantee that sorting layers by this score leads to a better overall QAT outcome, given the interactions between layers.
-
The necessity of the proposed scoring metric is not fully established. Although the authors compare different curvature approximations and sensitivity-based ordering, it remains unclear whether simpler gradient-based metrics could achieve similar performance without estimating inverse Fisher information.
-
The two proposed components are relatively independent. Oats improves the layerwise training order, while ScaleKernel optimizes the backward implementation of scale training. Their combination does not introduce a unified algorithmic contribution beyond integrating these two techniques.
Questions for authors
Please refer to the weaknesses.
Minor: The provided code seems not accessible.
Reply to the reviewer SF8r
We thank Reviewer SF8r for three precise conceptual objections, and we address each below.
W1: "The derived recoverable loss measures the potential benefit of adapting an individual layer, but it does not necessarily guarantee that sorting layers by this score leads to a better overall QAT outcome, given the interactions between layers."
A: We agree, and Apx. B.6 says so. It gives the exchange model that would be needed to justify sorting and shows that neither condition it requires follows from Theorem 1: a layer also leaves errors that later layers must absorb, and the score would additionally have to rank the residual fractions 1−ρi. Ascending Oats is a schedule the experiments support; we do not derive it from the theorem.
The measurement does account for interactions in expectation. Proposition 1 shows that under complementary partitions E[Cl] retains both the individual responses of the other layers and the norm of their combined response, so reinforcement and cancellation survive without forming an interaction matrix. Tab. 13 quantifies the residual: nonadditivity 0.104 at K=1, with the sampled response at cosine 0.897 to the direct one.
The ordering ablation bears on the global question. Perturbing the Oats order by increasing rank distance gives, on Qwen at 2 bit:
The outcome is unchanged down to τb≈0.87: LCB drifts by 0.70 pp and BFCL is flat, in fact 0.10 pp higher for the most divergent draw than for the default, which is well inside the seed spread. An arbitrary order loses most of the gain. We have no measurements between τb≈0.87 and ≈0, so this locates no threshold; what it shows is that the result survives the perturbation an imperfect score actually produces and does not survive discarding the score.
The two directions make the same point. Ascending and descending Oats use identical per-layer scores and differ only in arrangement, and that arrangement is worth 4.91 pp BFCL (p<0.001, three seeds), with held-out CE moving the same way (1.687 against 1.736). Ascending Oats also beats the two competing score-based orders, sensitivity by 3.83 pp and reference potential by 2.39 pp.
W2: "It remains unclear whether simpler gradient-based metrics could achieve similar performance without estimating inverse Fisher information."
A: Apx. C.3 identifies the development-selected ordering baseline as the raw external response, which is the same Δg~ without curvature weighting. Removing the inverse Fisher costs 0.87 pp LCB and 2.68 pp BFCL end to end, and 0.15 Kendall τ in predicting measured recovery.
The curvature term is also cheap. It is one of the four scoring traversals in Eq. (46), about 0.2 h of an 11.07 h pipeline, estimated by ordinary backward passes on labels sampled from the model (Lemma 1) and stored as one value per quantization group, so applying its inverse is a division rather than a matrix inversion. Tab. 14 shows the group-mean form differs from the full diagonal by 3.2%.
Both halves are needed for the reason Theorem 1 gives: the gradient response measures how much adaptation the other layers' errors demand of a layer, and the curvature determines how much of that demand the layer can meet. A layer with a large response but stiff curvature cannot use it. The two one-sided scores show this directly. Own-layer sensitivity uses curvature without the cross-layer response and is uninformative at −0.07; the raw response uses the response without curvature and reaches 0.71; combining them reaches 0.86.
W3: "The two proposed components are relatively independent. Oats improves the layerwise training order, while ScaleKernel optimizes the backward implementation of scale training. Their combination does not introduce a unified algorithmic contribution."
A: The independence is a verified design property, and it is what makes the attribution credible. There is no interaction, which is what allows the accuracy to be attributed to Oats and the memory to the kernel. It also means either component can be adopted alone.
Minor: "The provided code seems not accessible."
A: We apologise for this, We correct the link in the abstract to
https://anonymous.4open.science/r/stout-613D/readme.md.