acceptodds
Under review as a conference paper at ICLR 2027

Global FP4 Quantization via Low-Dimensional Convex Optimization

Abstract

Native FP4 quantization encodes small weight groups with shared scales, but correlated inputs couple their reconstruction errors. Independent groupwise selection neglects these interactions, while exhaustive joint search grows exponentially with the number of groups. For reconstruction metrics with full within-group curvature and a shared rank- component, we formulate a convex relaxation with an -dimensional dual. We prove that any feasible candidate mixture can be reduced to at most mixed groups without changing its aggregate residual or increasing the relaxed objective. LDC-FP4 uses this construction to coordinate codes and scales and recover native encodings with a residual-variance bound. Across Llama and Qwen models, LDC-FP4 improves language modeling and downstream performance over the evaluated quantization baselines. Matched-candidate comparisons show that global coordination improves reconstruction beyond independent selection and outperforms a wide-beam search baseline at significant lower calibration cost. All optimization is confined to calibration, leaving the native FP4 representation and inference operators unchanged.

Primary area:
optimization
Keywords:
post-training quantization, FP4 quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_REV5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the "encoding quality" problem in post-training quantization of native FP4 (NVFP4: 16 E2M1 codes sharing one E4M3 scale, plus one tensor-level scale). Independent per-group selection drops cross terms, while exhaustive joint search is a KBK^B combinatorial explosion. The authors' solution is to write the curvature HH as "block-diagonal DD (exact within-group) + low-rank UU⊤UU^\top (shared across groups)"; then each candidate needs only a scalar cost α\alpha and an rr-dimensional residual vector vv, and the whole-row objective collapses to L0+∑bαb+∥∑bvb∥2L_0+\sum_b\alpha_b+\|\sum_b v_b\|^2. On top of this, the paper gives a convex relaxation and its rr-dimensional dual, and proves a support-reduction theorem: any feasible mixture can be reduced to "at most rr mixed groups" without increasing the objective or changing the aggregate residual, paired with a residual-variance rounding bound.

Critical strengths
  1. The core reframing is good. The step "global coupling lives only in an rr-dimensional subspace" turns a KBK^B combinatorial search into "one rr-dimensional price variable λ\lambda coordinating all groups + a support reduction that compresses 512 groups down to 16"; the transformation itself carries real information and lands directly on the algorithm (2B→2r2^B\to2^r). The authors validated the central claim cleanly: the matched-candidate experiment holds the candidate list fixed and varies only the solver (independent / coordinate / pairwise / beam / LDC), then compares against beam again under equal GPU time—this is the right way to separate "the credit of coordination" from "the credit of candidates."

  2. Zero deployment cost + a structural safety net. All optimization is confined to calibration; D,UD,U, the mixture pp, λ\lambda, and the candidate tables are all discarded after use, so the deployed format and inference operators are identical to native GPTAQ (throughput ratio 1.00/1.00). Moreover, at the end of every row the method takes a "smaller-of-the-two" between the new encoding and the GPTAQ initialization under the original objective, which makes "no worse than GPTAQ" a per-row structural guarantee rather than an empirical observation. For a method positioned as "replacing the rounding step," this plug-in-plus-floor design is genuinely practical.

Critical weaknesses
  1. The generality of "native FP4" is validated on only one format—MXFP4 is missing. The title and wording pitch the method at the level of "native FP4," yet every experiment is NVFP4 (E2M1 codes + an E4M3 shared scale, group=16). The other mainline in industry is OCP MXFP4, which uses an E8M0 pure-exponent shared scale with group=32. This is not a matter of changing a hyperparameter—it moves two of the method's load-bearing points at once: (a) the "scale" dimension of candidates goes from a grid with a mantissa to powers of two, so the scale-neighborhood candidate construction in §4.2 must be redone; (b) group going from 16 to 32 means DbD_b goes from 16×16 to 32×32, changing both the cost of "keeping all of the within-group block" and the fraction of "across-group interaction absorbed by the low-rank term." The authors repeatedly say "the optimization domain is the product of the supplied lists" and "format-agnostic," yet nowhere give a single number under MXFP4, nor even a discussion of how Theorem 1's dual dimension / support bound changes when the group size changes. Thus, for a method argued under the banner of "native FP4," its cross-format applicability is currently an assertion, not a demonstration.

  2. Model-architecture coverage is narrow, and the method's key assumption is precisely architecture-dependent. All six models come from just the Llama-3.1 and Qwen3 families, and all are dense and homogeneous (GQA + RoPE + a standard SiLU MLP, BF16 norm/head). The heart of the method is that "the curvature's across-group coupling can be captured by the rank-rr low-rank component of D+UU⊤D+UU^\top," but the structure of the input second moment Z⊤ZZ^\top Z depends strongly on architecture: expert-level sparse activation in MoE turns the aggregated HH into a mixture distribution and the shared directions may not be low-rankable at all; the attention-low-rank coupling of MLA (DeepSeek-style) rewrites ZZ; strong outlier-channel architectures (e.g., Gemma) concentrate UU's energy heavily, making the rank-rr "shared directions" assumption harder to hold. The default r=16r=16 is chosen empirically on these two families via the η\eta percentiles, and the appendix reports that the rank meeting the 90th-percentile η≤10%\eta\le10\% has median 16 but a 90th percentile already at 64—showing that even "low rank suffices" drifts with model/layer. Without an architecturally distinct model (at least one MoE, one non-GQA / MLA / different-norm model), I cannot tell whether this is "a general encoder" or "an encoder tuned well on Llama/Qwen." This directly weakens any extrapolation beyond the abstract's "across Llama and Qwen models."

  3. Even within the tested scope, the net gain over the strongest baseline is small while the cost (calibration) is high, and together these cap the practical significance. In perplexity, LDC's margin over the strongest competitor is only on the order of ~0.1 (e.g., Llama-3.1-8B 7.53 vs FAAR 7.64), with a maximum six-task-average advantage of +0.7pp. The method's real selling point in fact lies on the cost-quality frontier (under matched candidates it is 0.8% better than Beam-256 yet uses only 41.5% of its calibration time, i.e., a 2.41× speedup), not in absolute quality. The problem is that this very selling point is the one most in need of cross-format, cross-architectural support, yet it is validated only on a single NVFP4 + single hardware (B200) + two corpora; 70B calibration takes 43.9 GPU-hours, and under MXFP4 / a larger group / a different architecture—where both the 2r2^r enumeration and the DbD_b size change—whether this "cheaper coordination" still holds is entirely unknown. For a paper argued around "lower-cost global coordination," the evidentiary surface is too narrow.

Questions for authors
  1. Can you provide results for MXFP4 (E8M0 shared exponent, group=32), or at least a feasibility/cost analysis of moving the group from 16 to 32 and replacing the scale set with powers of two? Specifically: how would the scale-neighborhood candidates of §4.2 be reconstructed accordingly, and how does the calibration cost change once DbD_b becomes 32×32?

  2. Can you validate on at least one architecturally distinct model—a MoE (expert sparse activation) and an MLA / non-GQA model? My concern is: on such architectures, can the rank-rr shared curvature still capture across-group coupling at low rank (report the η\eta percentiles), does the default r=16r=16 need per-architecture retuning, and does LDC's gain over GPTAQ/FAAR persist? If the low-rank assumption fails on certain architectures, please state the architectural boundary within which the method applies.

  3. Regarding practical significance: please reproduce the core cost claim—"2.41× cheaper than Beam-256"—on non-NVFP4 formats and at least one non-B200 hardware, and report the variance of the perplexity/accuracy gains across calibration seeds and across models, so that readers can judge whether +0.1 ppl / +0.7pp is stably larger than run-to-run fluctuation, and whether it justifies ~44 GPU-hours of calibration on a 70B model.

10 comments
AuthorsOpenReview ↗

Response to Reviewer REV5 (1/4): MXFP4 transfer and the rank guarantee

Thank you for the care you took with both the derivation and the experiments. We especially appreciated your discussion of how one shared vector can guide the encoding choices of an entire row. We are glad you found the comparison with beam search at equal GPU time informative. Your comments on the final check against the original row objective were also encouraging. We designed that check to make the encoder practical to use, and we appreciate your attention to how it protects reconstruction quality without adding work at inference.

Your suggestion to test MXFP4 gave us a way to examine how the solver behaves when both the group size and the scale grid change. We kept the rank fixed and measured encoding time on B200 and H100 SXM.

On Qwen3-8B quantized to OCP MXFP4 (group size 32 and E8M0 power-of-two shared scales) at the unchanged default rank r=16r=16, LDC-FP4 reduces WikiText-2 token perplexity from 9.918 to 9.786. Complete end-to-end calibration requires 0.61 GPU-hours, compared with 0.29 GPU-hours for the GPTAQ initialization. In a companion matched-candidate experiment across 12 linear modules, MXFP4 coordination lowers held-out NMSE by 10.8% relative to independent selection and achieves a 2.43×2.43\times encoding speedup over Beam-256 on B200.

We respectfully invite reconsideration of the concern that increasing the group size from 16 to 32 or switching to E8M0 scales alters the dual dimension or Theorem 1's support bound. Both quantities are governed solely by the shared rank rr. At r=16r=16, the dual variable λ∈R16\lambda\in\mathbb R^{16} always has 16 coordinates, and support reduction trims any feasible fractional mixture to at most 16 mixed groups while preserving the mixture's aggregate shared residual. As in constructive Shapley-Folkman trimming for separable problems under affine constraints [4], the proof counts the rr linear aggregate-residual equations across group simplices and is independent of E4M3 mantissa bits or the number of weights per group.

To adapt candidate construction in Section 4.2 to MXFP4, each group's candidate list enumerates neighboring hardware-valid E8M0 power-of-two scales around the initial scale, pairing each scale with a complete 32-element E2M1 code vector evaluated under the 32×3232\times32 diagonal block DbD_b. For a row of width dd, rank rr, and candidate count KK, doubling group width gg from 16 to 32 halves the group count B=d/gB=d/g. Precomputing local quadratic costs scales as O(BKg2)=O(dKg)O(BKg^2)=O(dKg) and doubles, computing shared residual vectors O(BKgr)=O(dKr)O(BKgr)=O(dKr) is unchanged, and each dual evaluation O(BKr)=O(dKr/g)O(BKr)=O(dKr/g) coordinates half as many groups. Storage for dense local blocks grows as O(dg)O(dg), while shared-factor storage remains O(dr)O(dr). Our measured MXFP4 timings confirm that these scaling expressions remain highly practical end to end.

Recent work underscores the value of accuracy optimization tailored to hardware-native formats, including OCGQuant [1], ARCQuant [2], and WUSH [3]. Together with our NVFP4 benchmarks, these MXFP4 results establish LDC-FP4 across both mainline native FP4 standards under a rank-controlled recovery guarantee.

References

[1] Yao et al. OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization. EMNLP 2026.

[2] Meng et al. ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs. ACL 2026.

[3] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[4] Dubois-Taine and d'Aspremont. Frank-Wolfe meets Shapley-Folkman: a systematic approach for solving nonconvex separable problems with linear constraints. Mathematical Programming, 219, 501–551, 2026.

AuthorsOpenReview ↗

Response to Reviewer REV5 (2/4): MLA and mixture-of-experts architectures

To evaluate transfer beyond dense GQA Transformers, we tested DeepSeek-V2-Lite, which combines multi-head latent attention (MLA) with sparse mixture-of-experts (MoE) layers comprising both shared and routed experts. Quantizing to NVFP4 at the unchanged default rank r=16r=16, LDC-FP4 lowers WikiText-2 token perplexity from 7.463 to 7.309. Complete calibration takes 1.58 GPU-hours, compared with 0.86 GPU-hours for the GPTAQ initialization, providing direct end-to-end validation on both architectural features raised in your review.

Using prespecified seed 42 at the unchanged rank r=16r=16, we evaluated directional approximation errors across module categories in Qwen3-8B (MXFP4) and DeepSeek-V2-Lite (NVFP4). Each entry below reports the median directional error and the percentage of evaluated modules whose own 90th-percentile directional error is at most 10%.

Model / formatModule categoryEvaluated / target modulesMedian error %Modules meeting criterion %
Qwen3-8B / MXFP4Attention144 / 1442.897.2
Qwen3-8B / MXFP4Dense MLP108 / 1083.496.3
DeepSeek-V2-Lite / NVFP4MLA projections108 / 1083.196.3
DeepSeek-V2-Lite / NVFP4First-layer dense MLP3 / 34.9100.0
DeepSeek-V2-Lite / NVFP4Shared experts78 / 784.194.9
DeepSeek-V2-Lite / NVFP4Routed experts312 / 4,9924.794.9

Across MLA projections, the first-layer dense MLP, shared experts, and a 312-module routed-expert sample, median directional errors remain low (3.1% to 4.9%) and 94.9% to 100.0% of modules satisfy the 10% tail criterion without retuning the default rank r=16r=16.

We also evaluated a targeted rank increase to r=32r=32 on the small subset of modules that exceeded the 10% threshold at r=16r=16. Raising rr to 32 brings all 4 attention modules and all 4 dense-MLP modules in Qwen3-8B, all 4 MLA modules and all 4 shared-expert modules in DeepSeek-V2-Lite, and 15 of the 16 routed-expert follow-up modules within the 10% criterion. Default r=16r=16 transfers directly across these architectures. Module-level directional error also guides additional rank capacity where it is useful.

LDC-FP4 fits reconstruction curvature from the inputs of each linear module. Its objective and support-reduction proof apply to the resulting group costs and shared residuals across attention and feed-forward architectures. Recent literature aligns with this structure, as MoEQuant [1] shows the effectiveness of expert-conditioned calibration statistics for MoE quantization and MHA2MLA [2] exploits low-rank structure within MLA projections. Together, these module-level diagnostics and end-to-end DeepSeek-V2-Lite gains confirm that rank-rr curvature coordination generalizes reliably to MLA and MoE models.

References

[1] Chen et al. MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance. ICML 2025.

[2] Ji et al. Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs. ACL 2025.

AuthorsOpenReview ↗

Response to Reviewer REV5 (3/4): Encoding efficiency across GPUs and formats

To verify that our cost-quality advantage extends beyond NVFP4 and B200, we ran a matched-candidate comparison on a prespecified set of 12 linear modules spanning blocks 0, 11, 23, and 35 (q_proj, o_proj, and down_proj). The evaluation covers NVFP4 and MXFP4 on NVIDIA B200 together with MXFP4 on NVIDIA H100 SXM.

Format / GPUSolverOriginal loss / initializationHeld-out NMSE / IndependentMedian encoding time sSpeedup over Beam-256
NVFP4 / B200Independent0.9521.00078.62.73×
NVFP4 / B200Beam-2560.8860.923214.31.00×
NVFP4 / B200LDC-FP40.8930.91991.72.34×
MXFP4 / B200Independent0.9611.00071.43.20×
MXFP4 / B200Beam-2560.8540.881228.81.00×
MXFP4 / B200LDC-FP40.8660.89294.12.43×
MXFP4 / H100 SXMIndependent0.9611.000119.83.28×
MXFP4 / H100 SXMBeam-2560.8540.881392.61.00×
MXFP4 / H100 SXMLDC-FP40.8660.892174.52.25×

Across every format and GPU configuration, LDC-FP4 retains a substantial held-out NMSE reduction over independent group selection while cutting encoding time by 2.25×2.25\times to 2.43×2.43\times relative to Beam-256. On MXFP4, LDC-FP4 achieves an identical 0.892 held-out NMSE ratio on both B200 and H100 SXM alongside 2.43×2.43\times and 2.25×2.25\times speedups, confirming that 32×3232\times32 local blocks and power-of-two scale candidates preserve fast coordination across GPU architectures.

These 12-module cross-format and cross-GPU benchmarks complement the full-study NVFP4 solver evaluation in Table 9, where LDC-FP4 outperforms Beam-256 on both held-out reconstruction error and teacher-student token KL at 2.41×2.41\times lower calibration time.

Offline optimization for efficient deployed models is a major theme in recent quantization research, including SpinQuant [1], GPTAQ [2], and WUSH [3]. Q-Strata [4] similarly demonstrates the benefit of coordinating coupled quantization candidates across MoE blocks, including on DeepSeek-V2-Lite. LDC-FP4 brings fast global coordination to within-row scale-and-code selection across formats and GPUs. The exported checkpoints use standard native representations and unchanged inference operators.

References

[1] Liu et al. SpinQuant: LLM Quantization with Learned Rotations. ICLR 2025.

[2] Li et al. GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration. ICML 2025.

[3] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[4] Lee et al. Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs. EMNLP 2026.

Reviewer_iPEVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LDC-FP4, a post-training quantization method that coordinates group encodings while retaining the native NVFP4 representation and inference operators. It approximates the reconstruction metric using full within-group blocks and a shared low-rank component, constructs finite candidates for each group's codes and scale, and solves a convex relaxation through a low-dimensional dual. A rank-controlled support reduction preserves the aggregate residual while leaving only a limited number of mixed groups, which are resolved into native encodings with a residual-variance bound and subsequent refinement. Experiments on six Llama and Qwen models evaluate quantization quality, matched-candidate optimization, recovery, and computational cost, primarily in W4A4 with additional W4A16 evaluation.

Critical strengths
  1. A coherent optimization contribution under native-format constraints. The low-dimensional coordination, support reduction, and discrete recovery procedure are closely connected to the structure of grouped FP4 encoding. The method improves the selection of legal codes and scales without retaining auxiliary optimization variables or adding inference operators. This gives the optimization contribution a concrete deployment motivation.

  2. Controlled experiments support the contribution of the coordination procedure. The evaluation includes comparisons with identical candidate lists, reconstruction objectives, and final refinement, together with component ablations and uncertainty estimates. The reported six-task average improves from FAAR's 74.9 to 75.6, with a paired uncertainty interval of [+0.31, +1.09] percentage points. The matched-candidate results also show lower reconstruction error than independent selection and Beam-256, at substantially lower calibration cost than Beam-256. These controls help attribute the gains to the proposed procedure.

Critical weaknesses
  1. The scope of the framework and its relationship to other quantization formats need clearer evaluation. BPDQ [1] constructs group-specific grids through bit-plane coefficients and refines them using Hessian-aware error compensation. It provides a relevant alternative for improving quantization quality by increasing representation flexibility. LDC-FP4 has a distinct motivation in retaining native NVFP4, but the manuscript does not compare the quality, storage, and computational trade-offs between these approaches. A methodological discussion and a controlled comparison in a shared W4A16 setting would strengthen the positioning of the contribution. Such a comparison should preserve each method's intended representation and account for actual storage, including metadata, as well as calibration and inference costs.

The distinction between the general optimization framework and the NVFP4-specific implementation also needs clarification. With fixed finite candidate lists, Eqs. (5)-(12), the rank-controlled support reduction, and the residual-variance recovery argument depend on local candidate costs and shared residual vectors, rather than on a nonuniform FP4 codebook or exponent bits. They therefore appear applicable to legal INT4 candidates, and potentially to 2-/3-bit candidates, provided that candidate combinations respect the target format's shared-parameter constraints. In contrast, the group size, E4M3 scale neighborhood, E2M1 code generation, and packing are specific to the implemented NVFP4 format. The current experiments establish effectiveness for NVFP4 but do not determine which gains depend on those choices. Clarifying this boundary would better establish the contribution's scope; mathematical portability alone does not establish comparable empirical gains or calibration costs in another format.

  1. The practical gains are modest relative to the additional offline work. The positive downstream improvements are supported by the reported uncertainty analysis, but their magnitude remains limited. The paper reports calibration costs of 2.91 GPU-hours for Qwen3-8B and 43.86 GPU-hours for Llama-3.1-70B. Under identical candidate lists, LDC-FP4 takes 8.87 times the time of independent selection, although it is faster and more accurate than Beam-256. Its inference throughput matches the native initialization under the same format and backend. The existing cost measurements are useful; a clearer account of when the quality improvements justify the extra calibration work would strengthen the practical significance, particularly for settings with limited reuse of the quantized model.

Overall assessment. I recommend weak acceptance. The native-format construction and controlled empirical evidence constitute a useful contribution. The moderate gains, incomplete comparison with variable-grid quantization, and unclear separation between the general framework and its NVFP4-specific realization limit the strength of the recommendation. The theoretical guarantees should be interpreted within their stated finite-candidate and structured-metric setting. My topic-specific confidence is limited, as reflected in the confidence score.

Reference

[1] Junyu Chen et al. BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models. ICML 2026, PMLR 306:15272–15289. Proceedings; official implementation.

Questions for authors
  1. How does the proposed shared low-dimensional coordination differ from BPDQ's Hessian-aware error propagation? Please clarify the methodological contribution and the expected quality and deployment trade-offs between coordination within the native FP4 candidate space and optimization of a more flexible quantization grid.

  2. Could the authors add a controlled W4A16 comparison with BPDQ on a common Qwen3 checkpoint, using matched activation precision, calibration data, and evaluation, at comparable actual storage budgets? Please report metadata overhead and calibration cost, and deployment latency and memory where both implementations support common hardware. A unified perplexity protocol is necessary: BPDQ's official implementation reports document-level word perplexity, whereas this paper uses token-based windowed perplexity. This comparison would help assess whether the observed native-format quality gains justify the offline cost relative to a variable-grid alternative.

  3. Clarification of format dependence. Which components genuinely require FP4? Would the relaxation, low-dimensional dual, and support-reduction result remain valid for fixed finite lists of complete legal INT4 encodings? Please distinguish these components from format-dependent candidate generation, scales/zero points, grouping, and packing. In particular, continuously optimized scales or parameters shared across groups would require additional handling rather than an unchanged product-of-finite-lists formulation. This would clarify which benefits arise from general cross-group coordination and which depend on NVFP4's small groups and discrete scales. Any claim of unchanged native inference should be relative to an available backend for the target format.

  4. Lower-bit applicability and recovery limits. Could the framework also be instantiated for 2-/3-bit weights, and what would limit its quality-cost trade-off? Coarser candidate sets may increase the potential value of coordination, but they may also increase the surviving residual variances in Proposition 2; at most r mixed groups does not by itself imply a small rounding penalty. Lower weight precision alone does not change H when the inputs and reconstruction objective are fixed, although candidate-error geometry, effective residual dimension, and approximation quality along those directions can change. A focused discussion is needed to delimit the claims. A small matched-candidate INT4 or lower-bit experiment would be useful supporting evidence, rather than requiring a comprehensive new benchmark suite.

3 comments
AuthorsOpenReview ↗

Response to Reviewer iPEV (1/3): BPDQ and the controlled W4A16 comparison

Thank you for your encouraging assessment of LDC-FP4. We especially appreciate the attention you gave to the connection between the optimization and the model that ultimately runs at inference. Being able to recover native FP4 weights from the relaxed solution was a design goal from the outset, and we are glad that its practical value came through in your reading. We are also grateful for your comments on the experiments with identical candidate lists. It was encouraging to see that you found those comparisons informative about the contribution of coordination itself.

Your suggestion to compare with BPDQ gives us a complementary test in which each method retains its own weight format. Table 4 reports token perplexity on the same WikiText-2 windows and includes each method’s metadata in its checkpoint size.

How does LDC-FP4 differ from BPDQ in methodology and trade-offs?

BPDQ [1] improves low-bit fidelity by expanding group-level representation flexibility through learned scalar bit-plane coefficients that form a variable grid, paired with sequential Hessian-aware error compensation. LDC-FP4 operates within the fixed hardware-native FP4 candidate space and jointly optimizes complete legal scale-and-code tuples across all groups of an output row through a shared rr-dimensional dual variable. Updating that dual vector adjusts candidate preferences across every group in the row simultaneously. Rank-controlled support reduction then preserves the fractional mixture's coordinated aggregate residual while eliminating fractional weights outside at most rr groups, and the residual-variance recovery bound controls the objective change when rounding those remaining rr mixed groups into discrete native encodings.

Recent post-training quantization research explores complementary design axes, including variable bit-plane grids in BPDQ [1], adaptive second-moment transforms in WUSH [2], outlier-companion channel grouping for NVFP4 in OCGQuant [3], and trellis-coordinated discrete encoding in QTIP [4]. When evaluating LDC-FP4 as an encoder, we hold the coordinate transform, candidate lists, and post-refinement fixed so that reconstruction and speed advantages directly reflect row-wide candidate coordination.

Controlled W4A16 quality and storage results

Section 5.3 and Table 4 now include the requested controlled W4A16 comparison on Qwen3-8B. Both methods preserve their intended weight representations and run with BF16 activations. Because BPDQ's official repository computes document-level word-normalized perplexity, we evaluate both checkpoints under a unified token-based WikiText-2 procedure using 2,048-token windows and a stride of 1,024. The comparison accounts for all quantization metadata and complete checkpoint storage.

MethodWeight format / groupEffective bits/weightMetadata MiBModel GiBToken PPL
BPDQVariable grid / 1284.674558.06.0989.684 ± 0.018
LDC-FP4NVFP4 / 164.549454.55.9979.527 ± 0.011

Token PPL is reported as mean ± sample standard deviation across three paired calibration seeds.

Under matched activation precision and a unified evaluation setup, LDC-FP4 achieves lower WikiText-2 token perplexity with fewer effective bits per weight, 103.5 MiB less quantization metadata, and a smaller complete checkpoint. Because LDC-FP4 outputs standard NVFP4 encodings and discards all auxiliary optimization state after calibration, the exported model remains directly compatible with native FP4 inference kernels. In Appendix B.7, matched-hardware benchmarks verify unchanged prefill and decode throughput relative to native GPTAQ. The controlled BPDQ comparison establishes the quality and storage advantage of coordinated NVFP4 encoding relative to a variable-grid representation. Our native-kernel benchmarks confirm zero serving overhead on supported hardware.

References

[1] Chen et al. BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models. ICML 2026.

[2] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[3] Yao et al. OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization. EMNLP 2026.

[4] Tseng et al. QTIP: Quantization with Trellises and Incoherence Processing. NeurIPS 2024.

AuthorsOpenReview ↗

Response to Reviewer iPEV (2/3): Format transfer and lower-bit encodings

Equations (5) through (12), the rr-dimensional dual, rank-controlled support reduction, and the residual-variance recovery bound depend solely on finite per-group candidate lists characterized by local scalar costs αbk\alpha_{bk} and rr-dimensional shared-residual vectors vbkv_{bk}. None of these optimization or recovery results require a nonuniform FP4 codebook or floating-point exponent bits, so they apply equally to integer and floating-point group encodings.

Format-specific components reside in the group width, legal scale and zero-point grids, candidate list construction, and binary checkpoint packing. For INT4, each complete candidate tuple combines a group's integer codes with a legal scale and zero point. For OCP MXFP4, each candidate pairs 32 E2M1 codes with a neighboring legal E8M0 power-of-two shared scale. Once legal group candidates are scored, the same dual solver and support-reduction algorithm operate directly on their local costs and shared residuals. Parameters shared across multiple groups are either held fixed during a candidate-selection solve or bundled into a joint candidate unit so that every selected combination remains hardware-valid, preserving native inference speed relative to an available backend for that target format.

We have instantiated this pipeline end to end in MXFP4. On Qwen3-8B with group size 32 and E8M0 scales at the unchanged default rank r=16r=16, LDC-FP4 reduces WikiText-2 token perplexity from 9.918 to 9.786, requiring 0.61 GPU-hours for full calibration compared with 0.29 GPU-hours for the GPTAQ initialization. This cross-format experiment complements our native NVFP4 evaluations by exercising both a larger group width and a pure-exponent scale grid.

Modularizing representation geometry and discrete encoding optimization aligns well with recent low-bit quantization literature. WUSH [1] derives adaptive transforms for both floating-point and integer quantizers, QuIP# [2] and VPTQ [3] design structured vector codebooks for extreme compression, and AQLM [4] pairs learned additive codebooks with joint block-level reconstruction. LDC-FP4 supplies a principled cross-group coordination stage once a target format defines its finite legal candidate lists.

The mathematical framework extends directly to finite lists of complete legal two-bit or three-bit group encodings. When layer inputs and reconstruction targets are fixed, the curvature matrix HH and shared rank rr remain unchanged, and Theorem 1 still trims any feasible mixture to at most rr mixed groups while preserving the mixture's aggregate shared residual. Candidate codebook geometry enters Proposition 2 through the within-group residual variance

Vb(pb)=∑kpbk∥vbk∥22−∥∑kpbkvbk∥22.V_b(p_b) = \sum_k p_{bk} \Vert v_{bk} \Vert_2^2 - \left\Vert \sum_k p_{bk} v_{bk} \right\Vert_2^2.

Having at most rr mixed groups bounds how many groups undergo discrete rounding, while Vb(pb)V_b(p_b) measures the dispersion of surviving residual vectors within each mixed group. Whenever low-bit candidate lists admit cross-group error cancellation in the shared subspace, dual coordination exploits that cancellation and confines the remaining rounding variance to at most rr groups. Our new MXFP4 evaluation demonstrates that the default shared rank r=16r=16 already transfers effectively across a doubled group size and a coarser power-of-two scale grid.

References

[1] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[2] Tseng et al. QuIP#: Even Better LLM Quantization with Hadamard

AuthorsOpenReview ↗

Response to Reviewer iPEV (3/3): Practical value and calibration cost

Calibration runs once offline to produce a standard packed checkpoint whose improved accuracy benefits every subsequent inference request without adding serving operators or runtime memory overhead. Against FAAR across our six-model suite, LDC-FP4 improves the six-task macro-average from 74.9 to 75.6, MMLU from 73.0 to 73.4, GSM8K from 86.8 to 87.1, and GPQA Diamond from 38.6 to 38.8, and lowers teacher-student token KL from 0.0596 to 0.0529. In addition, the newly added paired-seed evaluation in Table 3 demonstrates lower perplexity and higher task accuracy on every individual model across calibration seeds.

Investing offline compute to maximize the quality of a fast deployment representation is well established in post-training quantization. SpinQuant [1] optimizes rotation matrices prior to deployment, GPTAQ [2] performs error-compensated asymmetric calibration, WUSH [3] fits data-dependent block transforms, and FlatQuant [4] invests multi-hour offline calibration to learn affine transforms for efficient deployment. For checkpoints deployed repeatedly or distributed for reuse, the quality improvement benefits subsequent workloads at unchanged native-kernel throughput.

Within the discrete optimization stage, LDC-FP4 also establishes a favorable cost-quality frontier against wide combinatorial search. On identical candidate lists in our full-study comparison (Table 9), LDC-FP4 achieves lower reconstruction error and lower token KL than Beam-256 using 41.5% of its calibration time. It also outperforms beam search under a matched per-layer GPU-time budget. Table 8 reports the complete end-to-end calibration breakdown, including moment collection, curvature factorization, candidate generation, dual coordination, discrete recovery, initialization, and final refinement.

A new matched-candidate benchmark on a 12-module layer set confirms that this encoding speedup holds across formats and GPU generations. Relative to Beam-256, LDC-FP4 accelerates encoding by 2.34×2.34\times for NVFP4 on B200, 2.43×2.43\times for MXFP4 on B200, and 2.25×2.25\times for MXFP4 on H100 SXM. MXFP4 held-out NMSE improves by 10.8% relative to independent group selection. Moreover, our new end-to-end Qwen3-8B MXFP4 and DeepSeek-V2-Lite NVFP4 checkpoints reduce perplexity with complete calibration costs of 0.61 and 1.58 GPU-hours, respectively, offering a practical operating point for producing reusable native models.

References

[1] Liu et al. SpinQuant: LLM Quantization with Learned Rotations. ICLR 2025.

[2] Li et al. GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration. ICML 2025.

[3] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[4] Sun et al. FlatQuant: Flatness Matters for LLM Quantization. ICML 2025.

Reviewer_dGEaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a decomposition of the reconstruction metrics in FP4 quantization, which leads to a convex optimization formulation approximating the initial problem. This allows one to go beyond the assumption of independent groups, while keeping the optimization tractable.

Critical strengths

The problem studied is interesting and valuable to the community.

The paper provides a valuable mathematical analysis of the problem.

The paper is in general well written.

Critical weaknesses
  1. Methodology:

1.1: Concretely, what is the assumption behind the decomposition into diagonal + low-rank, in particular the low-rank part?

1.2: In Eq. 1, what is the purpose of S?

1.3: How feasible is it to solve the convex relaxation of Eq. 7? It still cannot be split into subproblems at the block level, so I would expect it to be expensive.

1.4: I suspect that Section 3.4 aims to address the issue I mentioned above, although not stated explicitly. However, then how expensive is it to solve for \lambda?

  1. Experiments: The results in Tables 1 and 2 are fine, although not truly impressive.
Questions for authors

See Weaknesses above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer dGEa (1/2): Curvature modeling and regularization

Thank you for your encouraging review. We are glad you see value for the quantization community in studying how errors interact across groups. We particularly appreciate your positive assessment of the mathematical analysis. Finding a tractable way to account for those interactions was our motivation, so your interest in the derivation and your kind words about its presentation were especially welcome.

1.1. What supports the block-diagonal plus low-rank curvature model?

LDC-FP4 decomposes the layer reconstruction curvature as H=D+UU⊤+RH=D+UU^\top+R, combining a dense block-diagonal matrix DD aligned with the quantization groups and a shared rank-rr factor UU⊤UU^\top across groups. Section 3.5 includes the approximation residual RR in the original-metric analysis. Each dense block of DD models within-group curvature, and the columns of UU capture the dominant shared directions of cross-group coupling. Under this decomposition, every complete group candidate is summarized by a scalar local cost and a compact rr-dimensional shared-residual vector, turning coupled row-level encoding into a structured optimization problem.

Empirical properties of LLM activations motivate this structure. Data-dependent coordinate transforms such as SpinQuant [1] and WUSH [2] reshape second-moment geometry to improve low-bit quantization. In our primary benchmarks, the WUSH transform is held fixed across competing encoders so that quality differences isolate the selection of native scales and codes. Because LDC-FP4 fits its curvature factors directly from each layer's calibration activations, the shared rank-rr subspace tracks the input covariance of the specific linear module being encoded.

Recent curvature-aware quantization methods provide independent support for pairing local blocks with a low-rank component. FIMA-Q [4] uses diagonal-plus-low-rank Fisher estimation to capture informative quantization curvature at tractable cost, and LS-ViT [5] combines local block and low-rank Hessian terms during low-bit reconstruction. For large language models, ResQ [6] applies activation PCA to concentrate higher precision within a low-dimensional subspace carrying high activation variance. Building on this geometric property, LDC-FP4 estimates both dense group curvature and shared low-rank directions during calibration to coordinate native candidate choices, discarding all auxiliary factors prior to deployment.

Direct diagnostics and component ablations verify both terms on held-out data. At the default rank r=16r=16, the directional approximation error has a median of 2.4% and a 90th percentile of 7.7%. Removing the shared low-rank factor increases held-out reconstruction error by 6.5%, and removing dense within-group blocks increases it by 20.4%. Raising the rank from 16 to 32 yields a further 0.8% reduction in reconstruction error. Gains also concentrate in output rows with high shared-error fractions, reducing reconstruction error by 20.1% in the highest shared-error quartile compared with 3.2% in the lowest quartile. Section 5.5 and Tables 12 and 13 consolidate these diagnostics and ablations.

1.2. What is the purpose of SS?

In Equation (1), the penalty γ(q−w)⊤S(q−w)\gamma(q-w)^\top S(q-w) regularizes deviations from the pretrained weights in activation-scaled coordinates. We set S=BlkDiag⁡(Z⊤Z/n)+δIS=\operatorname{BlkDiag}(Z^\top Z/n)+\delta I with δ>0\delta>0. The dense group blocks preserve local within-group correlations, and the ridge term δI\delta I guarantees positive definiteness. Adding γS\gamma S ensures that H=Z⊤Z/n+γSH=Z^\top Z/n+\gamma S maintains strictly positive curvature even along input directions with weak calibration excitation.

At the same time, SS preconditions the low-rank curvature approximation through the whitened matrix C=S−1/2HS−1/2C=S^{-1/2}HS^{-1/2}. Fitting the shared factor in these normalized coordinates balances activation scales across groups. Section 4.1 details this normalization and maps the fitted factors back to the original weight coordinates. Following the asymmetric calibration formulation introduced by GPTAQ [3], the reconstruction target pairs quantized-prefix inputs with full-precision BF16 teacher outputs to compensate for accumulated upstream quantization error.

References

[1] Liu et al. SpinQuant: LLM Quantization with Learned Rotations. ICLR 2025.

[2] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[3] Li et al. GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration. ICML 2025.

[4] Wu et al. FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation. CVPR 2025.

[5] Hwang et al. LS-ViT: Least-Squares Hessian Based Block Reconstruction for Low-Bit Post-Training Quantization of Vision Transformers. CVPR 2026.

[6] Saxena et al. ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals. ICML 2025.

AuthorsOpenReview ↗

Response to Reviewer dGEa (2/2): Optimization cost and empirical gains

1.3 and 1.4. How is the coupled relaxation solved efficiently?

Passing to the rr-dimensional Lagrangian dual makes global cross-group coordination computationally efficient. For each output row, the dual objective takes the form

g(λ)=L0−∥λ∥22+∑bmin⁡k(αbk+2λ⊤vbk)g(\lambda) = \mathcal L_0 - \Vert \lambda \Vert_2^2 + \sum_b \min_k \big( \alpha_{bk} + 2\lambda^\top v_{bk} \big)

Conditioned on a fixed dual vector λ∈Rr\lambda \in \mathbb{R}^r, candidate scores decouple completely across all BB groups, channeling global coupling through a single rr-dimensional vector. Our GPU implementation replaces each hard inner minimum with a smooth soft minimum and batches candidate scoring and gradient evaluation across output rows in parallel. With BB groups and at most KK candidates per group, each dual evaluation requires O(BKr)O(BKr) operations. Under our default settings of r=16r=16 and K≤33K \leq 33, the coordination state remains small even on wide linear layers. Section 4.3 now presents this complexity analysis directly alongside Algorithm 1.

Once dual optimization produces a fractional primal mixture, rank-controlled support reduction trims surplus fractional weights while preserving the mixture's aggregate shared residual, leaving at most rr mixed groups across the row. Subsequent discrete recovery only needs to resolve those remaining rr groups, concentrating discrete search onto a compact subproblem. Recent work such as DiscQuant [1] shows the value of constructive geometric rounding on fixed quantization grids, and our primal trimming step connects directly to the constructive Shapley-Folkman framework of Dubois-Taine and d'Aspremont [5] for separable nonconvex problems under linear coupling constraints. Fixing the rr-dimensional aggregate residual imposes rr affine constraints across the product of group simplices, yielding the rank-rr support guarantee over complete native scale-and-code encodings.

Measured calibration timings confirm this computational advantage. Under identical candidate lists in Table 9, LDC-FP4 reaches a normalized reconstruction error of 0.939 and a teacher-student token KL of 0.941, improving upon Beam-256 (0.947 and 0.949) while reducing normalized calibration time from 21.38 to 8.87, a 2.41×2.41\times speedup. Under an equal GPU-time budget, LDC-FP4 achieves 1.4% lower held-out reconstruction error than beam search. Initialization, candidate generation, dual coordination, discrete recovery, and final refinement are all included in these end-to-end timings.

2. How should the practical gains be assessed?

LDC-FP4 delivers consistent accuracy improvements across models and tasks at a fixed coordinate transform and hardware-native deployment format. It reduces WikiText-2 and C4 perplexity on every evaluated model and leads all quantized baselines on the six-task macro-average, MMLU, GSM8K, GPQA Diamond, and teacher-student token KL. Our new evaluations on Qwen3-8B in MXFP4 and DeepSeek-V2-Lite in NVFP4 likewise achieve clear perplexity improvements over format-matched GPTAQ initializations.

Improving accuracy within strict hardware-native FP4 representations is an active focus of recent quantization research, including WUSH [2], OCGQuant [3], and ARCQuant [4]. LDC-FP4 contributes a principled optimization solver that coordinates legal group encodings across an entire weight row. Because all auxiliary optimization variables are discarded after calibration and the exported checkpoint uses standard packed weights and native inference kernels, every downstream inference call benefits from improved accuracy with zero serving overhead.

References

[1] Chee et al. DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory. COLT 2025.

[2] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.

[3] Yao et al. OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization. EMNLP 2026.

[4] Meng et al. ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs. ACL 2026.

[5] Dubois-Taine and d'Aspremont. Frank-Wolfe meets Shapley-Folkman: a systematic approach for solving nonconvex separable problems with linear constraints. Mathematical Programming, 219, 501–551, 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and responses

Dear Reviewers and Area Chair,

We sincerely thank Reviewers dGEa, iPEV, and REV5 for their constructive reviews and thorough engagement with our manuscript. We are encouraged by the positive recognition of our problem formulation, mathematical analysis, controlled experiments, and native deployment design across the reviews. Reviewer dGEa describes our problem as "interesting and valuable to the community," highlights our "valuable mathematical analysis," and notes that the paper is "in general well written." Reviewer iPEV identifies a "coherent optimization contribution under native-format constraints," commends our controlled attribution of gains to cross-group coordination, and reports "no substantive concerns about the paper’s soundness." Reviewer REV5 notes that our "core reframing is good," observes that the matched-candidate experiments cleanly separate the benefit of coordination from candidate construction, and highlights our practical zero-deployment-cost design with its per-row objective safeguard.

Guided by the reviewers' questions, we have revised the manuscript and completed new evaluations expanding format, architectural, hardware, and statistical coverage.

  • Controlled W4A16 quality and storage comparison with variable-grid quantization. Section 5.3 and Table 4 compare LDC-FP4 with BPDQ on Qwen3-8B under BF16 activations and a unified windowed token-perplexity evaluation, including quantization metadata and complete model size. LDC-FP4 achieves lower WikiText-2 token perplexity with fewer effective bits per weight, less metadata, and a smaller complete checkpoint. The exported weights retain standard NVFP4 kernel compatibility.
  • End-to-end OCP MXFP4 transfer and complexity analysis. Instantiating LDC-FP4 for MXFP4 (groups of 32 and E8M0 power-of-two shared scales) at the unchanged default rank r=16r=16 reduces Qwen3-8B WikiText-2 token perplexity from 9.918 to 9.786 in 0.61 GPU-hours of full calibration. We also clarify why the rr-dimensional dual and rank-rr support bound remain invariant to group width and scale format, accompanied by explicit computational and storage scaling expressions for g=32g=32.
  • Validation on MLA and mixture-of-experts architectures. On DeepSeek-V2-Lite, which combines MLA projections with shared and routed MoE experts, LDC-FP4 at the default rank r=16r=16 reduces WikiText-2 token perplexity from 7.463 to 7.309 in 1.58 GPU-hours. Module-level curvature diagnostics confirm low median directional errors across MLA projections, shared experts, and routed experts, with targeted rank-32 follow-ups resolving nearly all remaining high-error modules.
  • Encoding speedups across formats and GPU generations. A new 12-module matched-candidate benchmark evaluates NVFP4 on B200, MXFP4 on B200, and MXFP4 on NVIDIA H100 SXM. LDC-FP4 achieves encoding speedups of 2.34×2.34\times, 2.43×2.43\times, and 2.25×2.25\times over Beam-256 alongside consistent held-out NMSE reductions over independent selection, complementing the full-study NVFP4 solver comparison in Table 9.
  • Paired-seed consistency across all six models. Table 3 reports model-specific paired differences (mean ± sample standard deviation over three shared calibration seeds) relative to FAAR for WikiText-2 perplexity, C4 perplexity, and six-task accuracy. Every evaluated model improves on all three metrics, with seed-to-seed standard deviations consistently smaller than the observed improvements.
  • Clearer presentation of curvature modeling and dual optimization. The revised manuscript expands the exposition of activation-scaled regularization and whitening via SS, parallel GPU evaluation of the smoothed rr-dimensional dual alongside Algorithm 1, rank-controlled support reduction on fractional mixtures, and discrete recovery, supported by streamlined pseudocode in the appendix.

These additions extend our original matched-candidate and downstream evaluations across a second mainline FP4 standard, an architecture combining MLA and sparse MoE layers, and a second GPU generation. LDC-FP4 provides a principled global encoding method that delivers consistent quality improvements, efficient offline calibration, and unchanged native FP4 serving.

We believe these updates have substantially strengthened the manuscript, and we remain happy to address any further questions during the discussion phase. Thank you again for your time and dedication in reviewing our work.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.