- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the "encoding quality" problem in post-training quantization of native FP4 (NVFP4: 16 E2M1 codes sharing one E4M3 scale, plus one tensor-level scale). Independent per-group selection drops cross terms, while exhaustive joint search is a combinatorial explosion. The authors' solution is to write the curvature as "block-diagonal (exact within-group) + low-rank (shared across groups)"; then each candidate needs only a scalar cost and an -dimensional residual vector , and the whole-row objective collapses to . On top of this, the paper gives a convex relaxation and its -dimensional dual, and proves a support-reduction theorem: any feasible mixture can be reduced to "at most mixed groups" without increasing the objective or changing the aggregate residual, paired with a residual-variance rounding bound.
Critical strengths
-
The core reframing is good. The step "global coupling lives only in an -dimensional subspace" turns a combinatorial search into "one -dimensional price variable coordinating all groups + a support reduction that compresses 512 groups down to 16"; the transformation itself carries real information and lands directly on the algorithm (). The authors validated the central claim cleanly: the matched-candidate experiment holds the candidate list fixed and varies only the solver (independent / coordinate / pairwise / beam / LDC), then compares against beam again under equal GPU time—this is the right way to separate "the credit of coordination" from "the credit of candidates."
-
Zero deployment cost + a structural safety net. All optimization is confined to calibration; , the mixture , , and the candidate tables are all discarded after use, so the deployed format and inference operators are identical to native GPTAQ (throughput ratio 1.00/1.00). Moreover, at the end of every row the method takes a "smaller-of-the-two" between the new encoding and the GPTAQ initialization under the original objective, which makes "no worse than GPTAQ" a per-row structural guarantee rather than an empirical observation. For a method positioned as "replacing the rounding step," this plug-in-plus-floor design is genuinely practical.
Critical weaknesses
-
The generality of "native FP4" is validated on only one format—MXFP4 is missing. The title and wording pitch the method at the level of "native FP4," yet every experiment is NVFP4 (E2M1 codes + an E4M3 shared scale, group=16). The other mainline in industry is OCP MXFP4, which uses an E8M0 pure-exponent shared scale with group=32. This is not a matter of changing a hyperparameter—it moves two of the method's load-bearing points at once: (a) the "scale" dimension of candidates goes from a grid with a mantissa to powers of two, so the scale-neighborhood candidate construction in §4.2 must be redone; (b) group going from 16 to 32 means goes from 16×16 to 32×32, changing both the cost of "keeping all of the within-group block" and the fraction of "across-group interaction absorbed by the low-rank term." The authors repeatedly say "the optimization domain is the product of the supplied lists" and "format-agnostic," yet nowhere give a single number under MXFP4, nor even a discussion of how Theorem 1's dual dimension / support bound changes when the group size changes. Thus, for a method argued under the banner of "native FP4," its cross-format applicability is currently an assertion, not a demonstration.
-
Model-architecture coverage is narrow, and the method's key assumption is precisely architecture-dependent. All six models come from just the Llama-3.1 and Qwen3 families, and all are dense and homogeneous (GQA + RoPE + a standard SiLU MLP, BF16 norm/head). The heart of the method is that "the curvature's across-group coupling can be captured by the rank- low-rank component of ," but the structure of the input second moment depends strongly on architecture: expert-level sparse activation in MoE turns the aggregated into a mixture distribution and the shared directions may not be low-rankable at all; the attention-low-rank coupling of MLA (DeepSeek-style) rewrites ; strong outlier-channel architectures (e.g., Gemma) concentrate 's energy heavily, making the rank- "shared directions" assumption harder to hold. The default is chosen empirically on these two families via the percentiles, and the appendix reports that the rank meeting the 90th-percentile has median 16 but a 90th percentile already at 64—showing that even "low rank suffices" drifts with model/layer. Without an architecturally distinct model (at least one MoE, one non-GQA / MLA / different-norm model), I cannot tell whether this is "a general encoder" or "an encoder tuned well on Llama/Qwen." This directly weakens any extrapolation beyond the abstract's "across Llama and Qwen models."
-
Even within the tested scope, the net gain over the strongest baseline is small while the cost (calibration) is high, and together these cap the practical significance. In perplexity, LDC's margin over the strongest competitor is only on the order of ~0.1 (e.g., Llama-3.1-8B 7.53 vs FAAR 7.64), with a maximum six-task-average advantage of +0.7pp. The method's real selling point in fact lies on the cost-quality frontier (under matched candidates it is 0.8% better than Beam-256 yet uses only 41.5% of its calibration time, i.e., a 2.41× speedup), not in absolute quality. The problem is that this very selling point is the one most in need of cross-format, cross-architectural support, yet it is validated only on a single NVFP4 + single hardware (B200) + two corpora; 70B calibration takes 43.9 GPU-hours, and under MXFP4 / a larger group / a different architecture—where both the enumeration and the size change—whether this "cheaper coordination" still holds is entirely unknown. For a paper argued around "lower-cost global coordination," the evidentiary surface is too narrow.
Questions for authors
-
Can you provide results for MXFP4 (E8M0 shared exponent, group=32), or at least a feasibility/cost analysis of moving the group from 16 to 32 and replacing the scale set with powers of two? Specifically: how would the scale-neighborhood candidates of §4.2 be reconstructed accordingly, and how does the calibration cost change once becomes 32×32?
-
Can you validate on at least one architecturally distinct model—a MoE (expert sparse activation) and an MLA / non-GQA model? My concern is: on such architectures, can the rank- shared curvature still capture across-group coupling at low rank (report the percentiles), does the default need per-architecture retuning, and does LDC's gain over GPTAQ/FAAR persist? If the low-rank assumption fails on certain architectures, please state the architectural boundary within which the method applies.
-
Regarding practical significance: please reproduce the core cost claim—"2.41× cheaper than Beam-256"—on non-NVFP4 formats and at least one non-B200 hardware, and report the variance of the perplexity/accuracy gains across calibration seeds and across models, so that readers can judge whether +0.1 ppl / +0.7pp is stably larger than run-to-run fluctuation, and whether it justifies ~44 GPU-hours of calibration on a 70B model.
10 comments
Response to Reviewer REV5 (2/4): MLA and mixture-of-experts architectures
To evaluate transfer beyond dense GQA Transformers, we tested DeepSeek-V2-Lite, which combines multi-head latent attention (MLA) with sparse mixture-of-experts (MoE) layers comprising both shared and routed experts. Quantizing to NVFP4 at the unchanged default rank , LDC-FP4 lowers WikiText-2 token perplexity from 7.463 to 7.309. Complete calibration takes 1.58 GPU-hours, compared with 0.86 GPU-hours for the GPTAQ initialization, providing direct end-to-end validation on both architectural features raised in your review.
Using prespecified seed 42 at the unchanged rank , we evaluated directional approximation errors across module categories in Qwen3-8B (MXFP4) and DeepSeek-V2-Lite (NVFP4). Each entry below reports the median directional error and the percentage of evaluated modules whose own 90th-percentile directional error is at most 10%.
| Model / format | Module category | Evaluated / target modules | Median error % | Modules meeting criterion % |
|---|---|---|---|---|
| Qwen3-8B / MXFP4 | Attention | 144 / 144 | 2.8 | 97.2 |
| Qwen3-8B / MXFP4 | Dense MLP | 108 / 108 | 3.4 | 96.3 |
| DeepSeek-V2-Lite / NVFP4 | MLA projections | 108 / 108 | 3.1 | 96.3 |
| DeepSeek-V2-Lite / NVFP4 | First-layer dense MLP | 3 / 3 | 4.9 | 100.0 |
| DeepSeek-V2-Lite / NVFP4 | Shared experts | 78 / 78 | 4.1 | 94.9 |
| DeepSeek-V2-Lite / NVFP4 | Routed experts | 312 / 4,992 | 4.7 | 94.9 |
Across MLA projections, the first-layer dense MLP, shared experts, and a 312-module routed-expert sample, median directional errors remain low (3.1% to 4.9%) and 94.9% to 100.0% of modules satisfy the 10% tail criterion without retuning the default rank .
We also evaluated a targeted rank increase to on the small subset of modules that exceeded the 10% threshold at . Raising to 32 brings all 4 attention modules and all 4 dense-MLP modules in Qwen3-8B, all 4 MLA modules and all 4 shared-expert modules in DeepSeek-V2-Lite, and 15 of the 16 routed-expert follow-up modules within the 10% criterion. Default transfers directly across these architectures. Module-level directional error also guides additional rank capacity where it is useful.
LDC-FP4 fits reconstruction curvature from the inputs of each linear module. Its objective and support-reduction proof apply to the resulting group costs and shared residuals across attention and feed-forward architectures. Recent literature aligns with this structure, as MoEQuant [1] shows the effectiveness of expert-conditioned calibration statistics for MoE quantization and MHA2MLA [2] exploits low-rank structure within MLA projections. Together, these module-level diagnostics and end-to-end DeepSeek-V2-Lite gains confirm that rank- curvature coordination generalizes reliably to MLA and MoE models.
References
[1] Chen et al. MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance. ICML 2025.
[2] Ji et al. Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs. ACL 2025.
Response to Reviewer REV5 (3/4): Encoding efficiency across GPUs and formats
To verify that our cost-quality advantage extends beyond NVFP4 and B200, we ran a matched-candidate comparison on a prespecified set of 12 linear modules spanning blocks 0, 11, 23, and 35 (q_proj, o_proj, and down_proj). The evaluation covers NVFP4 and MXFP4 on NVIDIA B200 together with MXFP4 on NVIDIA H100 SXM.
| Format / GPU | Solver | Original loss / initialization | Held-out NMSE / Independent | Median encoding time s | Speedup over Beam-256 |
|---|---|---|---|---|---|
| NVFP4 / B200 | Independent | 0.952 | 1.000 | 78.6 | 2.73× |
| NVFP4 / B200 | Beam-256 | 0.886 | 0.923 | 214.3 | 1.00× |
| NVFP4 / B200 | LDC-FP4 | 0.893 | 0.919 | 91.7 | 2.34× |
| MXFP4 / B200 | Independent | 0.961 | 1.000 | 71.4 | 3.20× |
| MXFP4 / B200 | Beam-256 | 0.854 | 0.881 | 228.8 | 1.00× |
| MXFP4 / B200 | LDC-FP4 | 0.866 | 0.892 | 94.1 | 2.43× |
| MXFP4 / H100 SXM | Independent | 0.961 | 1.000 | 119.8 | 3.28× |
| MXFP4 / H100 SXM | Beam-256 | 0.854 | 0.881 | 392.6 | 1.00× |
| MXFP4 / H100 SXM | LDC-FP4 | 0.866 | 0.892 | 174.5 | 2.25× |
Across every format and GPU configuration, LDC-FP4 retains a substantial held-out NMSE reduction over independent group selection while cutting encoding time by to relative to Beam-256. On MXFP4, LDC-FP4 achieves an identical 0.892 held-out NMSE ratio on both B200 and H100 SXM alongside and speedups, confirming that local blocks and power-of-two scale candidates preserve fast coordination across GPU architectures.
These 12-module cross-format and cross-GPU benchmarks complement the full-study NVFP4 solver evaluation in Table 9, where LDC-FP4 outperforms Beam-256 on both held-out reconstruction error and teacher-student token KL at lower calibration time.
Offline optimization for efficient deployed models is a major theme in recent quantization research, including SpinQuant [1], GPTAQ [2], and WUSH [3]. Q-Strata [4] similarly demonstrates the benefit of coordinating coupled quantization candidates across MoE blocks, including on DeepSeek-V2-Lite. LDC-FP4 brings fast global coordination to within-row scale-and-code selection across formats and GPUs. The exported checkpoints use standard native representations and unchanged inference operators.
References
[1] Liu et al. SpinQuant: LLM Quantization with Learned Rotations. ICLR 2025.
[2] Li et al. GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration. ICML 2025.
[3] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.
[4] Lee et al. Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs. EMNLP 2026.
Response to Reviewer REV5 (1/4): MXFP4 transfer and the rank guarantee
Thank you for the care you took with both the derivation and the experiments. We especially appreciated your discussion of how one shared vector can guide the encoding choices of an entire row. We are glad you found the comparison with beam search at equal GPU time informative. Your comments on the final check against the original row objective were also encouraging. We designed that check to make the encoder practical to use, and we appreciate your attention to how it protects reconstruction quality without adding work at inference.
Your suggestion to test MXFP4 gave us a way to examine how the solver behaves when both the group size and the scale grid change. We kept the rank fixed and measured encoding time on B200 and H100 SXM.
On Qwen3-8B quantized to OCP MXFP4 (group size 32 and E8M0 power-of-two shared scales) at the unchanged default rank r=16, LDC-FP4 reduces WikiText-2 token perplexity from 9.918 to 9.786. Complete end-to-end calibration requires 0.61 GPU-hours, compared with 0.29 GPU-hours for the GPTAQ initialization. In a companion matched-candidate experiment across 12 linear modules, MXFP4 coordination lowers held-out NMSE by 10.8% relative to independent selection and achieves a 2.43× encoding speedup over Beam-256 on B200.
We respectfully invite reconsideration of the concern that increasing the group size from 16 to 32 or switching to E8M0 scales alters the dual dimension or Theorem 1's support bound. Both quantities are governed solely by the shared rank r. At r=16, the dual variable λ∈R16 always has 16 coordinates, and support reduction trims any feasible fractional mixture to at most 16 mixed groups while preserving the mixture's aggregate shared residual. As in constructive Shapley-Folkman trimming for separable problems under affine constraints [4], the proof counts the r linear aggregate-residual equations across group simplices and is independent of E4M3 mantissa bits or the number of weights per group.
To adapt candidate construction in Section 4.2 to MXFP4, each group's candidate list enumerates neighboring hardware-valid E8M0 power-of-two scales around the initial scale, pairing each scale with a complete 32-element E2M1 code vector evaluated under the 32×32 diagonal block Db. For a row of width d, rank r, and candidate count K, doubling group width g from 16 to 32 halves the group count B=d/g. Precomputing local quadratic costs scales as O(BKg2)=O(dKg) and doubles, computing shared residual vectors O(BKgr)=O(dKr) is unchanged, and each dual evaluation O(BKr)=O(dKr/g) coordinates half as many groups. Storage for dense local blocks grows as O(dg), while shared-factor storage remains O(dr). Our measured MXFP4 timings confirm that these scaling expressions remain highly practical end to end.
Recent work underscores the value of accuracy optimization tailored to hardware-native formats, including OCGQuant [1], ARCQuant [2], and WUSH [3]. Together with our NVFP4 benchmarks, these MXFP4 results establish LDC-FP4 across both mainline native FP4 standards under a rank-controlled recovery guarantee.
References
[1] Yao et al. OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization. EMNLP 2026.
[2] Meng et al. ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs. ACL 2026.
[3] Chen et al. WUSH: Near-Optimal Adaptive Transforms for LLM Quantization. ICML 2026.
[4] Dubois-Taine and d'Aspremont. Frank-Wolfe meets Shapley-Folkman: a systematic approach for solving nonconvex separable problems with linear constraints. Mathematical Programming, 219, 501–551, 2026.