- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper propose a QAD framework for MXFP4 and NVFP4, the main idea is to keep majority of the model weight fixed while updating a only a small amount of important parameters for NVFP4/MXFP4. They also propose trainable residual compensation, that is an extra trainable weight set for important parameters of specific importants layers, after selecting those layers using a gradient-based heuristic under specific model budget.
Critical strengths
-
The framework is practical, target a practical setting in quantization, for NVFP4/MXFP4
-
Good empirical result. The proposed framework outperform recent PTQ baseline across Llama/Qwen models.
-
Training efficiency. Updates only a small subset of parameters, significantly reduces trainable-state memory.
Critical weaknesses
-
Limited technical novelty. Parameter-efficient QAT and selective weight training are not new ideas. ARCQuant already use residual compensation, while the metrics proposed for channel selection and allocation are quite simple.
-
Unfair comparison. The paper propose a QAD framework, but the compared target are mostly PTQ method. PTQ often costs a lot less than QAD. And based on the ablation table, most of the improvement come from QAD. Would be better if we have some parameter-efficient QAD baseline
-
Does the proposed channel selection metric work better than simple metrics, like random, or gradient-based selection?
-
Why GRCA use NLL gradient but not KL despite the framework is trained with KL?
Questions for authors
Please answer Question in Weakness
2 comments
Thank you for the rebuttal. The new experiments partly address my concerns. The channel-selection results show gains over random selection, but I would still like to see how the proposed score compares with using gradients to select which weight columns to train.
However, some of my concerns are not fully addressed. My concern about limited novelty remains partially solved. LoDiQ learns from the full-precision model’s predictions, while the new QLoRA and EfficientQAT baselines do not, so not exactly fair comparison. Therefore, I would like to keep my original rating.
Thank you for recognizing LoDiQ’s practical relevance, accuracy improvements, and reduced trainable-state memory. We have strengthened the comparisons and clarified the contributions below.
1. Contribution beyond selective training and residual compensation
LoDiQ combines error-localized model-level distillation with explicitly controlled low-bit recovery capacity. Propagated-error analysis identifies the weight columns to update, independently trainable residual weights provide additional recovery capacity, and GRCA allocates that capacity under a global parameter budget.
The distinction from ARCQuant goes beyond making residual compensation trainable. ARCQuant uses an activation-magnitude threshold to determine a layer-dependent number of residual channels and duplicates their quantized weights. LoDiQ fixes the correction width at K=128 and independently optimizes the primary and residual weight blocks through model-level distillation. Each compensated layer expands its GEMM reduction dimension by only 128, making additional storage and computation predictable and limiting expansion where threshold-based compensation would select more channels. GRCA further controls which layers receive this expansion.
The learned correction remains in the target FP4 format and is packed into expanded low-bit GEMMs without a separate high-precision adapter branch. Our contribution concerns this combination of localized learning, budgeted recovery capacity, and uniform W4A4 deployment, not selective updates or residual compensation alone.
2. Training-based comparisons
We agree that PTQ comparisons alone are insufficient. Tables 3 and 19 now compare FP4-adapted QLoRA and EfficientQAT on Llama-3.2-3B, Llama-3-8B, and Qwen3-8B under both formats. For Qwen3-8B MXFP4:
Retained QLoRA adapters add high-precision branch computation; merging removes the branch but incurs requantization error. EfficientQAT is adapted from integer weight quantization to FP4 W4A4. MXFP4 scale optimization exhibits instability in our experiments, motivating validation-based checkpoint selection. Its training cost is substantial: Qwen3-8B MXFP4 requires 11.08 single-H20 hours, compared with 0.93 for LoDiQ. Section 4.3 and Appendix A.2 specify the configurations and timing scope.
These adapted baselines use next-token NLL and therefore constitute training-based comparisons, not parameter-efficient QAD baselines. We also extended full-parameter QAD to the original 800M-token scale using official code and data. On AceReason-Nemotron-1.1-7B, QAD obtains 61.89/66.67 on AIME25/26 using 495.13 GPU-hours; LoDiQ-Max obtains 62.33/67.67 with 1M tokens and 1.03 GPU-hours. Tables 4(b) and 9 report costs and accuracy at these respective budgets.
3. Comparisons with simpler reordering strategies
Table 7(a) compares random, activation-amax, and error-localized reordering followed by distillation. Each trains the last K=128 weight columns after reordering and uses the full residual-compensation budget of LoDiQ-Max.
Error-localized reordering achieves the highest zero-shot accuracy on both models and lower perplexity on Llama-3-8B.
4. Why NLL gradients for GRCA instead of KL?
GRCA allocates residual capacity before distillation. NLL gradients provide a teacher-independent model-level sensitivity signal for this decision. The subsequent KL objective optimizes the allocated parameters to recover teacher predictions; the two objectives serve different purposes.
Appendix A.3, Table 11 compares NLL- and KL-based allocation at the same 50% residual-parameter budget:
NLL yields lower perplexity on both models, with mixed zero-shot differences. NLL scoring uses a BF16 reference forward; KL scoring uses a quantized tail-only reference and a detached teacher. Thus, this compares two allocation implementations, not an isolated loss substitution. NLL remains our default because it provides effective allocation without requiring a teacher during scoring.
Thank you for the constructive suggestions.