- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes MorphoQuant, a post-training quantization (PTQ) framework for W4A4 omni-modal LLMs. It has two components:
- MDSC scores each input channel by activation range over log mean magnitude and selects channels above a local or global μ+2σ threshold. Within these channels, elements with |X| > γ·α_c keep their full quantization residual in a higher-precision branch: Y = W_Q X_Q + W_Q ΔX_sparse + b.
- MPO selects a per-channel activation clipping boundary α_c by grid search, minimizing an ℓ3 reconstruction loss plus a cosine loss. Because the residual threshold is γ·α_c, the boundary search also changes the compensated set.
Weights are NF4 (bitsandbytes). Experiments cover Qwen2.5-Omni-3B/7B (ScienceQA, MMMU, Video-MME, AIR-Bench) and InternVL2.5-8B (ScienceQA, MMMU). The paper reports non-collapsed W4A4 accuracy where the reproduced MBQ and MQuant collapse, further gains with HiF4/HiF8 activation formats, and a three-row ablation.
Coupling the clipping boundary with residual selection is sensible. However, the omni-modal motivation does not shape the method, the delta over existing outlier-retention methods is not isolated, and the experiments cannot support the claims.
Critical strengths
S1. The paper evaluates audio (AIR-Bench) and video (Video-MME) on a genuine omni model, where public W4A4 results are scarce. This could become a useful reference point. It is not decisive, because the numbers are hard to interpret without an FP16 reference and against collapsed baselines. The authors also hedge their claims carefully ("reproduced baselines", "we do not infer hardware acceleration").
Critical weaknesses
W1. The novelty delta is not isolated, and the cost of the high-precision branch is not reported (primary concern).
- Delta not isolated. Relative to QUIK, Atom and LLM.int8() (all cited in Sec. 2.3), the delta is element-wise selection within selected channels, plus a threshold tied to α_c. There is no comparison against channel-level retention under a matched high-precision budget, so it is unclear whether this delta matters.
- Cost not reported. MDSC keeps the full residual at selected positions, and the selection is input-dependent at inference. The main text reports neither residual density, γ, effective bits per activation, nor latency and memory. Without these, "W4A4" is not comparable to pure W4A4 baselines; a dense enough branch approaches A16. The authors list these as future work, but the current claims depend on them.
- The INT4 tensor-core story in Fig. 1 is unsupported. Per-input-channel activation boundaries α_c cannot be factored out of an integer GEMM unless they are folded into the weights, which the paper does not discuss. In addition, bitsandbytes NF4 weights are dequantized to FP16 at matmul time.
- The omni-modal framing does not reach the method. Neither MDSC nor MPO uses modality information, so Sec. 3.2 motivates the paper but not the method.
W2. The experiments are uninformative.
- No full-precision reference. Without FP16 rows, the degradation cannot be measured.
- Baselines appear broken. MBQ and MQuant score near 0 at W4A4, and MQuant also collapses at W8A8 on Video-MME (0.37) and AIR-Bench (0.00). The paper's own ablation "Baseline" (54.73/30.44) far exceeds MBQ (9.88/0.00). Both point to broken baseline adaptations, and outperforming collapsed baselines says little about the method.
- Strong W4A4 baselines are missing on Qwen2.5-Omni: QuaRot, SpinQuant, OmniQuant, QUIK and Atom.
- The ablation cannot attribute the gains. Table 2 has three rows on one model, with no variance and no sensitivity analysis for γ, λ_cos, p or the 2σ threshold. The largest gain (+12.3 ScienceQA, +10.3 MMMU) comes from changing the search objective, yet the objective of the intermediate configuration is never stated.
Questions for authors
see Weaknesses