- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MAQuant investigates post-training quantization on the input side of multimodal large language models. It learns independent transformations for the encoder and modality-shared transformations (MSL) at each calibration position of the decoder; EAP increases the weight of significant activation reconstruction error in the calibration loss, while CME uses channel errors from other modalities to guide the reconstruction of the current modality. Under W4A4, the average accuracy across six tasks of Qwen2.5-Omni-3B and 7B reaches 63.30% and 68.00%, respectively. The authors also report a 2.54–3.04× prefill speedup relative to FP16 on the A100.
Critical strengths
- The research question is clear, and the methodological design is grounded in observations. Figure 2 illustrates the impact of calibration data composition on transformation, the substantial quantization error associated with significant activations, and the overlap of high-error channels across different modalities. These observations support leveraging modal information during calibration while keeping the transformations and weights at each position in the decoder shared across modalities.
- The accuracy performance after low-bit quantization is of practical value. Under the W4A4 setting, Qwen2.5-Omni-3B achieves a 20.21 percentage point higher average accuracy across six tasks than FlatQuant; the 7B model reaches approximately 97.8% of the average six-task accuracy of its FP16 version (Table 1).
- Ablation experiments and deployment measurements provide complementary evidence. With identical calibration data and optimization budgets, incorporating either EAP or CME improves the average score on two tasks. The authors also measured the prefill latency of the native INT4 backend and accounted for auxiliary transformations in storage estimation (Figure 4a; Table 3; Appendix A.7).
Critical weaknesses
-
The core new contributions relative to prior work remain to be defined. FlatQuant has adopted learnable transformations and reconstruction optimization, MBQ has leveraged modality-sensitive weighted calibration, and MASQuant has investigated modality-specific smoothing and shared quantization weights in vision, audio, and text models. The MSL of MAQuant inherits the transformation parameterization of FlatQuant, and the relatively clear new designs are the salient activation weighting in EAP and cross-modal channel error exchange in CME. Therefore, the core novelty mainly depends on the independent value of these designs compared with existing transformation and weighted calibration approaches; merely extending the method to full-modal scenarios is insufficient to fully demonstrate the added contributions (Sections 4.1–4.3).
-
Ablations support the effectiveness of auxiliary objectives, yet the effects of key design choices have not been isolated. Figure 4(a) shows that under the same calibration budget, adding EAP or CME to MSL improves the average score of the two tasks by 0.94 and 1.05 percentage points respectively, which constitutes valuable evidence. Nevertheless, CME has not been compared with within-modality error or combined-modality error weighting; EAP adopts both salient token filtering and exponential weighting, lacking a control that computes vanilla MSE on the same batch of tokens. Hence, existing results cannot determine whether the gains truly rely on cross-modal exchange and exponential weights, or whether similar effects can be achieved via more general error reweighting (Sections 4.2–4.3, 5.3; Appendices B.2–B.3).
-
The source and stability of the accuracy advantages in the main table remain to be verified. Under the W4A4 setting, the average six-task gain of MAQuant over FlatQuant is 20.21 percentage points on the 3B model and 0.86 percentage points on the 7B model. The substantial gain on the 3B model is worthy of recognition, but Appendices A.2 and A.6 do not uniformly specify the quantization granularity, dynamic or static settings, and calibration optimization budget for each baseline in the main table, nor do they report fluctuations across different calibration samples and random seeds. This makes it difficult to fully verify the comparison conditions and limits the interpretation of marginal advantages and gain differences across model scales (Table 1).
Questions for authors
-
Comparison of core new contributions and mechanisms: Could you first clarify the technical differences between MSL and FlatQuant under the same quantization range, and then test CME and EAP with fixed MSL, data and optimization budget? Specifically, is the cross-modal error adopted by CME better than the weighted error of the same modality or combined modalities? Does the exponential weighting of EAP outperform ordinary MSE calculated on the same set of top-5% tokens? These comparisons will directly affect my judgment on the independent value of the newly proposed mechanisms.
-
Comparison conditions and accuracy stability: Could you list the quantization granularity, dynamic or static settings, calibration data and optimization budget for each baseline in the main table, and report the mean values and fluctuations with different calibration samples and random seeds under the representative 3B/7B W4A4 configuration? Can these results explain why the gains over FlatQuant differ greatly between the two model sizes, and verify whether the 0.86 percentage point advantage on the 7B model is stable?
-
Supplementary clarification: Deployment measurements and performance trade-offs. Appendices A.7 and C.4 have limited efficiency evaluation to prefill. Could you further specify which model modules are included in the timing measurements in Table 3 and the scales of visual and audio inputs, and report the latency fluctuations from repeated measurements? MAQuant achieves roughly a 1%–2% latency improvement compared with SpinQuant, yet its peak VRAM usage reaches 9.82 GB at batch=8, which is higher than SpinQuant’s 9.00 GB. These measurement details help assess whether the minor speedup is stable and identify applicable scenarios where lower latency comes with higher VRAM consumption; this issue mainly affects the evaluation of efficiency gains.