acceptodds
Desk rejected at ICLR 2027

MAQuant: Modality-Aware Quantization for Omnimodal Large Language Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Omnimodal large language model (OLLM) deliver strong capabilities but incursubstantial memory and computation costs. Low-bit quantization is a practical remedy, yet existing methods mainly target Large Language Model(LLM) or Large Vision Language Model(LVLM) with modality-agnostic objectives, failing to handle heterogeneous activation distributions, outlier-dominated degradation, and cross-modal error coupling in OLLM. We propose MAQuant, a modality-aware post-training quantization framework that combines Modality-Specific Learnable Transformation (MSL), Exponent-weighted Activation Penalty (EAP), and Cross-modal Error Exchange (CME) to learn modality-specific quantization friendly spaces, protect salient activations, and share channel-wise sensitivity across modalities. EAP and CME are calibration-only objectives and add no inference time loss computation. Experiments on diverse benchmarks and low-bit settings show that MAQuant consistently outperforms strong LLM and LVLM quantization baselines. On Qwen2.5-Omni-7B, MAQuant reduces storage by 68.6% under W4A4 while retaining 97.4% of full-precision average performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
quantization, omnimodel

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kDwyOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MAQuant investigates post-training quantization on the input side of multimodal large language models. It learns independent transformations for the encoder and modality-shared transformations (MSL) at each calibration position of the decoder; EAP increases the weight of significant activation reconstruction error in the calibration loss, while CME uses channel errors from other modalities to guide the reconstruction of the current modality. Under W4A4, the average accuracy across six tasks of Qwen2.5-Omni-3B and 7B reaches 63.30% and 68.00%, respectively. The authors also report a 2.54–3.04× prefill speedup relative to FP16 on the A100.

Critical strengths
  1. The research question is clear, and the methodological design is grounded in observations. Figure 2 illustrates the impact of calibration data composition on transformation, the substantial quantization error associated with significant activations, and the overlap of high-error channels across different modalities. These observations support leveraging modal information during calibration while keeping the transformations and weights at each position in the decoder shared across modalities.
  2. The accuracy performance after low-bit quantization is of practical value. Under the W4A4 setting, Qwen2.5-Omni-3B achieves a 20.21 percentage point higher average accuracy across six tasks than FlatQuant; the 7B model reaches approximately 97.8% of the average six-task accuracy of its FP16 version (Table 1).
  3. Ablation experiments and deployment measurements provide complementary evidence. With identical calibration data and optimization budgets, incorporating either EAP or CME improves the average score on two tasks. The authors also measured the prefill latency of the native INT4 backend and accounted for auxiliary transformations in storage estimation (Figure 4a; Table 3; Appendix A.7).
Critical weaknesses
  1. The core new contributions relative to prior work remain to be defined. FlatQuant has adopted learnable transformations and reconstruction optimization, MBQ has leveraged modality-sensitive weighted calibration, and MASQuant has investigated modality-specific smoothing and shared quantization weights in vision, audio, and text models. The MSL of MAQuant inherits the transformation parameterization of FlatQuant, and the relatively clear new designs are the salient activation weighting in EAP and cross-modal channel error exchange in CME. Therefore, the core novelty mainly depends on the independent value of these designs compared with existing transformation and weighted calibration approaches; merely extending the method to full-modal scenarios is insufficient to fully demonstrate the added contributions (Sections 4.1–4.3).

  2. Ablations support the effectiveness of auxiliary objectives, yet the effects of key design choices have not been isolated. Figure 4(a) shows that under the same calibration budget, adding EAP or CME to MSL improves the average score of the two tasks by 0.94 and 1.05 percentage points respectively, which constitutes valuable evidence. Nevertheless, CME has not been compared with within-modality error or combined-modality error weighting; EAP adopts both salient token filtering and exponential weighting, lacking a control that computes vanilla MSE on the same batch of tokens. Hence, existing results cannot determine whether the gains truly rely on cross-modal exchange and exponential weights, or whether similar effects can be achieved via more general error reweighting (Sections 4.2–4.3, 5.3; Appendices B.2–B.3).

  3. The source and stability of the accuracy advantages in the main table remain to be verified. Under the W4A4 setting, the average six-task gain of MAQuant over FlatQuant is 20.21 percentage points on the 3B model and 0.86 percentage points on the 7B model. The substantial gain on the 3B model is worthy of recognition, but Appendices A.2 and A.6 do not uniformly specify the quantization granularity, dynamic or static settings, and calibration optimization budget for each baseline in the main table, nor do they report fluctuations across different calibration samples and random seeds. This makes it difficult to fully verify the comparison conditions and limits the interpretation of marginal advantages and gain differences across model scales (Table 1).

Questions for authors
  1. Comparison of core new contributions and mechanisms: Could you first clarify the technical differences between MSL and FlatQuant under the same quantization range, and then test CME and EAP with fixed MSL, data and optimization budget? Specifically, is the cross-modal error adopted by CME better than the weighted error of the same modality or combined modalities? Does the exponential weighting of EAP outperform ordinary MSE calculated on the same set of top-5% tokens? These comparisons will directly affect my judgment on the independent value of the newly proposed mechanisms.

  2. Comparison conditions and accuracy stability: Could you list the quantization granularity, dynamic or static settings, calibration data and optimization budget for each baseline in the main table, and report the mean values and fluctuations with different calibration samples and random seeds under the representative 3B/7B W4A4 configuration? Can these results explain why the gains over FlatQuant differ greatly between the two model sizes, and verify whether the 0.86 percentage point advantage on the 7B model is stable?

  3. Supplementary clarification: Deployment measurements and performance trade-offs. Appendices A.7 and C.4 have limited efficiency evaluation to prefill. Could you further specify which model modules are included in the timing measurements in Table 3 and the scales of visual and audio inputs, and report the latency fluctuations from repeated measurements? MAQuant achieves roughly a 1%–2% latency improvement compared with SpinQuant, yet its peak VRAM usage reaches 9.82 GB at batch=8, which is higher than SpinQuant’s 9.00 GB. These measurement details help assess whether the minor speedup is stable and identify applicable scenarios where lower latency comes with higher VRAM consumption; this issue mainly affects the evaluation of efficiency gains.

Reviewer_zaqDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MAQuant applies post-training quantization to the input-side encoders, connectors, and shared decoder of omnimodal language models. It combines learnable transformations with an Exponent-weighted Activation Penalty (EAP) and Cross-modal Error Exchange (CME). EAP emphasizes salient activations, while CME uses the channel error profiles of other modalities to weight reconstruction. Experiments evaluate Qwen2.5-Omni-3B and 7B under W4A8 and W4A4 on six benchmarks. The paper also provides component ablations, activation analyses, storage estimates, and prefill measurements.

Critical strengths

The reported accuracy gains are practically meaningful, and the component ablation gives some support for the added objectives. On the 3B model under W4A4, MAQuant raises the six-task average from FlatQuant's 43.09 to 63.30. Its three-task average also exceeds the adapted MQuant baseline by 8.04 percentage points. On the 7B component ablation, adding EAP and CME to MSL raises the two-task average from 54.83 to 56.82 under the same calibration budget. Together, these results provide a credible empirical reason to investigate the proposed calibration approach.

Critical weaknesses

The adaptations used in the most relevant baseline comparison are not described well enough. Section 5.1 states that MQuant and MASQuant are extended to match MAQuant's quantization scope, but does not explain how their original procedures are applied to the additional modules. In particular, it is unclear whether MQuant's modality-specific static scales and MASQuant's low-rank compensation are retained, and what quantizer granularity is used. These components are part of the original methods, so their treatment matters when interpreting Table 2. I do not take the low baseline scores as evidence of an unfair implementation, but the comparison needs this information before the large advantage can be confidently attributed to MAQuant.

The evidence does not yet separate the benefit of cross-modal exchange from ordinary error reweighting. Equations (6) to (8) add reconstruction weights derived from detached channel error profiles. Figure 4(a) shows that adding this objective helps, but does not establish whether the useful information must come from other modalities. Equation (8) also sums token errors, whereas the reconstruction objective averages them. A comparison with the target modality's own error profile, using the same normalization and token sampling, would test the specific reason for exchanging profiles. This distinction matters because the transformations are adopted from FlatQuant and the cross-modal objective is a central claimed contribution.

Questions for authors

How were MQuant and MASQuant adapted for Table 2? Please specify how the additional modules were calibrated, which original scaling or compensation components were retained, and the quantizer granularity. A clear account would allow me to judge whether the main comparison reflects the proposed calibration method rather than differences introduced by the adaptations.

Does CME outperform an otherwise identical objective that uses each target modality's own normalized channel error profile? A comparison on the existing ablation configuration, with the same loss normalization and token sampling, would answer whether cross-modal exchange itself provides the claimed benefit. This result would materially affect my assessment of the method's contribution.

Reviewer_t6x5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes MAQuant, a PTQ method for MLLMs. It quantizes the vision and audio encoders, their connectors, and the shared decoder.

The method has three parts. First, MSL, which follows FlatQuant and learns transformations that reshape activations and weights to make them more suitable for quantization. Each encoder has its own transformations, while each decoder layer learns one transformation shared across all modalities. Second, EAP uses a calibration loss that focuses on the 5% of tokens with the largest activations. Third, CME adjusts the weight of each modality’s per-channel error using the error profiles of the other modalities. Channels sensitive to quantization for audio also receive more weight for text and vision tokens.

The authors test Qwen2.5-Omni-3B and 7B at W4A8 and W4A4 on six benchmarks: GQA, ScienceQA, OmniBench, MMAU, PIQA, and HellaSwag. They compare MAQuant with SmoothQuant, FlatQuant, and MBQ. For the 3B model, they also include adapted versions of MQuant and MASQuant.

MAQuant achieves the highest average score across the six tasks among the quantization methods in all four settings, though with small margins on the 7B models. At W4A4, the 7B model scores 68.00, compared with 67.14 for FlatQuant and 69.52 for FP16.

Critical strengths

S1 Full-scope quantization with real kernels: The paper quantizes the encoders and connectors together with the decoder, and it measures speed with real INT4 kernels rather than simulating them

S2 Useful observation on cross-modal error: The analysis in Sec. 3 and Fig. 2c, showing that the channels with the largest quantization error have a big overlap same across text, vision and audio, and increasingly so in deeper layers, is interesting on its own. I am not aware of prior work measuring this for omni-modal models.

Critical weaknesses

W1 Limited novelty & motivation for method over FlatQuant, MBQ: (a) The core of the method, the learned transformation MSL is essentially FlatQuant [A] applied to every layer, and it is not modality-specific in the decoder. Each decoder layer uses one transformation shared by all tokens (A.3). The "modality-conditioned" loss in Eq. 2 is just the ordinary MSE over all tokens, since the per-modality weights cancel.

(b) MBQ [B] already combines a modality-weighted calibration loss with FlatQuant as MBQ(Rot), which is very similar to MAQuant's design (FlatQuant's transformation plus a modality calibration loss). Does Table 1 use MBQ(Rot) or is it the smoothing variant ? thats unclear.

MAQunat seems like an incremental addition over MBQ and FlatQuant, possibly with even missing empirical evidence against MBQ (rot).

W2 Missing recent baselines in evaluation Closely related W4A4 and modality-aware methods are not compared against. QuaRot [C] and SpinQuant [D] are run for latency in Table 3 but never for accuracy. DuQuant [F] is cited but not run. MBQ(Rot) [B] may not have been run (W1(b)). Quant Experts [E] is neither cited nor run. MQuant and MASQuant are compared only on 3B, on three tasks. The paper gives very few details on how the Table 1 baselines were run, including their calibration data, quantization scope, and optimization settings. What was FlatQuant calibrated on, and was it run with the same scope and epochs as MAQuant?

W3 No std deviations and small gains in accuracy on 7B: The main tables have no variance, so it is hard to judge the significance of the improvements. On 7B the gain over FlatQuant is ~1.1 (W4A8) and ~0.8 (W4A4) on the six-task average, and on 3B W4A8 MBQ also the gain is small. The ablation rows similarly have small gains. These small gains might be within the variance on these benchmarks.

W4 Design choices inside each component are not ablated: Fig. 4a shows that each component helps when switched on. It does not show that the way each component is built is needed. Many of them have a simpler version that is never compared/ motivated. e.g.

(a) CME: The weights for a modality come from the other modalities' error profiles. Fig. 2c shows these profiles overlap a lot with the modality's own profile. So It remains unclear whether these gains require cross-modal guidance or can be obtained using each modality’s own error profile. What happens if each modality uses its own profile, or one profile pooled over all tokens?

(b) EAP: It selects the top 5% tokens and then weights their errors by magnitude with α = 5. The two are always used together. The paper says α ≤ 2 turns the weighting off (l.303), but no such row exists. How much comes from selection and how much from weighting?

  • [A] FlatQuant: Flatness Matters for LLM Quantization. Sun et al., ICML 2025.
  • [B] MBQ: Modality-Balanced Quantization for Large Vision-Language Models. Li et al., CVPR 2025.
  • [C] QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. Ashkboos et al., NeurIPS 2024.
  • [D] SpinQuant: LLM Quantization with Learned Rotations. Liu et al., ICLR 2025.
  • [E] Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization. Jia et al., CVPR 2026.
  • [F] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs. Lin et al., NeurIPS 2024.
Questions for authors

Q1 Does Table 1 use MBQ(Rot) or is it the smoothing variant ? Q2 What was FlatQuant calibrated on ?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.