acceptodds
Withdrawn from ICLR 2027

MorphoQuant: Morphology-Driven Quantization for Omni-modal Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Conventional Post-Training Quantization (PTQ) methods struggle with 4-bit Omni-modal Large Language Models (OLLMs) due to the extreme distribution heterogeneity and disparate outlier patterns across modalities. To address this, we propose MorphoQuant, a morphology-directed PTQ framework engineered to preserve cross-modal morphology and mitigate outlier loss. Specifically, we introduce Morphology-Driven Sparse Compensation (MDSC), which identifies high-dispersion activation channels and retains large activation residuals into a channel-wise sparse computational branch. Complementing this, we propose Morphology-Preserving Optimization (MPO), which optimizes the activation clipping boundary using a reconstruction objective with a cosine-similarity term; the boundary also determines which residuals enter the compensation branch. We evaluate the method on Qwen2.5-Omni-3B and 7B using vision-language, video-language, and audio-language benchmarks, and on InternVL2.5-8B using vision-language benchmarks. Under the evaluated W4A4 setting, MorphoQuant maintains non-collapsed task performance where the reproduced W4A4 multimodal baselines frequently fail. Alternative activation formats further improve results in some settings. These findings suggest that fine-grained residual selection coupled with clipping-boundary optimization is a promising approach to low-bit quantization of omni-modal models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Model, Model Compression, Model Efficiency, Post-Training Quantization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Qcf5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MorphoQuant, a post-training quantization (PTQ) framework for W4A4 omni-modal LLMs. It has two components:

  • MDSC scores each input channel by activation range over log mean magnitude and selects channels above a local or global μ+2σ threshold. Within these channels, elements with |X| > γ·α_c keep their full quantization residual in a higher-precision branch: Y = W_Q X_Q + W_Q ΔX_sparse + b.
  • MPO selects a per-channel activation clipping boundary α_c by grid search, minimizing an ℓ3 reconstruction loss plus a cosine loss. Because the residual threshold is γ·α_c, the boundary search also changes the compensated set.

Weights are NF4 (bitsandbytes). Experiments cover Qwen2.5-Omni-3B/7B (ScienceQA, MMMU, Video-MME, AIR-Bench) and InternVL2.5-8B (ScienceQA, MMMU). The paper reports non-collapsed W4A4 accuracy where the reproduced MBQ and MQuant collapse, further gains with HiF4/HiF8 activation formats, and a three-row ablation.

Coupling the clipping boundary with residual selection is sensible. However, the omni-modal motivation does not shape the method, the delta over existing outlier-retention methods is not isolated, and the experiments cannot support the claims.

Critical strengths

S1. The paper evaluates audio (AIR-Bench) and video (Video-MME) on a genuine omni model, where public W4A4 results are scarce. This could become a useful reference point. It is not decisive, because the numbers are hard to interpret without an FP16 reference and against collapsed baselines. The authors also hedge their claims carefully ("reproduced baselines", "we do not infer hardware acceleration").

Critical weaknesses

W1. The novelty delta is not isolated, and the cost of the high-precision branch is not reported (primary concern).

  • Delta not isolated. Relative to QUIK, Atom and LLM.int8() (all cited in Sec. 2.3), the delta is element-wise selection within selected channels, plus a threshold tied to α_c. There is no comparison against channel-level retention under a matched high-precision budget, so it is unclear whether this delta matters.
  • Cost not reported. MDSC keeps the full residual at selected positions, and the selection is input-dependent at inference. The main text reports neither residual density, γ, effective bits per activation, nor latency and memory. Without these, "W4A4" is not comparable to pure W4A4 baselines; a dense enough branch approaches A16. The authors list these as future work, but the current claims depend on them.
  • The INT4 tensor-core story in Fig. 1 is unsupported. Per-input-channel activation boundaries α_c cannot be factored out of an integer GEMM unless they are folded into the weights, which the paper does not discuss. In addition, bitsandbytes NF4 weights are dequantized to FP16 at matmul time.
  • The omni-modal framing does not reach the method. Neither MDSC nor MPO uses modality information, so Sec. 3.2 motivates the paper but not the method.

W2. The experiments are uninformative.

  • No full-precision reference. Without FP16 rows, the degradation cannot be measured.
  • Baselines appear broken. MBQ and MQuant score near 0 at W4A4, and MQuant also collapses at W8A8 on Video-MME (0.37) and AIR-Bench (0.00). The paper's own ablation "Baseline" (54.73/30.44) far exceeds MBQ (9.88/0.00). Both point to broken baseline adaptations, and outperforming collapsed baselines says little about the method.
  • Strong W4A4 baselines are missing on Qwen2.5-Omni: QuaRot, SpinQuant, OmniQuant, QUIK and Atom.
  • The ablation cannot attribute the gains. Table 2 has three rows on one model, with no variance and no sensitivity analysis for γ, λ_cos, p or the 2σ threshold. The largest gain (+12.3 ScienceQA, +10.3 MMMU) comes from changing the search objective, yet the objective of the intermediate configuration is never stated.
Questions for authors

see Weaknesses

Reviewer_g5EjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MorphoQuant, a post-training quantization method for omni-modal LLMs that accept text, image, video, and audio inputs. It uses 4-bit weights and activations, with a small high-precision branch to correct errors from activation outliers. The method first identifies the few channels whose activation ranges are much larger than their typical values. For these channels, the high-precision branch preserves the error from values outside the 4-bit range and adds a correction to the output (MDSC).

The method then selects a clipping range for each channel through grid search. The search uses a loss that combines reconstruction error and cosine similarity (MPO). This clipping range also determines which values enter the correction branch.

The authors test Qwen2.5-Omni-3B/7B on image benchmarks (ScienceQA and MMMU), a video benchmark (Video-MME), and an audio benchmark (AIR-Bench). They also test InternVL2.5 8B on ScienceQA and MMMU.

Critical strengths

S1 Evaluation beyond image-text, with strong performance at W4A4: The paper tests video and audio, not just images, unlike previous work. Actually works at W4A4 unlike rpevious works

S2 Simple, cheap procedure: The method needs no training, only per-channel statistics and a grid search on calibration samples.

Critical weaknesses

Major Weaknesses:

W1 Novelty and Performance comparison over Quant Experts [A] and MASQuant [B]: The paper cites neither [A] nor [B]. [A] already observes that a small subset of activation channels accounts for most of the quantization error, that these channels vary across modalities and tokens, and corrects their error with an extra high-precision path. [B] already quantizes Qwen2.5-Omni-3B/7B, the same models used here. It uses modality-specific scales plus a correction branch, and evaluates on audio, MMMU and OmniBench against an FP16 reference. MorphoQuant's side branch (MDSC) follows the same idea as [A]. The paper should explain what MorphoQuant add beyond [A] and [B], and compare against both on Qwen2.5-Omni.

W2 Insufficient ablations: The contributions of some of the core parts/claims of the paper are unclear, and are not isolated via ablation:

(a) Channel selection: how does dispersion-based selection compare with magnitude-only or even random selection at the same residual density? This would clarify the benefit of the proposed score.

(b) Compensation and clipping: Table 2 adds MDSC together with clipping-boundary search, so their contributions are confounded. What does each contribute separately?

(c) Optimization objective: Does adding the cosine loss improve accuracy over using the same p=3 reconstruction loss alone?

W3 Missing baselines on Qwen2.5-Omni: QuaRot [C] and SpinQuant [D] are common W4A4 baselines, but QuaRot is run only on InternVL2.5 and SpinQuant isn't run at all. Running QuaRot on Qwen2.5-Omni with the same NF4 weights would show whether MorphoQuant beats a rotation-based W4A4 method.

W4. Efficiency gains are not measured. The paper reports no inference latency or memory measurements, so the practical benefit of low-bit activations after accounting for the high-precision correction branch remains unclear.

  • [A] Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization. Jia et al., CVPR 2026.
  • [B] MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models. Hu et al., CVPR 2026.
  • [C] QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. Ashkboos et al., NeurIPS 2024.
  • [D] SpinQuant: LLM Quantization with Learned Rotations. Liu et al., ICLR 2025.

Minor weaknesses

MW1 FP16 reference: Table 1 has no FP16 row. The only FP16 numbers are in the supplement's Video-MME table, and it reports MorphoQuant at 62.61, while Table 1 reports 50.11. The W4A16 NF4 reference also looks low: 43.56/49.44 MMMU on 3B/7B, against 53.1/59.2 on the official Qwen2.5-Omni model card. Please add FP16 rows under your protocol.

MW2 Limited image evaluation: Only two image benchmarks (ScienceQA and MMMU) are evaluated. Previous quantization work, e.g. [B], also tests TextVQA, VizWiz, and OCRBench, covering capabilities not tested here.

Questions for authors

What is the "Baseline" row in Table 2? How were MBQ and MQuant adapted to audio and video, and with which weight quantizer? Can you add FP16 numbers under your protocol?

Reviewer_JHggOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MorphoQuant, a post-training quantization framework for omni-modal large language models designed to address heterogeneous activation distributions and outlier patterns across modalities under low-bit quantization.

The proposed framework consists of two components. First, Morphology-Driven Sparse Compensation (MDSC) identifies activation channels with unusually large dispersion and selectively preserves the quantization residuals of large-magnitude activation elements in an additional correction branch. The forward computation combines a dense quantized matrix multiplication with sparse residual compensation.

Second, Morphology-Preserving Optimization (MPO) optimizes activation clipping boundaries through a reconstruction objective combining an Lp error term and cosine similarity. The optimized clipping boundaries also determine the element-wise thresholds used for residual selection, coupling the two components.

The authors evaluate the method on Qwen2.5-Omni-3B/7B across vision-language, video-language, and audio-language benchmarks, and on InternVL2.5-8B for vision-language understanding. The experiments demonstrate substantially better task performance than several reproduced W4A4 baselines that frequently produce degenerate outputs. An additional progressive ablation examines the contribution of the combined method components.

Critical strengths

S1. The paper addresses an important and challenging low-bit quantization setting for omni-modal models.

Quantizing models that process images, audio, video, and text is challenging because the activation distributions encountered across modalities can differ substantially. The paper targets this practically relevant setting rather than evaluating exclusively on conventional text-only LLMs.

The proposed combination of coarse channel selection and fine-grained element-wise residual correction is a reasonable approach to limiting the impact of extreme activation values while preserving a low-bit dense computation path.

S2. The method preserves meaningful task performance in difficult W4A4 configurations.

The results on Qwen2.5-Omni indicate that MorphoQuant preserves substantially better performance than several reproduced low-bit multimodal quantization baselines that fail in the evaluated settings.

The evaluation includes vision-language, video-language, and audio-language benchmarks, making it broader in modality coverage than evaluations restricted to visual question answering.

These results suggest that the proposed approach may be useful for maintaining model accuracy under aggressive quantization, although its practical efficiency and advantage over robust competing implementations remain insufficiently established.

Critical weaknesses

W1. The claimed efficiency advantages are not substantiated by an analysis or measurement of the sparse compensation overhead.

A central motivation of MorphoQuant is to preserve low-bit computational efficiency by separating dense quantized computation from sparse residual correction.

However, the proposed forward computation includes an additional term involving a dynamically selected sparse activation residual. Its actual cost depends on the residual density, precision, data representation, memory access patterns, and execution kernels.

The paper does not report the fraction of retained activation elements, the corresponding correction FLOPs, the additional memory footprint, or measured inference latency and throughput. Consequently, it is unclear whether the added compensation branch remains inexpensive in practice, particularly for long video or audio inputs.

Moreover, the implementation uses NF4 weight quantization through bitsandbytes, while the motivation emphasizes efficient INT4 Tensor Core computation. These are not equivalent execution mechanisms without an explicit explanation of the kernels and data conversions involved.

The absence of these measurements is consequential because improved W4A4 accuracy alone does not establish that the method provides an advantageous accuracy–latency or accuracy–memory trade-off.

W2. The experimental comparisons and ablations do not adequately establish the distinct contributions of the proposed method.

The principal W4A4 results compare MorphoQuant against reproduced MBQ and MQuant configurations that frequently produce degenerate outputs, including some failures under W8A8. While these results demonstrate that the proposed implementation is more robust in the tested configurations, they provide limited evidence of superiority over competitive low-bit quantization methods operating correctly.

The manuscript does not sufficiently investigate why these baselines fail or whether the failures are attributable to quantization difficulty, model-specific adaptation, or implementation choices.

Furthermore, Table 2 only compares an initial baseline, a combined MDSC-plus-clipping-search configuration, and the full method with the composite objective. It does not independently evaluate MDSC, clipping-boundary optimization, the Lp reconstruction objective, and cosine similarity.

This is particularly important because sparse outlier correction and cosine-based reconstruction are established techniques. Without appropriate component-wise and alternative-method controls, the empirical evidence does not demonstrate which proposed design choices are essential or uniquely responsible for the observed improvements.

Together, these limitations substantially weaken the paper's main empirical and methodological claims.

Questions for authors

Q1. Sparse compensation cost and effective low-bit execution (related to W1).

Could the authors report the average and maximum density of the retained activation residuals across layers, models, and input modalities?

Please also clarify the numerical precision and storage format of the sparse residual branch, the overhead of constructing the dynamic element-wise masks, and the actual execution mechanism for the main quantized matrix multiplication.

Since NF4 weight quantization does not automatically imply native INT4 Tensor Core execution, it would be important to specify whether the proposed implementation uses low-bit integer kernels or dequantization-based operations.

End-to-end prefill/decode latency, memory usage, and throughput comparisons would materially affect my assessment of the practical efficiency claims.

Q2. Reliability of the W4A4 baseline comparisons (related to W2).

Could the authors explain why several reproduced MBQ and MQuant configurations produce degenerate outputs, including failures under W8A8?

In particular, were the methods applied to identical model components, calibration data, and activation quantization configurations? What adaptations were required for the omni-modal architectures?

A comparison against a stable W4A4 baseline or a simpler outlier-preserving quantization method under matched conditions would help determine whether MorphoQuant offers an advantage beyond preventing baseline implementation failures.

Q3. Independent contributions of MDSC and MPO (related to W2).

Could the authors provide a factorial ablation separating channel dispersion selection, element-wise residual compensation, clipping-boundary optimization, and the Lp/cosine reconstruction terms?

It would also be useful to compare the proposed dispersion metric against simpler channel-selection criteria, such as maximum activation magnitude, activation variance, or quantization reconstruction error.

These experiments would help establish which components are responsible for the reported gains.

Q4. Optimization of the coupled clipping objective.

The cosine-similarity objective is defined over complete token feature vectors, whereas clipping boundaries are selected through independent per-channel grid searches.

Could the authors clarify how the globally coupled objective is evaluated during the per-channel search? Are other channel boundaries held fixed, and is the procedure performed iteratively?

A more precise description would improve both reproducibility and the technical understanding of MPO.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.