acceptodds
Under review as a conference paper at ICLR 2027

SPARCQuant: Sparse Adaptive Repair of Critical Channels for MLLMs Quantization

Abstract

Post-training quantization (PTQ) for multimodal large language models is challenging because heterogeneous modalities induce distinct activation distributions in the shared decoder. Smoothing-based quantization has proven effective and is widely adopted for large language model quantization, yet directly extending it to these models can cause smoothing factors estimated from mixed calibration data to be dominated by high-magnitude modalities, thereby over-smoothing other modalities and increasing activation quantization error. Against this backdrop, the analysis in this paper reveals a previously overlooked structural property: **high sensitivity to modality-specific smoothing mismatch is concentrated in a small subset of channels**. Motivated by this finding, this paper proposes **SPARCQuant**, a sensitivity-guided framework for **sparse adaptive repair of critical channels** in multimodal large language model quantization. SPARCQuant selects a layer- and module-adaptive base modality to define shared smoothing and ranks channels using robust activation-range discrepancy and activation-magnitude importance. A coarse-to-fine proxy reconstruction search with online scheduling then **automatically selects the sparse channel support and its size**, adapting the repair to each layer, module, and non-base modality **without manual channel selection or per-layer budget tuning**. Selected-channel ridge compensation corrects residual errors while preserving a single shared quantized weight. Experiments on vision-language and omni-modal models demonstrate **improved robustness under low-bit quantization**. A CUDA implementation with custom kernels enables efficient deployment, achieving **2.66×–2.87× end-to-end inference speedup over BF16**.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal large language models, low-bit quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dQebOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

SPARCQuant is a post-training weight-activation quantization method for Large Multimodal Models, designed to mitigate the cross-modal smoothing mismatch caused by disparities in activation distribuitions across text, vision, and audio. Observing that a small fraction of input channels accounts for a disproportionate share of mismatch scores and recoverable output errors, this paper proposes applying modality-specific smoothing and compensation exclusively to selected channels while maintaining shared quantized main weights.

Specifically, SPARCQuant adaptively selects a base modality for the linear modules in each layer and ranks the channels of non-base modalities by jointly considering activation magnitudes and inter-modal range discrepancies. During the calibration phase, it determines the number of channels to repair via a coarse-to-fine search with a complexity penalty. Modality-specific scales deviate from the shared scales only on these selected channels. Subsequently, leveraging the quantizaed inputs of the chaosen channels, output residual compensation is fitted via ridge regression. The enthire pipeline employs layer-wise calibration and parameter fitting, eliminating the need for full-model retraining.

Critical strengths
  • Insightful Analysis of Channel-Level Misalignment: The paper provides a precise diagnosis of channel-level misalignment issues in MLLM quantization, identifying a critical gap overlooked by prior methods. Based on this insight, the proposed algorithmic design effectively targets these discrepancies, offering a well-motivated solution.

  • Comprehensive Experimental Validation: The method is thoroughly evaluated across a wide range of models, quantization settings, and benchmarks, demonstrating robust effectiveness. Notably, the inclusion of CUDA-based end-to-end inference time measurements provides strong evidence of its practical deployment viability.

Critical weaknesses
  • Concerns regarding robustness to calibration data. Unlike MBQ and MasQuant, which provide guarantees based on mathematical equivalence for linear layer operations, SPARCQuant’s output compensation via Ridge Regression relies entirely on data fitting. Moreover, both the base modality selection and the number of compensated channels are driven purely by data-based search. The authors should demonstrate the algorithm's robustness against calibration data variations through either theoretical analysis or empirical evidence.

  • Potentially significant optimization costs. The proposed method involves a multi-stage optimization pipeline consisting of (1) Statistical Base Selection, (2) Iterative Gradient Optimization with Budget Search, and (3) Closed-form Ridge Regression Fitting. However, the paper fails to quantify the total computational cost of this calibration process. This omission is critical when comparing SPARCQuant against lightweight quantization schemes such as SmoothQuant and MBQ.

Questions for authors

Clarification of Algorithm Pipeline: The current description of the algorithm flow is difficult to follow, as SPARCQuant involves multiple intertwined stages of search, regression, and training, with many implementation details omitted from the main text (e.g., the specific optimization process for smsm and δm, and the ridge regression fitting for Bm and cm). I strongly recommend that the authors provide a comprehensive pseudocode or an explicit algorithm pipeline in the revision to serve as a clear guide for readers.

6 comments
AuthorsOpenReview ↗

Response to Reviewer dQeb (Part1)

Response to Reviewer dQeb

We thank the reviewer for the questions about calibration robustness, calibration cost, and the algorithm description. We address each point with the relevant measurements and implementation details. We hope these clarifications address your concerns and sincerely ask you to consider raising our score.

AuthorsOpenReview ↗

Response to Reviewer dQeb (Part2)

Q1. Unlike MBQ and MASQuant, which provide guarantees based on mathematical equivalence for linear-layer operations, SPARCQuant's ridge compensation relies on data fitting, while base-modality selection and the number of compensated channels are determined by data-driven search. Can theoretical analysis or empirical evidence demonstrate its robustness to variations in calibration data?

Response:

The concern is whether changing calibration data changes the fitted correction, selected base, or number of compensated channels. We test this at three levels. Pre-quantization equivalence does not show that a fitted correction works on new inputs, so held-out reconstruction is the direct check.

  1. Held-out reconstruction: Fit on one set and evaluate on disjoint inputs at fixed K=32K=32. If the correction memorizes fitting inputs, held-out NMSE should increase; base agreement tests whether the selected base is stable.
  2. Independent calibration sets: Rerun the full pipeline five times with different 128-example sets. Base selection, budget search, scale optimization, and ridge fitting all vary, so the score SD measures end-to-end sensitivity.
  3. Calibration size: Use nested sets of 32, 64, and 128 examples. A large score change would indicate dependence on calibration size; small changes would indicate stable behavior.

For downstream evaluation, Qwen3-VL uses COCO calibration examples and complete benchmark splits; MiniCPM-o uses OmniBench calibration examples and complete splits for LibriSpeech, FLEURS, TED-LIUM, MMMU, and OmniBench. No evaluation labels are used during calibration.

Held-out reconstruction (W4A6). The appendix scales NMSE by 10310^3; values below are unscaled. Base agreement compares the 32- and 64-sequence choices.

ModelBase agreementNMSE (32)NMSE (64)
Qwen3-VL-2B97.02%0.026340.02643
MiniCPM-o-2.695.24%0.040150.04090

NMSE changes by only 0.00009 and 0.00075, while base agreement remains 95--97%. Thus, the fitted correction generalizes to held-out inputs and the base choice is stable.

Five independent 128-example sets (W4A6). Entries are mean ±\pm sample SD over five runs. Qwen task scores and MiniCPM vision scores are higher-is-better; MiniCPM speech results are WER, where lower is better.

Qwen3-VL-2BMMMUTextVQAVizWizScienceQAOCRBenchAvg.
MBQ30.87 ± 0.3162.33 ± 0.5556.79 ± 0.3344.73 ± 0.8058.24 ± 0.3050.59 ± 0.25
MASQuant29.42 ± 0.6353.60 ± 0.5435.12 ± 0.9410.45 ± 1.0852.96 ± 1.0236.31 ± 0.38
SPARCQuant33.44 ± 0.3864.42 ± 0.3956.74 ± 0.4061.21 ± 0.7057.94 ± 0.4054.75 ± 0.22
MiniCPM-o-2.6LibriSpeech WERFLEURS WERTED-LIUM WERMMMUOmniBench
MBQ57.59 ± 1.2995.14 ± 1.6671.44 ± 1.9133.87 ± 0.8935.79 ± 0.52
MASQuant98.78 ± 2.64178.07 ± 5.88167.44 ± 4.5435.78 ± 0.1436.28 ± 0.42
SPARCQuant22.36 ± 0.7229.52 ± 0.2664.38 ± 0.9841.16 ± 0.4842.16 ± 0.38

Across sets, SPARCQuant's Qwen average is 54.75 vs. 50.59 for MBQ (SD 0.22). Its mean ASR WER is 38.75, vs. 74.72 for MBQ and 148.10 for MASQuant; its vision average is 41.66, vs. 34.83 and 36.03. These small SDs show that the full pipeline gives the same conclusion across calibration sets.

Calibration-size sweep for SPARCQuant (W4A6). Entries are mean ±\pm SD over five runs. Qwen columns are MMMU/TextVQA/VizWiz/ScienceQA/OCRBench/Avg.; MiniCPM columns are LibriSpeech/FLEURS/TED-LIUM WER/MMMU/OmniBench.

Qwen3-VL-2B (NN)MMMUTextVQAVizWizScienceQAOCRBenchAvg.
3233.27 ± 0.5463.82 ± 0.4756.21 ± 0.6560.06 ± 1.2657.68 ± 0.5654.21 ± 0.46
6433.22 ± 0.3964.20 ± 0.4456.56 ± 0.4660.96 ± 0.7957.84 ± 0.3954.56 ± 0.23
12833.44 ± 0.3864.42 ± 0.3956.74 ± 0.4061.21 ± 0.7057.94 ± 0.4054.75 ± 0.22
MiniCPM-o-2.6 (NN)LibriSpeech WERFLEURS WERTED-LIUM WERMMMUOmniBench
3223.95 ± 0.7732.03 ± 0.6566.85 ± 0.7340.60 ± 0.4742.09 ± 0.54
6422.96 ± 0.8430.21 ± 0.8365.20 ± 1.3340.96 ± 0.6542.14 ± 0.48
12822.36 ± 0.7229.52 ± 0.2664.38 ± 0.9841.16 ± 0.4842.16 ± 0.38

From 32 to 128 examples, the Qwen average changes only from 54.21 to 54.75; MiniCPM mean ASR WER changes from 40.94 to 38.75 and vision average from 41.35 to 41.66. Across independent 128-example sets, SDs are at most 0.70 points for Qwen task scores and 0.98 WER points for MiniCPM ASR. Together, these tests show that the correction generalizes, base and budget choices are stable, and the conclusion is not tied to one calibration set or size. These diagnostics are included in the revised appendix.

AuthorsOpenReview ↗

Response to Reviewer dQeb (Part3)

Q2. SPARCQuant involves a multi-stage optimization pipeline: statistical base selection, iterative gradient optimization with budget search, and closed-form ridge regression fitting. What is the total computational cost of this calibration process, and how does it compare with lightweight quantization methods such as SmoothQuant and MBQ?

Response:

We measured all four methods at W4A6 with 128 calibration examples on one RTX 5090 using four CPU threads. Timings are synchronized calibration-stage measurements and exclude model loading, preprocessing, deployment packing, and evaluation.

ModelSmoothQuantMBQMASQuantSPARCQuant
Qwen3-VL-2B13.72 s144.20 s208.30 s315.84 s
MiniCPM-o-2.626.18 s1995.44 s299.62 s523.46 s

SPARCQuant is 23.0×/20.0× slower than SmoothQuant and 1.52×/1.75× slower than MASQuant on Qwen3-VL-2B/MiniCPM-o-2.6. Compared with MBQ, it is 2.19× slower on Qwen3-VL-2B and 3.81× faster on MiniCPM-o-2.6. MBQ's MiniCPM time is high because it evaluates many candidate scales for every layer across tri-modal inputs.

For a given model checkpoint, quantization and calibration are one-time offline preparation steps. After calibration, the selected bases, budgets, scales, and compensation weights are fixed; inference does not rerun search, gradient optimization, or ridge fitting, so this calibration cost adds no runtime calibration overhead. The one-time cost is also amortized over all later inference requests. In post-training quantization, paying additional offline calibration time for a persistent accuracy gain is a standard and worthwhile trade-off: deployment quality improves without retraining the model or adding calibration computation during inference. SPARCQuant makes this trade-off to obtain the accuracy gains from sparse modality correction.

We have included the calibration-cost table and measurement protocol in the revised appendix.

Q3. The algorithm flow is difficult to follow because search, regression, and training involve multiple intertwined stages, and the main text omits implementation details, including the optimization of sms_m and δm\delta_m and the ridge regression fitting of BmB_m and cmc_m. Can the revision provide comprehensive pseudocode or an explicit algorithm pipeline to guide readers?

Response:

We have added an explicit pipeline and pseudocode to the revision. The new description separates statistical base selection, budget search, joint optimization of sms_m and δm\delta_m, and closed-form ridge fitting of BmB_m and cmc_m, and states the corresponding inputs, outputs, and update order.

Thank you for the constructive review. The added experiments quantify calibration robustness, and the revised cost and algorithm descriptions make the calibration procedure easier to reproduce. If these clarifications address your concerns, we would be grateful if you would consider raising our score.

Reviewer_pAhKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SPARCQuant, a post-training quantization framework for multimodal large language models (MLLMs). The method addresses cross-modal activation heterogeneity by selectively repairing channels that exhibit substantial modality-dependent smoothing mismatch. SPARCQuant integrates adaptive base-modality selection, sensitivity-guided channel ranking, automatic sparse-budget allocation, and ridge-regression compensation while maintaining a shared quantized weight matrix. The authors evaluate the framework on four vision-language and omni-modal models under W8A8, W4A8, and W4A6 quantization. Experiments cover visual question answering, OCR, multimodal understanding, and speech recognition. The paper also includes extensive diagnostics and a custom CUDA implementation.

Critical strengths

Strength 1: A well-motivated and comprehensive quantization framework. The paper addresses an important practical challenge in MLLM quantization: visual, textual, and audio tokens exhibit heterogeneous activation distributions, making shared smoothing factors potentially suboptimal across modalities. SPARCQuant develops a targeted sparse-repair strategy rather than applying dense modality-specific corrections. The overall framework is comprehensive, integrating base-modality selection, sensitivity-guided channel ranking, automatic budget search, ridge-regression compensation, and CUDA deployment. The appendix further strengthens the experimental investigation through held-out reconstruction tests, quantization-error localization, budget-search efficiency measurements, and calibration-size sensitivity analyses. These components make the work practically relevant and demonstrate substantial implementation effort.

Strength 2: Strong low-bit quantization results supported by broad experimental coverage. The paper evaluates four MLLMs across multiple quantization configurations and a diverse collection of visual and audio benchmarks. The improvements are particularly substantial under W4A6 quantization, where SPARCQuant consistently outperforms the evaluated baselines on most tasks. The main results, component ablations, and held-out reconstruction diagnostics provide complementary evidence of the effectiveness of sparse repair. Overall, the breadth of the experiments and the improvements under challenging quantization configurations are important strengths, although the interpretation of these results depends on resolving the evaluation baseline concerns discussed below.

Critical weaknesses

Weakness 1: The proposed sensitivity score is not sufficiently validated as an indicator of actual quantization-error importance. The paper demonstrates that the proposed sensitivity scores follow a long-tailed distribution and that a relatively small fraction of channels accounts for a disproportionate share of the measured cross-modal mismatch. However, this observation does not necessarily establish that these channels contribute most strongly to actual quantization errors or that the proposed score is the most effective criterion for identifying channels requiring repair. In Appendix A.9.1, the oracle error-localization results show that activation-only selection outperforms the proposed sensitivity score in terms of the fraction of output error removed. For example, under W4A6 with K=32, activation-only selection removes 16.30% of the output error on Qwen3-VL-2B, compared with 13.90% for the proposed score. The corresponding values on MiniCPM-o-2.6 are 13.77% and 11.40%. Although the held-out reconstruction experiments indicate that sensitivity-guided selection can perform better after compensation in certain configurations, the results remain model-dependent. Therefore, the central claim that sensitivity-guided mismatch identifies the most important repair channels requires stronger validation, preferably through matched-budget downstream evaluations against simpler channel-selection strategies.

Weakness 2: Insufficient evaluation details and substantial discrepancies between the reported BF16 baselines and official Qwen3-VL results. Table 1 reports BF16 MMMU accuracy of 37.33% for Qwen3-VL-2B and 51.78% for Qwen3-VL-8B. However, the official Qwen3-VL technical report reports MMMU accuracy of 53.4% and 69.6% for the corresponding Instruct models, respectively. These differences of 16.07 and 17.82 percentage points are substantial and require explanation. Although the paper specifies the models, calibration datasets, and benchmark names, the evaluation description does not provide sufficient information to establish direct comparability with the official results. If the authors employ a local evaluation protocol that differs from the official implementation, they should justify this choice and demonstrate that the protocol provides a reliable assessment of the original models. Ideally, the authors should first reproduce the expected BF16 baseline performance using the official evaluation procedure and then evaluate the quantized models under the same conditions. Without such clarification, it is difficult to determine whether the reported low-bit improvements accurately reflect the performance of the original models under standard evaluation conditions. This uncertainty reduces confidence in the experimental conclusions.

Questions for authors

Q1. What evidence demonstrates that the proposed sensitivity score is more effective than simpler activation-based channel-selection criteria? Appendix A.9.1 shows that activation-only selection removes more output error than the combined sensitivity score in the reported oracle analysis. Could the authors provide downstream task comparisons using activation-only, range-gap-only, and sensitivity-guided channel selection under identical K values and compensation parameter budgets? This would help clarify the independent contribution of the proposed sensitivity criterion.

Q2. Could the authors clarify the exact experimental protocol used to obtain the BF16 results in Table 1? In particular: 1.Were the full benchmark evaluation splits used, or were subsets selected? What were the dataset versions and evaluated sample counts? 2.Which exact Qwen3-VL checkpoints and model revisions were used? 3.If a different evaluation protocol was adopted, can the authors establish its validity by reproducing the official baseline or providing a controlled comparison? A response supported by reproducible baseline measurements would substantially affect my assessment.

4 comments
AuthorsOpenReview ↗

Response to Reviewer pAhK (Part1)

Response to Reviewer pAhK

We thank the reviewer for the questions about the sensitivity score and the BF16 evaluation protocol. We clarify the selector's role with matched-budget task results below. We hope these clarifications address your concerns and sincerely ask you to consider raising our score.

Q1. What evidence demonstrates that the proposed sensitivity score is more effective than simpler activation-based channel-selection criteria? Appendix A.9.1 shows that activation-only selection removes more output error than the combined sensitivity score in the reported oracle analysis. Could the authors provide downstream task comparisons using activation-only, range-gap-only, and sensitivity-guided channel selection under identical KK values and compensation parameter budgets? This would help clarify the independent contribution of the proposed sensitivity criterion.

Response:

The oracle test and SPARCQuant's repair process answer different questions. In the oracle test, selected quantized contributions are replaced directly with their floating-point values. This asks where the current output error is located, so activation magnitude is a strong criterion: at W4A6 and K=32K=32, activation-only selection removes 16.30% and 13.77% of output error on Qwen3-VL-2B and MiniCPM-o-2.6, compared with 13.90% and 11.40% for the sensitivity score. SPARCQuant instead uses the selected channels to change modality-specific smoothing and fit compensation while keeping one shared quantized weight. We therefore evaluate the selectors in the actual repair process.

For modality mm and base modality bb,

C_{m,i}=A_{m,i}G_{m,i},$$ where $A_{m,i}$ is normalized activation magnitude and $R_{m,i}$ is the robust activation range. Activation-only ranks by $A_{m,i}$; range-gap-only ranks by $G_{m,i}$; sensitivity-guided ranks by $C_{m,i}$. The combined score targets channels that are both active and poorly matched by the shared smoothing pattern. Thus, the sensitivity score is not intended to win the oracle replacement test. It is intended to select channels whose shared smoothing should be changed during repair. The existing held-out repair test uses this same smoothing-and-compensation process. At W4A6 and $K=32$, sensitivity-guided repair lowers held-out MAE relative to activation-only by 2.02% on Qwen3-VL-2B and 0.39% on MiniCPM-o-2.6, and relative to range-gap-only by 4.95% and 3.33%, respectively. This checks the selector after repair, rather than after the oracle replacement. To isolate the selector's contribution in downstream tasks, we fixed $K\in\{8,16,32\}$ and kept every other factor the same. All selectors used the same calibration samples, base modality, shared quantized weight, smoothing optimization, ridge regularization, compensation code, evaluation splits, and calibration seeds. Only the channel ranking changed, so each branch had the same $K d_{\mathrm{out}}+d_{\mathrm{out}}$ compensation parameters. No selector re-searched $K$. We ran this matched-budget comparison at W4A6 and checked the same protocol at W4A8. | Model | $K$ | Activation-only | Range-gap-only | Sensitivity-guided | | --- | ---: | ---: | ---: | ---: | | Qwen3-VL-2B, five-task Avg. $\uparrow$ | 8 | $51.20\pm0.18$ | $49.95\pm0.21$ | $53.60\pm0.17$ | | | 16 | $52.45\pm0.20$ | $50.10\pm0.24$ | $55.38\pm0.19$ | | | 32 | $52.20\pm0.22$ | $50.55\pm0.26$ | $55.26\pm0.20$ | | MiniCPM-o-2.6, mean ASR WER $\downarrow$ | 8 | $60.50\pm0.62$ | $64.00\pm0.75$ | $57.38\pm0.58$ | | | 16 | $67.20\pm0.57$ | $70.20\pm0.69$ | $61.68\pm0.54$ | | | 32 | $71.63\pm0.50$ | $74.33\pm0.64$ | $66.67\pm0.47$ | | MiniCPM-o-2.6, vision average $\uparrow$ | 8 | $40.30\pm0.24$ | $39.65\pm0.28$ | $41.75\pm0.22$ | | | 16 | $42.10\pm0.25$ | $41.25\pm0.30$ | $43.94\pm0.23$ | | | 32 | $41.45\pm0.27$ | $40.60\pm0.31$ | $43.00\pm0.24$ | At $K=32$, sensitivity-guided selection improves the Qwen3-VL average by 3.06 points over activation-only and 4.71 points over range-gap-only. On MiniCPM-o, it lowers mean ASR WER by 4.97 and 7.66 points, and raises the vision average by 1.55 and 2.40 points. The same trend holds at $K=8$ and $K=16$. We have included this matched-budget selector comparison in the revised appendix.
AuthorsOpenReview ↗

Response to Reviewer pAhK (Part2)

Q2. What exact protocol produced the Table 1 BF16 results? (1) Were full splits or subsets used, with what dataset revisions and counts? (2) Which checkpoints and revisions were used? (3) If the protocol differs, can you reproduce the official baseline or provide a controlled comparison?

Response: We rechecked the complete protocol below.

Full data coverage. We used the complete evaluation split for every benchmark with 0-shot evaluation: MMMU validation (900), TextVQA validation (5,000), VizWiz validation (4,319), ScienceQA-FULL test (4,241), and OCRBench test (1,000). Dataset revisions, in the same order, were lmms-lab/MMMU@364f2e2eb107, lmms-lab/textvqa@9c0699cd19768, lmms-lab/VizWiz-VQA@d428a2dae984, lmms-lab/ScienceQA@69dd4d6b6737, and echo840/OCRBench@92a54bd13843.

Models and local decoding. We used Qwen/Qwen3-VL-2B-Instruct@eb46e64af9b2 and Qwen/Qwen3-VL-8B-Instruct@1610d9f98460, both in BF16, with lmms-eval 0.5.0 and FlashAttention-2. Generation used at most 128 new tokens and greedy decoding (temperature=0, do_sample=false, top_p=None, num_beams=1, repetition penalty 1.0), together with the tokenizer EOS token and each task's stopping rule. All methods used the same system prompt, image processor, and answer parser.

Difference from the official setup. The Qwen3-VL technical report (Table 4) gives MMMU accuracies of 53.4 and 69.6 for 2B and 8B, respectively. It lists the small-Instruct settings as temperature=1.0, top_p=1.0, top_k=40, presence_penalty=2.0, and maximum output length 32,768. The current official MMMU script instead uses vLLM, temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5, and a 32,768-token limit, with its own prompt and answer-extraction procedure. Our evaluator uses greedy decoding and a 128-token cap. Thus, the local BF16 numbers are not direct reproductions of the official values. The main likely causes of the lower MMMU scores are sampling versus greedy decoding and the shorter generation limit; prompt construction, image preprocessing, data source, answer extraction, checkpoint/dataset revisions, stopping rules, random seeds, backend, and numerical details may also contribute. All BF16, RTN, SmoothQuant, MBQ, MASQuant, and SPARCQuant rows in the original tables used the same local protocol, and all reported benchmarks used their complete splits.

Controlled comparison with the official configuration. The public official workflow currently provides complete runnable evaluation only for MMMU-val and OCRBench. We therefore ran both complete splits (900 MMMU and 1,000 OCRBench samples) for each model. The local official-configuration runs use the published small-Instruct text preset for 2B, the published VL preset for 8B, vLLM, BF16, and a 32,768-token cap. The table reports MMMU-val/OCRBench; quantized rows are W4A6.

ModelConfiguration / methodMMMU-valOCRBench
Qwen3-VL-2BBF16, ours (original)37.3374.30
BF16, local official config45.4484.10
BF16, official report53.4085.80
MBQ, official config43.9081.60
MASQuant, official config43.2080.70
SPARCQuant, official config44.9083.40
Qwen3-VL-8BBF16, ours (original)51.7875.50
BF16, local official config53.4489.70
BF16, official report69.6089.60
MBQ, official config51.5087.90
MASQuant, official config50.6086.70
SPARCQuant, official config52.9089.10

Following the published presets still does not exactly reproduce the report: MMMU is 45.44 vs. 53.40 (2B) and 53.44 vs. 69.60 (8B); OCRBench is 84.10 vs. 85.80 and 89.70 vs. 89.60. Hence, public decoding parameters alone are insufficient; exact prompts, image processing, answer extraction, revisions, stopping/generation behavior, seeds, backend, and numerical details are also needed. We do not attribute the gap to one factor.

Independent public checks. lmms-eval PR #929 [1] reports MMMU 44.44/53.89 for Qwen3-VL-2B/8B. OmniThoughtVis [2] reports dense-model MMMU-val results of 47.9/60.0, and Attend, Transform, or Silence [3] reports 51.33 MMMU and 82.80 OCRBench for Qwen3-VL-8B. These independent results are also below the official MMMU values, supporting protocol sensitivity rather than an issue unique to our implementation.

References

  1. EvolvingLMMs-Lab, lmms-eval, Pull Request #929, Qwen3-VL evaluation results. https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/929
  2. Yuanhao Yue, Chengyu Wang, Yuanjie Lyu, Lei Shen, and Jun Huang, “OMNITHOUGHTVIS: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models,” arXiv:2605.11629, 2026. https://arxiv.org/abs/2605.11629
  3. Zhaoyang Luo et al., “Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference,” arXiv:2606.31903, 2026. https://arxiv.org/abs/2606.31903
AuthorsOpenReview ↗

Response to Reviewer pAhK (Part3)

Thank you for the constructive review. The matched-budget results isolate the sensitivity score's contribution, and Q2 provides the complete BF16 evaluation protocol. If these clarifications address your concerns, we would be grateful if you would consider raising our score.

AuthorsOpenReview ↗

Follow-up to Reviewer pAhK

Dear Reviewer pAhK,

Thank you again for your positive and constructive review. As the discussion period nears its close, we wanted to politely follow up with a brief summary of how our response and revised manuscript address your two main questions:

  • Matched-budget validation of the sensitivity score (W1, Q1): We clarified why the oracle replacement probe (Appendix A.12.1) measures raw error location rather than post-repair performance, whereas SPARCQuant selects channels for actual smoothing adjustment and compensation. In new matched-budget W4A6 downstream evaluations at fixed K∈{8,16,32}K \in \{8, 16, 32\} with identical compensation parameter counts (Appendix A.10, Table 13), sensitivity-guided selection consistently outperforms both activation-only and range-gap-only selection across all KK (at K=32K=32: +3.06/+4.71 Qwen3-VL-2B avg., −4.97/−7.66 MiniCPM-o mean ASR WER, and +1.55/+2.40 vision avg.).
  • BF16 evaluation protocol and official-config comparison (W2, Q2): We documented the exact model checkpoints, dataset revisions, full 0-shot evaluation splits, and uniform lmms-eval greedy/128-token decoding setup shared by all methods in Table 1. We also ran controlled W4A6/BF16 comparisons under the published official vLLM/32k-token presets on full MMMU-val (900) and OCRBench (1,000): SPARCQuant remains consistently ahead of MBQ and MASQuant (44.90/83.40 vs. 43.90/81.60 for MBQ on 2B; 52.90/89.10 vs. 51.50/87.90 on 8B) and stays close to local official BF16 (45.44/84.10 and 53.44/89.70), alongside independent public reports noting similar MMMU protocol sensitivity.

We would greatly appreciate it if you could review our detailed response and revised manuscript before the discussion window closes. Please let us know if any questions remain—we look forward to hearing from you and hope these additions strengthen your confidence in the paper.

Best regards,
The Authors

Reviewer_6dXxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes a post-training quantization method for multimodal LLMs. It uses per-channel scaling to move activation outliers into the weights before quantization. Text, vision, and audio tokens have different activation ranges in the shared decoder. As a result, scales that work well for one modality can over-smooth the others. The authors measure these differences against a reference (“base”) modality and weight each channel’s range difference by its activation size. They find that a small number of channels account for most of the mismatch. The method uses the base modality’s smoothing and one quantized weight matrix for all tokens. For the other modalities, it adds separate scales and a small correction matrix only on the top-K channels. The correction matrix is fitted with ridge regression. The method selects the base modality and K automatically for each layer and module.

The authors test Qwen3-VL-2B/8B and two omni-modal models, MiniCPM-o-2.6 and InteractiveOmni-4B, on VQA, speech recognition, MMMU, and OmniBench. They compare the method with RTN, SmoothQuant, MBQ, and MASQuant at W8A8, W4A8, and W4A6. On Qwen3-VL, gains at W8A8 and W4A8 are small. Gains are much larger at W4A6 and on speech recognition tasks with the omni-modal models. The authors also report a 2.66–2.87× speedup over BF16 with their CUDA implementation.

Critical strengths

S1 Robust at W4A6, including audio: SPARCQuant stays usable at W4A6 on the omni-modal models,signifcantly better than baselines.

S2 Testing speedups with real kernels: Latency and memory tests come from real INT4/INT8 kernels.

Critical weaknesses

W1 Novelty over Quant Experts [A], MASQuant: The claim of a "previously overlooked phenomenon" (l.097) is already made by [A], which is not cited. [A] already observes that a small subset of channels is important for quantization error, and that these channels vary across modalities and tokens. MASQuant already motivates modality-specific smoothing scales and compensation. It is unclear what SPARCQuant adds beyond these two works: channel-guided correction is already used in [A], and modality-specific smoothing with compensation is already used in MASQuant. The paper should explain how the changes in sparse formulation, channel score, adaptive base selection, and budget search improve on these designs, and isolate their contributions experimentally. A comparison with a rotation-based baselines such as QuaRot or SpinQuant on the same models and bit-widths would also strengthen the evaluation.

W2 Insufficient ablations: The contributions of some of the core parts/claims of the paper are unclear, and are not isolated via ablation:

(a) Channel selection: random support and full SPARCQuant use different K values in Table 5, so the effect of channel selection is confounded with the repair budget. Matched-K comparisons are provided for local reconstruction error, but how much does selection matter for downstream performance at the same K?

(b) Compensation structure: how does the selected-channel correction compare with a low-rank correction (MASQuant-like) at the same extra parameter count? This would clarify the benefit of restricting compensation to selected input channels.

(c) Sparse smoothing: what happens when compensation is retained on the same selected channels, but all modalities use only the shared base scale (δ = 0)? Removing compensation does not isolate the contribution of the modality-specific scale residuals beyond the correction branch.

W3 Accuracy gains are small: Are they above variation due to eval noise The main tables have no variance, so it is hard to judge the significance of the accuracy improvements, which in some cases may be smaller than the variance (W8A8/W4A8).

[A] Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization. Jia et al., CVPR 2026

Minor Weaknesses:

MW1 The claim of no per-layer budget tuning still relies on a depth-dependent schedule whose candidate sets, penalties, and late-tail thresholds are not specified. In Fig. 5, late-layer budgets appear uniform across modules; please clarify how much this allocation is determined by the schedule versus reconstruction-based search.

MW2 Writing clarity: Some mathematical details are unclear/missing. For example, Λ in Eq. (10) is unexplained, and the exact reconstruction loss used for budget search in Eq. (4) is not specified.

Questions for authors

What distinguishes the claimed observation from Quant Experts [A], and what does SPARCQuant add beyond channel-guided correction in [A] and modality-specific smoothing with compensation in MASQuant? Can you provide controlled comparisons that isolate these contributions?

7 comments
AuthorsOpenReview ↗

Response to Reviewer 6dXx (Part 1)

Response to Reviewer 6dXx

We thank the reviewer for the constructive questions about novelty, controlled ablations, statistical variation, budget selection, and mathematical definitions. We address each point below and state the added evidence and clarifications directly. We hope these clarifications address your concerns and sincerely ask you to consider raising our score. All the answers are as follows:

AuthorsOpenReview ↗

Response to Reviewer 6dXx (Part 2)

Q1. Quant Expert, which is not cited, already observes that a small subset of channels is important for quantization error and that these channels vary across modalities and tokens. How does the claimed "previously overlooked phenomenon" differ from this observation, and what does SPARCQuant add beyond its channel-guided correction?

Response:

We apologize for omitting Quant Expert. This was an oversight in our literature review. We have added its citation and a direct comparison in the revised Introduction and Related Work. Our finding differs fundamentally from that of Quant Expert. Its channel selection and expert correction do not specifically target cross-modal smoothing mismatch. We identify a small set of channels specifically in multimodal smoothing-based quantization. When modalities share a base smoothing pattern, their activation-weighted mismatch is concentrated in these channels. The details are as follows.

The two methods follow different technical approaches. Quant Expert performs token-dependent error reconstruction: it identifies important channels for each token, uses their selection patterns to build shared and routed low-rank experts, and applies token-specific corrections. Our method follows MASQuant's modality-specific smoothing route: it adjusts activation scales for each modality, keeps one shared quantized weight, and fits compensation for the residual error. Our further contribution is to discover that, in this multimodal smoothing-based quantization setting, the smoothing mismatch is concentrated in a small set of important channels, and then use these channels to guide sparse scale correction and selected-channel compensation. The specific differences are analyzed below.

Quant Expert defines channel importance for each token. Its score is ut,i=∣xt,i∣wˉiu_{t,i}=|x_{t,i}|\bar w_i, where xt,ix_{t,i} is the activation of input channel ii for token tt, and wˉi\bar w_i is the mean absolute weight connected to that channel. It selects the kk largest scores for each token, then counts how often each channel is selected: fi=ni/Tf_i=n_i/T, where nin_i is its selection count over TT calibration tokens. The per-token set is small because kk is small. Figure 3 further shows that only a few channels are selected frequently; others matter for particular tokens. Selection frequency and which channels appear together guide the construction of shared and routed low-rank experts. Each token uses the shared expert and one routed expert. These correction branches read the full input. In plain words, Quant Expert uses important-channel patterns to build experts that correct errors for different tokens.

Our method is mainly designed for smoothing-based quantization of multimodal models. Smoothing rescales activations before quantization rounds them to a limited set of values. A shared scale may shrink one modality too much, making small values round to zero. Too little shrinking can leave large values that force wider gaps between the available levels, losing detail in other channels. We use Dm→b,i=Am,i∣log⁡(Rm,i/Rb,i)∣D_{m\rightarrow b,i}=A_{m,i}|\log(R_{m,i}/R_{b,i})|, where Rm,iR_{m,i} and Rb,iR_{b,i} are channel ii's typical activation ranges in modality mm and base modality bb. Smoothing scales depend on these ranges. Equal ranges give zero mismatch; larger relative differences suggest that one shared scale may not suit both. Am,iA_{m,i} is the average absolute activation, normalized across channels, so larger activations receive higher priority. Figure 1 shows that a few channels have much higher scores than the rest. We select the highest-scoring channels as Im\mathcal I_m, adjust their smoothing scales, and fit Δym=x~m,ImBm+cm\Delta y_m=\widetilde x_{m,\mathcal I_m}B_m+c_m by ridge regression. Here, x~\widetilde x is the smoothed, quantized input. Only selected input channels feed this correction, which can adjust all output channels. This avoids the learned full-input projection of a dense low-rank correction. The selected set is fixed after calibration, so no token-level routing is needed. In plain words, we find where shared smoothing fits poorly, then adjust the scales and correct the output using only those channels.

Both works recognize that channels are not equally important, and we do not claim this general point as new. A channel can be selected frequently by Quant Expert because its activations and weights are large, yet receive a zero mismatch score from us if Rm,i=Rb,iR_{m,i}=R_{b,i}. MASQuant already discussed cross-modal smoothing mismatch in smoothing-based quantization. Our further contribution is to show that the activation-weighted mismatch is concentrated in a few channels, and to use this finding for sparse modality-specific smoothing and selected-input-channel correction. Quant Expert's frequency analysis does not establish this smoothing-specific concentration.

AuthorsOpenReview ↗

Response to Reviewer 6dXx (Part 3)

Q2. How does SPARCQuant compare with Quant Expert on the main quantization benchmarks?

Response:

Quant Expert's code is not yet available in its official repository, which says that the code is “Coming soon.” We therefore compare its published W4A6, W4A8, and BF16 results from Tables 1 and 2 of the paper with SPARCQuant evaluated on the corresponding models and datasets.

Qwen2VL-2B

MethodPrecisionMMMUOCRBenchScienceQATextVQAVizWizAI2DChartQADocVQAInfoVQAMMStarMuirBenchAvg.
BF16 (reported)BF1639.8974.9076.9677.7265.7370.0172.0487.2858.4943.4626.1962.97
Quant Expert (reported)W4A633.7868.2071.8473.1859.6265.4564.6082.7551.8442.0432.8858.74
SPARCQuant (ours)W4A637.1073.1075.9777.1564.5069.4470.8286.6957.3243.3126.0561.95
Quant Expert (reported)W4A837.3372.1074.6775.3461.5968.3669.2884.4653.9741.2934.1261.14
SPARCQuant (ours)W4A839.6874.8076.7177.6165.3269.9272.0287.2758.2043.1825.9362.79

InternVL2-8B

MethodPrecisionMMMUOCRBenchScienceQATextVQAVizWizAI2DChartQADocVQAInfoVQAMMStarMuirBenchAvg.
BF16 (reported)BF1648.0076.9097.1276.9160.6182.0982.6089.9766.9259.3636.1270.60
Quant Expert (reported)W4A644.8974.3096.2374.7759.0079.4080.4887.7763.1255.2734.1568.13
SPARCQuant (ours)W4A644.5676.9096.9976.8060.6182.0282.6089.9766.7859.0836.0170.21
Quant Expert (reported)W4A845.5675.6096.7376.0859.7781.0681.6088.3164.8356.0934.3169.09
SPARCQuant (ours)W4A845.5176.9097.0676.8660.6082.0882.6089.8766.9259.3536.1270.35

The SPARCQuant averages are 61.95, 62.79, 70.21, and 70.35, compared with 58.74, 61.14, 68.13, and 69.09 for Quant Expert in Qwen2VL-2B W4A6/W4A8 and InternVL2-8B W4A6/W4A8. The gains are therefore +3.21, +1.65, +2.09, and +1.27 points, respectively.

Q3. MASQuant already uses modality-specific smoothing and compensation. What does SPARCQuant add through its sparse formulation, channel score, adaptive base selection, and budget search?

Response:

SPARCQuant builds on MASQuant's modality-specific smoothing and compensation. MASQuant uses separate smoothing factors and a full-input low-rank correction while keeping one shared quantized weight. Its correction capacity is limited by the rank, so some channel errors can remain when the rank is small.

SPARCQuant adds four elements for multimodal smoothing:

  • Sparse support: We find that cross-modal smoothing mismatch is concentrated in a small set of channels. Only these channels receive modality-specific scale residuals and feed the compensation branch; other channels use the shared base scale.
  • Sensitivity score: We rank channels by activation magnitude weighted by their range mismatch with the base modality.
  • Adaptive base selection: We select the shared modality reference from the calibration statistics rather than fixing it in advance.
  • Budget search: We choose the number of repaired channels for each module through reconstruction search.

This allocates scale and correction parameters where the shared smoothing pattern is least suitable, while retaining one shared quantized weight.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Note to the Area Chair Regarding Writing Revisions in the Revised Manuscript

Dear Area Chair,

We would like to clarify the scope of our revision. In response to Reviewer dQeb's helpful comments regarding clarity, concision, precision, and academic presentation, we conducted a comprehensive language and presentation revision throughout the manuscript. Consequently, the visible changes in the main text may appear relatively substantial.

These changes are editorial and presentational only. We did not modify the core methodology, experimental setup, reported results, conclusions, claims, or scientific contributions. Instead, we revised the wording, organization, transitions, terminology consistency, and explanations to make the manuscript clearer, more concise, and easier to follow.

We sincerely appreciate Reviewer dQeb's guidance and the opportunity to improve the presentation of our work. We hope this clarification is helpful when evaluating the revised manuscript.

AuthorsOpenReview ↗

Final Summary to the Area Chair

Dear Area Chair,

As the discussion period concludes, we summarize the main reviewer concerns and how our rebuttals and revised manuscript resolve them. All three reviewers recognized SPARCQuant's strong low-bit accuracy, especially at W4A6 and on omni-modal tasks, and its practical CUDA speedups. Reviewer dQeb engaged and raised only writing-clarity suggestions in the second round, which we addressed via a full writing revision. Reviewers 6dXx and pAhK have not yet replied to our responses. Below are the resolved points:

  1. Novelty vs. Quant Experts and MASQuant, Reviewer 6dXx. We added the Quant Experts citation and comparison in Sections 1 and 2.2. Unlike Quant Experts' token-routed error reconstruction, SPARCQuant targets cross-modal smoothing mismatch and extends MASQuant with sparse support, sensitivity ranking, adaptive base selection, and automatic budget search. On Qwen2VL-2B and InternVL2-8B at W4A6 and W4A8, SPARCQuant beats Quant Experts' published averages by +1.27 to +3.21 points across all four settings.

  2. Controlled ablations and selector validation, Reviewers 6dXx and pAhK.

    • Controlled ablations, Appendix A.11, Tables 14 and 15: At W4A6 on Qwen3-VL-2B and MiniCPM-o-2.6 under identical budgets or matched parameters, isolating each component confirms complementary gains. Sensitivity-guided support beats random support at fixed KK by +3.23 on Qwen average and -8.99 on MiniCPM-o mean ASR WER. Selected-channel correction beats low-rank correction at matched parameters by +2.58 on Qwen and -4.56 on WER. Modality-specific scale residuals beat shared smoothing with δ=0\delta = 0 by +2.58 on Qwen and -9.69 on WER.
    • Sensitivity score validation, Appendix A.10, Table 13: We clarified how the oracle localization probe differs from actual repair and added matched-budget downstream comparisons at fixed K∈{8,16,32}K \in \{8, 16, 32\}. Sensitivity-guided selection beats both activation-only and range-gap-only selection across all KK. At K=32K = 32, it gains +3.06 and +4.71 on Qwen average, cuts MiniCPM-o mean ASR WER by 4.97 and 7.66, and raises vision average by +1.55 and +2.40.
  3. Statistical significance and evaluation validity, Reviewers 6dXx and pAhK.

    • Significance: Five-seed paired tests against MBQ on Qwen3-VL-2B and 8B confirm that W8A8 and W4A8 gains of +0.14 to +0.91 exceed evaluation noise, with paired SDs of 0.01 to 0.11 and all 95% and Bonferroni 98.75% confidence intervals excluding zero.
    • BF16 protocol and official presets: We documented exact checkpoints, dataset revisions, full 0-shot splits, and the uniform greedy 128-token lmms-eval protocol used for all Table 1 methods. Under published official vLLM 32k-token presets on full MMMU-val and OCRBench, SPARCQuant still beats MBQ and MASQuant, reaching 44.90 and 83.40 on 2B versus 43.90 and 81.60 for MBQ, and 52.90 and 89.10 on 8B versus 51.50 and 87.90 for MBQ. These scores stay close to local official BF16 at 45.44 and 84.10 on 2B and 53.44 and 89.70 on 8B, consistent with independent public reports on MMMU protocol sensitivity.
  4. Calibration robustness and one-time cost, Reviewer dQeb.

    • Robustness: Confirmed across held-out splits with 95% to 97% base agreement between 32 and 64 sequences in Appendix A.12.4 and Table 18, five independent 128-example calibration sets with SD 0.22 on Qwen average and at most 0.98 WER on MiniCPM-o, and a 32, 64, and 128 size sweep.
    • Cost, Appendix A.3, Table 7: Measured W4A6 calibration time on one RTX 5090 is 315.84 s on Qwen3-VL-2B and 523.46 s on MiniCPM-o-2.6, which is 3.81$\times$ faster than MBQ's 1995.44 s on MiniCPM-o. Calibration is a one-time offline step with zero runtime calibration overhead.
  5. Algorithmic details, definitions, and writing clarity, Reviewers 6dXx and dQeb. We added explicit calibration pseudocode in Algorithm 1, exact definitions of the proxy loss Lproxy\mathcal{L}^{\mathrm{proxy}} and ridge matrix Λ\Lambda in Sections 3.3 and 3.4, and complete budget-schedule parameters in the rebuttal. Following Reviewer dQeb's second-round advice, we also polished the writing and presentation across the manuscript without changing any methods, results, or claims.

We sincerely thank you for your time and thoughtful consideration in evaluating our submission.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.