- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
SPARCQuant is a post-training weight-activation quantization method for Large Multimodal Models, designed to mitigate the cross-modal smoothing mismatch caused by disparities in activation distribuitions across text, vision, and audio. Observing that a small fraction of input channels accounts for a disproportionate share of mismatch scores and recoverable output errors, this paper proposes applying modality-specific smoothing and compensation exclusively to selected channels while maintaining shared quantized main weights.
Specifically, SPARCQuant adaptively selects a base modality for the linear modules in each layer and ranks the channels of non-base modalities by jointly considering activation magnitudes and inter-modal range discrepancies. During the calibration phase, it determines the number of channels to repair via a coarse-to-fine search with a complexity penalty. Modality-specific scales deviate from the shared scales only on these selected channels. Subsequently, leveraging the quantizaed inputs of the chaosen channels, output residual compensation is fitted via ridge regression. The enthire pipeline employs layer-wise calibration and parameter fitting, eliminating the need for full-model retraining.
Critical strengths
-
Insightful Analysis of Channel-Level Misalignment: The paper provides a precise diagnosis of channel-level misalignment issues in MLLM quantization, identifying a critical gap overlooked by prior methods. Based on this insight, the proposed algorithmic design effectively targets these discrepancies, offering a well-motivated solution.
-
Comprehensive Experimental Validation: The method is thoroughly evaluated across a wide range of models, quantization settings, and benchmarks, demonstrating robust effectiveness. Notably, the inclusion of CUDA-based end-to-end inference time measurements provides strong evidence of its practical deployment viability.
Critical weaknesses
-
Concerns regarding robustness to calibration data. Unlike MBQ and MasQuant, which provide guarantees based on mathematical equivalence for linear layer operations, SPARCQuant’s output compensation via Ridge Regression relies entirely on data fitting. Moreover, both the base modality selection and the number of compensated channels are driven purely by data-based search. The authors should demonstrate the algorithm's robustness against calibration data variations through either theoretical analysis or empirical evidence.
-
Potentially significant optimization costs. The proposed method involves a multi-stage optimization pipeline consisting of (1) Statistical Base Selection, (2) Iterative Gradient Optimization with Budget Search, and (3) Closed-form Ridge Regression Fitting. However, the paper fails to quantify the total computational cost of this calibration process. This omission is critical when comparing SPARCQuant against lightweight quantization schemes such as SmoothQuant and MBQ.
Questions for authors
Clarification of Algorithm Pipeline: The current description of the algorithm flow is difficult to follow, as SPARCQuant involves multiple intertwined stages of search, regression, and training, with many implementation details omitted from the main text (e.g., the specific optimization process for smsm and δm, and the ridge regression fitting for Bm and cm). I strongly recommend that the authors provide a comprehensive pseudocode or an explicit algorithm pipeline in the revision to serve as a clear guide for readers.
6 comments
Response to Reviewer dQeb (Part2)
Q1. Unlike MBQ and MASQuant, which provide guarantees based on mathematical equivalence for linear-layer operations, SPARCQuant's ridge compensation relies on data fitting, while base-modality selection and the number of compensated channels are determined by data-driven search. Can theoretical analysis or empirical evidence demonstrate its robustness to variations in calibration data?
Response:
The concern is whether changing calibration data changes the fitted correction, selected base, or number of compensated channels. We test this at three levels. Pre-quantization equivalence does not show that a fitted correction works on new inputs, so held-out reconstruction is the direct check.
- Held-out reconstruction: Fit on one set and evaluate on disjoint inputs at fixed . If the correction memorizes fitting inputs, held-out NMSE should increase; base agreement tests whether the selected base is stable.
- Independent calibration sets: Rerun the full pipeline five times with different 128-example sets. Base selection, budget search, scale optimization, and ridge fitting all vary, so the score SD measures end-to-end sensitivity.
- Calibration size: Use nested sets of 32, 64, and 128 examples. A large score change would indicate dependence on calibration size; small changes would indicate stable behavior.
For downstream evaluation, Qwen3-VL uses COCO calibration examples and complete benchmark splits; MiniCPM-o uses OmniBench calibration examples and complete splits for LibriSpeech, FLEURS, TED-LIUM, MMMU, and OmniBench. No evaluation labels are used during calibration.
Held-out reconstruction (W4A6). The appendix scales NMSE by ; values below are unscaled. Base agreement compares the 32- and 64-sequence choices.
| Model | Base agreement | NMSE (32) | NMSE (64) |
|---|---|---|---|
| Qwen3-VL-2B | 97.02% | 0.02634 | 0.02643 |
| MiniCPM-o-2.6 | 95.24% | 0.04015 | 0.04090 |
NMSE changes by only 0.00009 and 0.00075, while base agreement remains 95--97%. Thus, the fitted correction generalizes to held-out inputs and the base choice is stable.
Five independent 128-example sets (W4A6). Entries are mean sample SD over five runs. Qwen task scores and MiniCPM vision scores are higher-is-better; MiniCPM speech results are WER, where lower is better.
| Qwen3-VL-2B | MMMU | TextVQA | VizWiz | ScienceQA | OCRBench | Avg. |
|---|---|---|---|---|---|---|
| MBQ | 30.87 ± 0.31 | 62.33 ± 0.55 | 56.79 ± 0.33 | 44.73 ± 0.80 | 58.24 ± 0.30 | 50.59 ± 0.25 |
| MASQuant | 29.42 ± 0.63 | 53.60 ± 0.54 | 35.12 ± 0.94 | 10.45 ± 1.08 | 52.96 ± 1.02 | 36.31 ± 0.38 |
| SPARCQuant | 33.44 ± 0.38 | 64.42 ± 0.39 | 56.74 ± 0.40 | 61.21 ± 0.70 | 57.94 ± 0.40 | 54.75 ± 0.22 |
| MiniCPM-o-2.6 | LibriSpeech WER | FLEURS WER | TED-LIUM WER | MMMU | OmniBench |
|---|---|---|---|---|---|
| MBQ | 57.59 ± 1.29 | 95.14 ± 1.66 | 71.44 ± 1.91 | 33.87 ± 0.89 | 35.79 ± 0.52 |
| MASQuant | 98.78 ± 2.64 | 178.07 ± 5.88 | 167.44 ± 4.54 | 35.78 ± 0.14 | 36.28 ± 0.42 |
| SPARCQuant | 22.36 ± 0.72 | 29.52 ± 0.26 | 64.38 ± 0.98 | 41.16 ± 0.48 | 42.16 ± 0.38 |
Across sets, SPARCQuant's Qwen average is 54.75 vs. 50.59 for MBQ (SD 0.22). Its mean ASR WER is 38.75, vs. 74.72 for MBQ and 148.10 for MASQuant; its vision average is 41.66, vs. 34.83 and 36.03. These small SDs show that the full pipeline gives the same conclusion across calibration sets.
Calibration-size sweep for SPARCQuant (W4A6). Entries are mean SD over five runs. Qwen columns are MMMU/TextVQA/VizWiz/ScienceQA/OCRBench/Avg.; MiniCPM columns are LibriSpeech/FLEURS/TED-LIUM WER/MMMU/OmniBench.
| Qwen3-VL-2B () | MMMU | TextVQA | VizWiz | ScienceQA | OCRBench | Avg. |
|---|---|---|---|---|---|---|
| 32 | 33.27 ± 0.54 | 63.82 ± 0.47 | 56.21 ± 0.65 | 60.06 ± 1.26 | 57.68 ± 0.56 | 54.21 ± 0.46 |
| 64 | 33.22 ± 0.39 | 64.20 ± 0.44 | 56.56 ± 0.46 | 60.96 ± 0.79 | 57.84 ± 0.39 | 54.56 ± 0.23 |
| 128 | 33.44 ± 0.38 | 64.42 ± 0.39 | 56.74 ± 0.40 | 61.21 ± 0.70 | 57.94 ± 0.40 | 54.75 ± 0.22 |
| MiniCPM-o-2.6 () | LibriSpeech WER | FLEURS WER | TED-LIUM WER | MMMU | OmniBench |
|---|---|---|---|---|---|
| 32 | 23.95 ± 0.77 | 32.03 ± 0.65 | 66.85 ± 0.73 | 40.60 ± 0.47 | 42.09 ± 0.54 |
| 64 | 22.96 ± 0.84 | 30.21 ± 0.83 | 65.20 ± 1.33 | 40.96 ± 0.65 | 42.14 ± 0.48 |
| 128 | 22.36 ± 0.72 | 29.52 ± 0.26 | 64.38 ± 0.98 | 41.16 ± 0.48 | 42.16 ± 0.38 |
From 32 to 128 examples, the Qwen average changes only from 54.21 to 54.75; MiniCPM mean ASR WER changes from 40.94 to 38.75 and vision average from 41.35 to 41.66. Across independent 128-example sets, SDs are at most 0.70 points for Qwen task scores and 0.98 WER points for MiniCPM ASR. Together, these tests show that the correction generalizes, base and budget choices are stable, and the conclusion is not tied to one calibration set or size. These diagnostics are included in the revised appendix.
Response to Reviewer dQeb (Part3)
Q2. SPARCQuant involves a multi-stage optimization pipeline: statistical base selection, iterative gradient optimization with budget search, and closed-form ridge regression fitting. What is the total computational cost of this calibration process, and how does it compare with lightweight quantization methods such as SmoothQuant and MBQ?
Response:
We measured all four methods at W4A6 with 128 calibration examples on one RTX 5090 using four CPU threads. Timings are synchronized calibration-stage measurements and exclude model loading, preprocessing, deployment packing, and evaluation.
| Model | SmoothQuant | MBQ | MASQuant | SPARCQuant |
|---|---|---|---|---|
| Qwen3-VL-2B | 13.72 s | 144.20 s | 208.30 s | 315.84 s |
| MiniCPM-o-2.6 | 26.18 s | 1995.44 s | 299.62 s | 523.46 s |
SPARCQuant is 23.0×/20.0× slower than SmoothQuant and 1.52×/1.75× slower than MASQuant on Qwen3-VL-2B/MiniCPM-o-2.6. Compared with MBQ, it is 2.19× slower on Qwen3-VL-2B and 3.81× faster on MiniCPM-o-2.6. MBQ's MiniCPM time is high because it evaluates many candidate scales for every layer across tri-modal inputs.
For a given model checkpoint, quantization and calibration are one-time offline preparation steps. After calibration, the selected bases, budgets, scales, and compensation weights are fixed; inference does not rerun search, gradient optimization, or ridge fitting, so this calibration cost adds no runtime calibration overhead. The one-time cost is also amortized over all later inference requests. In post-training quantization, paying additional offline calibration time for a persistent accuracy gain is a standard and worthwhile trade-off: deployment quality improves without retraining the model or adding calibration computation during inference. SPARCQuant makes this trade-off to obtain the accuracy gains from sparse modality correction.
We have included the calibration-cost table and measurement protocol in the revised appendix.
Q3. The algorithm flow is difficult to follow because search, regression, and training involve multiple intertwined stages, and the main text omits implementation details, including the optimization of and and the ridge regression fitting of and . Can the revision provide comprehensive pseudocode or an explicit algorithm pipeline to guide readers?
Response:
We have added an explicit pipeline and pseudocode to the revision. The new description separates statistical base selection, budget search, joint optimization of and , and closed-form ridge fitting of and , and states the corresponding inputs, outputs, and update order.
Thank you for the constructive review. The added experiments quantify calibration robustness, and the revised cost and algorithm descriptions make the calibration procedure easier to reproduce. If these clarifications address your concerns, we would be grateful if you would consider raising our score.
Response to Reviewer dQeb (Part1)
Response to Reviewer dQeb
We thank the reviewer for the questions about calibration robustness, calibration cost, and the algorithm description. We address each point with the relevant measurements and implementation details. We hope these clarifications address your concerns and sincerely ask you to consider raising our score.