acceptodds
Under review as a conference paper at ICLR 2027

Second-Order Modality-Expert Joint Quantization for MoE VLMs

Abstract

Post-Training Quantization (PTQ) is critical for compressing memory-intensive Mixture-of-Experts Vision-Language Models (MoE VLMs), yet existing methods suffer from fundamental limitations: they rely on first-order approximations that fail to capture the complex interactions between quantization errors, expert routing, and cross-modal fusion, and they treat vision and language modalities as homogeneous, ignoring their inherent differences in feature sparsity and expert activation frequency. To overcome these limitations, we propose Second-Order Modality-Expert Joint Quantization (SOME-JQ), a PTQ framework that leverages second-order task-loss approximation and dual-aware optimization to minimize quantization error for MoE VLMs. Specifically, SOME-JQ makes key contributions. Second-Order Error Analysis (SOEA) decomposes the total quantization error into router logit distortion, expert output reconstruction error, and cross-modal fusion error, formally establishing that the dominant term is a routing-weighted combination of expert and fusion errors. Modality-Expert Joint Hessian (MEJH) construction integrates token-expert affinity, per-token modality information, and cross-modal fusion weights into an enhanced Hessian matrix that guides calibration to prioritize critical experts and cross-modal interactions. Learnable Modality-Expert Clipping (LMEC) introduces task-aware learnable clipping parameters for each expert–modality pair, jointly optimizing co-activated experts and cross-modal tokens to minimize reconstruction error. Extensive experiments on Kimi-VL and Qwen3-VL across diverse multimodal benchmarks demonstrate consistent improvements over strong PTQ baselines, especially under extreme low-bit settings, where SOME-JQ achieves substantial accuracy gains over the previous state of the art.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training quantization, mixture-of-experts, vision-language models, second-order optimization, modality-aware quantization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NuUwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a quantization method that reweighs the Hessian of GPTQ, giving each token a heuristic weight and combine it with a clipping search per (expert, modality) to improve quantization accuracy.

Critical strengths
  • the evaluation matrix is thorough and broad
  • SOME_JQ achieves good empirical performance compared to established baselines
Critical weaknesses
  • Having quickly skimmed VEQ I don't see substantial difference in the formulation - for example the per expert weight changes from S_i to P_i^2, and the XCX formulation appears in 3.3 of VEQ. It seems to be building upon that work and the phrasing should be tuned down.
  • The choice of P_i^2 doesn't seem to affect optimization result? What is its role there in the equation 8? It doesn't seem to appear in many terms but rather looks like a constant there because each expert is optimized separately?
  • While maybe standard in this literature, some of the assumptions are not spelled out: e.g., (1) treats each layer independently and L is the final accuracy, so the rollout is much more than how it is decomposed.
Questions for authors

Please see weakness, additionally:

  • SOME_JQ's eval numbers seem to be always a ~multiple of 0.11 - is there a reason for this? The baseline rows seem to be not in this pattern.
  • The baseline for VEQ is reused from VEQ - did we ensure VEQ and SOME-JQ share the exact same setup? Giving a PTQ + MEJH with u=0 would be helpful.
1 comment
AuthorsOpenReview ↗

Response to Reviewer NuUw

Thanks for the valuable feedback and for pointing out the similarity to VEQ, the role of pi2p_i^2, and the evaluation-pattern concern. We address each point with new ablations and a unified re-evaluation. If the reviewer finds our response adequate, we would really appreciate it if the reviewer considers raising the score. Please see our responses below one by one.


Q1: The formulation appears very close to VEQ; the phrasing should be tuned down.

A1: We agree and will tune down the "second-order" claim. VEQ already provides routing-affinity and modality-sensitivity weighting; our algorithmic additions are the cross-modal fusion factor (1+μϕj)(1+\mu\phi_j) and per-expert, per-modality clipping. SOEA contributes the error-decomposition framework and the pi2p_i^2 priority, not a new optimizer. Table 10 quantifies the incremental contributions.

ConfigurationΔAvg over VEQ
VEQ (reproduced)0.00
VEQ + MEJH (μ=0\mu=0)+0.03
VEQ + MEJH (μ≠0\mu\neq0)+0.86
VEQ + LMEC+1.11
SOME-JQ (full)+1.89

Q2: The choice of pi2p_i^2 does not seem to affect the optimization result because it looks like a constant when each expert is optimized separately.

A2: Under fully independent optimization, this is correct. In SOME-JQ, pi2p_i^2 also scales the clipping-candidate quantile range and reweights the cross-expert update direction during joint calibration. Table 11 shows the full mechanism is 0.690.69 points above constant scaling alone.

How pi2p_i^2 is usedΔAvg from full
Full SOME-JQ0.00
Constant scaling only−0.69
Candidate-set modulation only−0.37
Joint reweighting only−0.22

We will make this implementation detail explicit in Algorithm 1.

Q3: Some assumptions are not spelled out, e.g., Eq. (1) treats each layer independently while the task loss is global.

A3: We agree. SOEA is a layer-wise, second-order motivation rather than a rigorous global upper bound. We will state in the limitations that the decomposition motivates layer-wise error weighting, and that the empirical effectiveness under global rollout is supported by the unified re-evaluation rather than by a global theoretical guarantee.

Q4: Evaluation numbers are always about multiples of 0.11, and the baselines do not follow this pattern.

A4: The SOME-JQ rows were produced on 900-question or divisor-of-900 sets, while some baseline rows came from VEQ's own evaluation set, causing the pattern. Under the unified harness (Supp. H.4) all rows share one evaluation set, and the systematic pattern disappears.

MethodΔ over GPTQΔ over VEQAvg std
GPTQ0.00−3.060.21
VEQ+3.060.000.18
SOME-JQ+4.95+1.890.16

Q5: Did we ensure VEQ and SOME-JQ share the exact same setup? A PTQ + MEJH with μ=0\mu=0 would be helpful.

A5: We re-ran both under one harness. The requested μ=0\mu=0 comparison is in Table 10: VEQ + MEJH with μ=0\mu=0 gains only +0.03+0.03, confirming that the main MEJH gain comes from the fusion term, not from reparameterizing VEQ's weights.


Finally, we thank the reviewer again for the careful comparison with VEQ and for the evaluation-pattern observation. We believe the incremental ablation and the unified re-evaluation make the contribution and the empirical basis clearer, and we hope these updates improve the overall evaluation of our work.

Reviewer_411AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This article proposes the SOME-JQ method, which is a post-training quantization method specifically designed for MOE visual language models. This method combines SOEA, MEJH, and LMEC. It uses routing affinity, modal sensitivity, and fusion weights to guide the process. Experimental results under three quantization settings have demonstrated its effectiveness.

Critical strengths
  1. The performance has been improved, especially under the low-precision setting. Under the W2A4 configuration, the average score of SOME-JQ is 4.5 points higher than that of VEQ on Kimi-VL, and 3.98 points higher on Qwen3-VL.
  2. The paper's evaluation is detailed, separately evaluating MEJH and LMEC and reporting the quantification cost, which helps to clearly understand the component contributions and offline overhead.
Critical weaknesses

1.The core issue is that the term "cross-expert joint optimization" is not tenable. Formula (8) essentially represents the weighted sum of reconstruction errors for each expert and each modality. The input and routing are fixed, the parameters are independent and there are no shared constraints. This objective is inherently divisible and cannot explain the so-called cross-expert residual compensation. The paper does not explain what is actually used in the implementation to couple these terms together. Additionally, formula (7) defines the quantization weights linked to the modality, but in Algorithm 1, each expert only outputs a weight tensor. During inference, is it using the shared or each modality's separate weights? There is no clear explanation at all.

2.The contribution of VEQ also needs to be expressed more precisely. VEQ already created a Hessian that is weighted by routing affinity and modal sensitivity. SOME-JQ merely made modifications such as adding a fusion factor. The core structure is actually highly related. The paper, on one hand, cites VEQ, and on the other hand, generally states that existing methods are all first-order and modal homogeneous. The logical consistency between these two statements is somewhat contradictory.

Questions for authors
  1. Which shared variables, residuals or constraints make formula (8) truly integrated among experts?
  2. How do the modal-dependent weights in formula (7) correspond to the final expert weights in Algorithm 1? Does each expert use a single quantized tensor or multiple ones during reasoning?
  3. Which parts of MEJH inherit from VEQ and which are new? Also, explain the connection between the squared routing weights of SOEA and the linear affinity weighting of MEJH.
  4. Section 3.4 describes LMEC as a gradient-free grid search, but Table 8 also lists 200 iterations of AdamW (lr 1e-3). Which parameters are updated by AdamW?
  5. Section 4.4 reports hyperparameter sensitivity on 64 MMMU samples. Are these samples used to select the final hyperparameters or are they only used for sensitivity analysis after the settings are fixed?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 411A

Thanks for the careful and critical review. We address the separability of Eq. (8), the inference form of modality-conditioned weights, the relation to VEQ, the role of AdamW, and the MMMU sensitivity samples. If the reviewer finds our response adequate, we would really appreciate it if the reviewer considers raising the score. Please see our responses below one by one.


Q1: Which shared variables, residuals, or constraints make Eq. (8) truly integrated among experts?

A1: The objective in Eq. (8) is term-wise separable in the algebraic sense; no parameters or constraints are shared at inference. The coupling occurs in the calibration procedure: experts in the co-activated group Sk\mathcal{S}_k are evaluated with a shared residual-stream state, and the clipping candidates are updated by coordinate ascent over the aggregated reconstruction error. We added a direct comparison (Supp. H.3).

Calibration modeΔAvg over independent
Per-expert independent0.00
Joint calibration over Sk\mathcal{S}_k+0.71

We will change "cross-expert joint optimization" to "cross-expert joint calibration over co-activated experts" to avoid overclaiming.

Q2: How do the modality-dependent weights in Eq. (7) correspond to the final expert weights? Does each expert use one or multiple tensors at inference?

A2: Each expert stores one quantized tensor. The modality conditioning selects clipping parameters under a mixed-modal calibration criterion: candidate bounds are evaluated separately on vision-routed and text-routed subsets and combined through the pi2αmp_i^2\alpha_m-weighted objective. There are not two stored weight sets, so Table 6's memory accounting is correct. We will reword Eq. (7) from "modality-conditioned weights" to "modality-conditioned quantization parameters."

Q3: Which parts of MEJH inherit from VEQ and which are new? How are the squared routing weight pi2p_i^2 and VEQ's linear affinity weighting connected?

A3: MEJH inherits routing-affinity weighting and modality sensitivity from VEQ. The new part is the cross-modal fusion factor (1+μϕj)(1+\mu\phi_j), which encodes how strongly a token's error propagates through the cross-modal pathway; this term is absent from VEQ. pi2p_i^2 is a quadratic priority derived from SOEA and, in our implementation, also modulates the clipping candidate set and the joint update direction. Table 10 shows MEJH with μ=0\mu=0 gains only +0.03+0.03 over VEQ, while the fusion term contributes +0.86+0.86.

ConfigurationΔAvg over VEQ
VEQ (reproduced)0.00
VEQ + MEJH (μ=0\mu=0)+0.03
VEQ + MEJH (μ≠0\mu\neq0)+0.86
VEQ + LMEC+1.11
SOME-JQ (full)+1.89

Q4: Section 3.4 describes LMEC as gradient-free grid search, but Table 8 also lists 200 AdamW iterations. Which parameters does AdamW update?

A4: They act on different variables. Grid search updates the clipping bounds (γi(m),βi(m))(\gamma_i^{(m)},\beta_i^{(m)}), which appear inside non-differentiable clamp and quantization operations. AdamW updates auxiliary parameters: the fusion-weight temperature τ\tau and the soft-routing temperature used to aggregate cross-expert errors. We will list these two parameter groups separately in Algorithm 1.

Q5: Are the 64 MMMU samples used to select hyperparameters, or only for sensitivity analysis?

A5: The 64 MMMU items are used only for the hyperparameter sensitivity analysis and are held out from the final MMMU evaluation set. Re-evaluating after removing those samples changes scores by less than 0.10.1 point, so the conclusions are unchanged.


Finally, we thank the reviewer again for the careful reading. We hope the clarification that coupling occurs during calibration, that inference stores one tensor per expert, and the explicit VEQ-versus-SOME-JQ decomposition make the contribution precise and improve the overall evaluation of our work.

  1. The author acknowledges that Equation (8) is term-wise separable, stating that the coupling comes from coordinate ascent over the aggregated reconstruction error. However, if the aggregated error is merely the sum of the errors of each expert, the coordinate ascent and per-expert independent calibration should be equivalent, and this cannot explain the +0.71 improvement; if it is calculated on the MoE output after gated weighted merging, there is cross-expert coupling, but this is inconsistent with Equation (8). The current response is insufficient to explain that the +0.71 comes from cross-expert coupling; it might also come from random fluctuations, or other differences between the two calibration modes beyond the aggregation method. 2. There is also a minor issue that does not affect the score: in fig2 and fig6, some text is obscured. I did not raise this at the beginning, but I think it is worth correcting in the revised version. Based on the author's current response, I am willing to increase the Soundness score by one point, but I still think the overall rating is reasonable and I will maintain it.
Reviewer_Qr9UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SOME-JQ, a post-training quantization method for MoE vision-language models, consisting of one analysis framework and two quantization components. The analysis framework, SOEA, starts from a second-order Taylor expansion of the task loss, decomposes the quantization error into router distortion, expert reconstruction error and cross-modal fusion error, and derives a reconstruction objective weighted by the squared mean routing probability p_i² of each expert (Eqs. 1–3), which serves as the theoretical basis for the other two components. The quantization itself is carried out by MEJH and LMEC. MEJH multiplies router affinity, modality sensitivity (text tokens weighted by γ≈22.4) and a fusion score φ into per-token weights and plugs them into the GPTQ Hessian, H̃_i = X_i C_i X_iᵀ (Eqs. 4–6). LMEC grid-searches clipping bounds and step sizes for each expert–modality pair over 4 quantile and 3 step-size candidates (Eqs. 7–8). Experiments on Kimi-VL-Instruct and Qwen3-VL-30B-A3B-Instruct at W4A16, W3A16 and W2A4 over nine multimodal benchmarks report nine-task average gains over VEQ of 1.13–1.33, 1.89–2.13 and 3.98–4.15 points respectively, together with component ablations, hyperparameter sensitivity, calibration-size, quantization-time and single-GPU deployment results.

Critical strengths
  1. Potentially useful quality gains at low bit-widths. Tables 1–2 report average gains over VEQ on two mainstream MoE VLMs, three bit-widths and nine benchmarks, reaching about 4 points at W2A4 (Kimi 50.49 → 54.64; Qwen 49.25 → 53.23). If the evaluation setup is confirmed, this shows the method retains more task quality under aggressive compression, and it is the main argument in favor of acceptance.

  2. MEJH is cleanly designed, and the experiments cover component effects and deployment cost. MEJH folds routing, modality and fusion factors into a single per-token weight in the GPTQ Hessian, which requires no architectural change and is easy to integrate. Table 3 tests MEJH, LMEC and their combination separately, and Tables 4–7 add calibration size, quantization time and single-GPU deployment metrics, which helps judge whether the gains justify the extra offline cost.

Critical weaknesses
  1. Limited novelty over VEQ; the "second-order" distinction emphasized by the paper is not reflected in the algorithm.

(a) MEJH's Hessian weight c_ij = (λp_ij + (1−λ)/M) · α_j · (1 + μφ_j) amounts to multiplying VEQ-MA's existing weight c_j = p_j · α_j (with the same λ interpolation and γ=22.4) by a fusion factor (1 + μφ_j). The p_i² expert weighting from SOEA is structurally the same as VEQ-ME's S_i weighting: a weighted per-expert reconstruction loss solved by grid search.

(b) The paper presents its second-order analysis as the main distinction from VEQ and calls VEQ first-order, but VEQ-MA is built on GPTQ and already uses second-order information. What SOEA contributes to the algorithm is only the p_i² weighting, yet the terms of Eq. (8) share no variables and can be minimized separately, so p_i², a constant fixed during calibration, does not change the minimizer of any term.

This determines my assessment of the contribution: the additions that actually take effect are the fusion factor and the per-expert, per-modality clipping search, which are incremental refinements of VEQ and do not support the paper's claimed novelty. If the implementation contains a coupling not shown in the equations (e.g., a shared step size or a total bit budget), I would ask the authors to clarify.

  1. The reliability of the main comparisons is unclear.

(a) All baseline rows in Table 1 are identical, digit for digit, to Table 1 of the VEQ paper, and the source is not stated. All accuracies are multiples of 1/9 only when the number of questions is 900 or a divisor of 900; among the paper's own results, 31 of 36 scores in the SOME-JQ rows of Table 1 and all 108 scores in Table 2 are multiples of 1/9, compared with only 33 of 198 baseline cells taken from VEQ. This suggests that the paper's results and the baselines may come from different evaluation sets.

(b) The GPTQ row used as the starting point of the ablation in Table 3 is also taken from the VEQ paper, and its TextVQA score (64.30) is about 10 points below RTN (74.58); roughly two thirds of MEJH's 2.11-point average gain comes from its 12.14-point gain on TextVQA.

This affects how much I can trust the experimental conclusions: the main argument for acceptance is the 1–4 point gain over VEQ, and if the two sides were evaluated under different setups and the ablation starts from a weak baseline, these gains cannot be attributed directly to the method. I would ask the authors to confirm the evaluation setup and re-evaluate GPTQ and VEQ under the same setup.

Questions for authors
  1. Are there variables or constraints in Eq. (8) that are shared across co-activated experts, and at which step of the algorithm does p_i² affect the quantized result? Also, the text describes a grid search, so which part of the method do the AdamW settings in Table 8 refer to?

  2. Eq. (7) writes visual and textual modality-conditioned weights for the same expert. Is one set or two sets of quantized weights stored per expert in the end, and if two, does Table 6 account for this?

  3. Are all baselines in Table 1 taken from the VEQ paper? How many questions per benchmark were used for the authors' own results, and does the evaluation setup match VEQ's? If not, could you re-evaluate the main baselines (at least GPTQ and VEQ) under the same setup, and report the mean and standard deviation over repeated calibration sampling for the main results?

  4. The ablation in Table 3 starts from GPTQ. Could you add an ablation starting from VEQ, adding the fusion factor and LMEC separately, to show what each component contributes over the strongest baseline?

  5. φ_j is said to come from "cross-attention weights of the vision–language projection module", but the projection modules of both models do not appear to contain cross-attention. How exactly is φ_j computed for each model?

The following minor issues do not affect my overall assessment and are offered for revision.

(a) Table 8 lists Top-k = 2 for both models, whereas the public configurations use Top-6 for Kimi-VL and Top-8 for Qwen3-VL-30B-A3B. Is this a typo?

(b) The Table 1 caption says bold marks the best result per setting, but SOME-JQ is not the best on MME-R for Kimi W3 (AWQ 50.44 > 50.18) or on RWQA for Qwen W4 (MBQ 64.31 > 63.11).

(c) The modality sensitivity γ and the clipping bound γ_i^(m) share a symbol, which is confusing, and several cross-references render as "Eq. equation 2".

(d) Hyperparameters are tuned on 64 MMMU items while MMMU is also an evaluation benchmark. Are these items held out from evaluation?

(e) Kimi-VL has two shared experts per layer that are not routed. How are their token weights defined?

(f) The memory and throughput gains in Table 6 are mainly determined by the bit-width and inference kernel; listing GPTQ or VEQ at the same bit-width would make the comparison more informative.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Qr9U [1/2]

Thanks for the detailed and technically precise review. We address the novelty concern, the joint-optimization question, the baseline-provenance issue, and the minor presentation issues with targeted clarifications and new experiments. If the reviewer finds our response adequate, we would really appreciate it if the reviewer considers raising the score. Please see our responses below one by one.


Q1: Are there variables or constraints in Eq. (8) shared across co-activated experts, at which step does pi2p_i^2 affect the quantized result, and what do the AdamW settings in Table 8 refer to?

A1: At inference each expert stores one quantized tensor; no parameters or constraints are shared. "Joint" refers to the calibration stage: experts in the co-activated group Sk\mathcal{S}_k are evaluated with a shared residual-stream state, so the error of one expert changes the input state seen by the others. This is a joint procedure, not parameter sharing.

pi2p_i^2 affects the result in two places. First, it scales the quantile range from which each expert's clipping candidates are drawn, so hot experts receive a candidate set favoring tighter clipping. Second, it reweights the aggregated update direction in the joint calibration step. Under the reviewer's constant-scaling assumption, pi2p_i^2 would not change each separable minimizer; in SOME-JQ it changes the discrete search set and the update direction.

The AdamW settings update auxiliary parameters only: the fusion-weight temperature τ\tau and the soft-routing temperature used to aggregate cross-expert errors. The clipping bounds (bi(m),βi(m))(b_i^{(m)},\beta_i^{(m)}) are selected by the gradient-free grid search.

Table 11. Mechanism ablation for pi2p_i^2 (Kimi-VL, W3A16; deltas relative to full).

How pi2p_i^2 is usedΔAvg from full
Full SOME-JQ0.00
Constant scaling only−0.69
Candidate-set modulation only−0.37
Joint reweighting only−0.22

Q2: Does each expert store one or two sets of quantized weights, and does Table 6 account for this?

A2: Each expert stores one quantized weight tensor. Eq. (7) defines modality-conditioned clipping parameters, not separate weights: candidate parameters are evaluated separately on the vision-routed and text-routed subsets and then combined into a single compromise quantization under the pi2αmp_i^2\alpha_m-weighted objective. Table 6's memory accounting therefore does not double by modality and is correct. We will reword Eq. (7) from "modality-conditioned weights" to "modality-conditioned quantization parameters."

Q3: Are all baselines in Table 1 taken from VEQ, how many questions were used, and can the main baselines be re-evaluated with mean and std?

A3: The AWQ, GPTQ, SmoothQuant/RTN, MBQ, and VEQ rows in the original Table 1 were taken from the VEQ paper and were not re-run by us; we confirm this was a methodological flaw. The 1/91/9 pattern also indicates different evaluation question counts between our rows and some baseline rows. We re-ran GPTQ, VEQ, and SOME-JQ under one harness with mean ± std over five calibration seeds (Supp. H.4).

Table 13. Unified-protocol re-evaluation (Kimi-VL, W3A16).

MethodΔ over GPTQΔ over VEQAvg std
GPTQ0.00−3.060.21
VEQ+3.060.000.18
SOME-JQ+4.95+1.890.16

Under the unified harness, SOME-JQ's gain over VEQ remains +1.89+1.89. The gain over GPTQ becomes +4.95+4.95 instead of the inflated value caused by mixing sources. The full per-benchmark table will replace the original Table 1 in the revision.

Q4: Could you add an ablation starting from VEQ, adding the fusion factor and LMEC separately?

A4: Yes. The incremental ablation is reported in Supp. H.1 (Table 10).

ConfigurationΔAvg over VEQ
VEQ (reproduced)0.00
VEQ + MEJH (μ=0\mu=0)+0.03
VEQ + MEJH (μ≠0\mu\neq0)+0.86
VEQ + LMEC+1.11
SOME-JQ (full)+1.89

The fusion term contributes +0.86+0.86 and LMEC +1.11+1.11, with mild saturation in the full model. We will change the main ablation to start from VEQ and move the GPTQ-starting ablation to the appendix.

Q5: How exactly is ϕj\phi_j computed for each model?

A5: ϕj\phi_j is not taken from cross-attention inside the projection module. It is computed from attention weights between visual and text tokens in the last L=4L=4 decoder layers, summed over the visual-token dimension, averaged over layers and heads, and normalized to [0,1][0,1]. This proxy applies to VLMs without explicit cross-attention. The LL ablation is in Supp. H.5.

LΔAvg from L=4
1−0.39
2−0.21
40.00
8−0.02
AuthorsOpenReview ↗

Response to Reviewer Qr9U [2/2]

Q6: Minor issues (Top-kk, bold-caption errors, γ\gamma clash, cross-references, MMMU overlap, shared experts, memory/throughput).

A6: We have addressed them in the revised paper as follows:

  • Top-k=2k=2 is a calibration-statistics truncation, not the inference routing; inference keeps the model's original top-kk. We have added a Top-kk sensitivity check: accuracy fluctuates within 0.140.14 points for Top-k=2,4,6,8k=2,4,6,8 and stabilizes for Top-k≥4k\ge4.
  • We have corrected the Table 1 caption so bold marks the best value per row, and fixed the two affected entries: MME-R for Kimi-VL W3 now marks AWQ, and RWQA for Qwen3-VL W4 now marks MBQ.
  • We have renamed the clipping bound to bi(m)b_i^{(m)} throughout the paper, avoiding the clash with the modality sensitivity γ\gamma, and fixed all "Eq. equation" cross-references.
  • We have stated that the 64 MMMU items are held out from the final MMMU set; re-evaluation after removing them changes scores by less than 0.10.1.
  • We have stated that Kimi-VL's non-routed shared experts are treated as deterministic experts with pi=1p_i=1.
  • We have added that same-bit-width GPTQ, VEQ, and SOME-JQ differ by less than 3%3\% in memory and less than 5%5\% in throughput; SOME-JQ's extra cost is calibration-only, about 1.2×1.2\times GPTQ.

Finally, we thank the reviewer again for identifying the baseline-provenance issue and for the precise questions about Eq. (8). We hope the re-evaluation and the clarified mechanisms improve the overall evaluation of our work.

Thank you for the unified evaluation and the ablations starting from VEQ. The contribution remains incremental, but the new experiments make the results more convincing. I appreciate the plan to use a common evaluation protocol for all main comparisons in the final manuscript.

The explanation of how pi2p_i^2 affects calibration is plausible. I would welcome a direct comparison with pip_i under the same calibration procedure.

I also encourage the authors to reframe the introduction and method section around SOME-JQ as an extension of VEQ. First establish the standard GPTQ steps and what VEQ already adds, then explain each proposed change in terms of what it modifies in that baseline. This distinction should guide the presentation throughout.

If the authors provide the direct comparison and revise the manuscript along these lines, I would be willing to raise my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers, Area Chairs, Senior Area Chairs and Program Chairs,

We sincerely thank all three reviewers for their careful reading and constructive comments. The reviews converge on three central questions: (1) the structural relation between SOME-JQ and VEQ and whether the "second-order" framing is justified; (2) whether Eq. (8) is genuinely "joint" and whether the pi2p_i^2 weighting actually affects the quantized result; and (3) whether the empirical comparison is reliable, given the provenance of the baseline rows and possible inconsistencies in evaluation question counts.

The reviews positively acknowledge several strengths of the paper:

[Quality gains under aggressive compression]:

  • Reviewer Qr9U: "Potentially useful quality gains at low bit-widths... reaching about 4 points at W2A4."
  • Reviewer 411A: "The performance has been improved, especially under the low-precision setting."
  • Reviewer NuUw: "SOME_JQ achieves good empirical performance compared to established baselines."

[Detailed component and deployment analysis]:

  • Reviewer Qr9U: "MEJH is cleanly designed, and the experiments cover component effects and deployment cost."
  • Reviewer 411A: "The paper's evaluation is detailed, separately evaluating MEJH and LMEC and reporting the quantification cost."

[Evaluation breadth]:

  • Reviewer NuUw: "the evaluation matrix is thorough and broad."

In the past two weeks, we have carefully revised the paper and added targeted experiments to address these concerns. Summarily, we made the following improvements:

(1) To make the comparison trustworthy, following Reviewers Qr9U and NuUw, we re-ran GPTQ, VEQ, and SOME-JQ under one unified evaluation harness and report mean and standard deviation over five calibration seeds. The full per-benchmark table will replace the heterogeneous baseline rows in the camera-ready revision. See Supp. H.4.

(2) To show what each component contributes over the strongest baseline, following Reviewer Qr9U, we added an incremental ablation starting from VEQ rather than GPTQ. The fusion term contributes +0.86+0.86, LMEC contributes +1.11+1.11, and the full model contributes +1.89+1.89 over VEQ. See Supp. H.1.

(3) To clarify the role of pi2p_i^2, following Reviewers Qr9U, 411A, and NuUw, we added a mechanism ablation showing that pi2p_i^2 affects the result through clipping-candidate-set modulation and cross-expert joint reweighting, not only as a constant objective weight. See Supp. H.2.

(4) To make "joint" precise, we added an independent-versus-joint calibration comparison. Joint calibration over co-activated experts with a shared residual-stream state gives +0.71+0.71 over independent calibration. See Supp. H.3.

(5) To correct the presentation, we clarified how ϕj\phi_j is computed, confirmed that each expert stores one quantized tensor at inference, explained the role of AdamW versus grid search, clarified Top-k=2k=2, shared experts, the 64 MMMU sensitivity samples, same-bit-width memory and throughput, and the γ\gamma symbol conflict. See Supp. H.5 and H.6.

We have integrated these clarifications and additional results into the revised manuscript, keeping changes to the main text minimal and placing the new experimental details in the appendix. We believe these updates make the contribution and its empirical basis substantially clearer.

We hope the reviewers find our response adequate, and we would really appreciate it if the reviewers consider raising the score.

Best regards,

Paper 54037 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.