- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents HeteroQuant, a training-free calibration data selection framework for post-training quantization of Mixture-of-Experts (MoE) models. The proposed approach consists of three components: (1) DSPy-based multi-domain candidate generation, (2) AlphaQ-guided sample importance scoring based on expert structural importance and routing probabilities, and (3) redundancy-aware sample filtering based on layer-wise routing similarity. Experiments on three MoE architectures show improvements over several W4A4 PTQ baselines.
Critical strengths
1.The paper addresses a practical problem in MoE quantization: heterogeneous expert activation and insufficient coverage of important experts during calibration.
2.The framework is conceptually simple and does not require gradient-based model fine-tuning.
3.The experiments cover three different MoE models and report improvements on multiple downstream benchmarks.
Critical weaknesses
- Limited methodological novelty.
The proposed framework largely combines existing techniques, including DSPy-based synthetic data generation, AlphaQ-based expert importance estimation, and cosine-similarity-based greedy sample filtering. The principal algorithmic contribution is a weighted aggregation of expert importance and routing probabilities, followed by a conventional redundancy filter. The paper does not sufficiently establish why these particular formulations are necessary or superior to simpler alternatives, such as routing-frequency sampling, activation-based importance metrics, or standard diversity-based subset selection.
- Insufficient isolation of calibration data selection from other factors.
The main experiments compare HeteroQuant, which uses a newly generated multi-domain calibration pool, against baselines calibrated with WikiText-2. As a result, the improvements may be partly attributed to the change in data distribution rather than the proposed expert-aware selection strategy. Moreover, the paper includes expert-aware smoothing and scale-merging operations whose roles in the performance gains are not sufficiently isolated. A controlled comparison using an identical PTQ implementation and calibration candidate pool is necessary to establish the incremental contribution of the proposed selection algorithm.
- A gap between expert structural importance and quantization calibration sensitivity.
The motivation relies on pruning experiments demonstrating that removing structurally important experts causes severe performance degradation. However, pruning sensitivity does not necessarily imply quantization sensitivity or a greater marginal benefit from additional calibration samples. The paper does not directly demonstrate that AlphaQ-derived expert importance correlates with actual quantization-induced errors or calibration requirements. This weakens the justification for the central sample scoring objective.
- Unclear mechanism through which calibration data influence the reported W4A4 configuration.
The implementation employs dynamic per-token activation quantization together with expert-aware smoothing. Since dynamic activation quantization generally computes activation scales at inference time, the paper should explicitly identify which quantization parameters or transformations are estimated from calibration data and how the proposed selection strategy improves them. The relationship between the generic calibration formulation in Section 3.1 and the actual quantization configuration in Section 4.1 requires clarification.
- Limited ablation evidence and incomplete cost analysis.
The improvements attributed to individual components in Table 2 are relatively modest, and no variance estimates are provided. The complete method also does not achieve the best WikiText-2 perplexity among the ablated configurations. Furthermore, although the final calibration set contains 128 sequences, the overall pipeline requires synthetic data generation, target-model inference on 512 candidate sequences, spectral analysis, and routing-statistics collection. These additional costs should be quantified.
- Insufficient evaluation of robustness and generality.
The evaluation primarily focuses on W4A4, with reduced router precision evaluated on only one model. Additional experiments across quantization bit-widths, modern MoE architectures, candidate pool sizes, and redundancy thresholds would strengthen the generality claims. Important implementation details, particularly the similarity threshold and synthetic data generation configurations, should also be clarified.a
Questions for authors
1.How does AlphaQ-derived expert importance correlate with actual expert-wise quantization error, output reconstruction error, and the marginal benefit of additional calibration tokens?
2.What are the results when random sampling, routing-balanced sampling, importance-only sampling, and diversity-only sampling are applied to the same DSPy-generated candidate pool under an identical PTQ implementation?
3.Can the authors clarify exactly which quantization scales, smoothing factors, or other parameters are derived from calibration samples under dynamic per-token activation quantization?
4.Does the proposed calibration set consistently improve multiple independent PTQ methods when the quantization pipeline is held fixed?
5.What are the complete offline costs of generating and selecting calibration data, and how sensitive are the results to the candidate pool size and similarity threshold?