- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses accuracy degradation and computational redundancy in low-bit post-training quantization of VLMs, arising from heterogeneous activation distributions. The authors propose TRPQuant, a framework that moves beyond conventional modality-level token partitioning by dynamically categorizing tokens into Dense, Intermediate, and Sparse groups based on their channel-wise activation energy distributions. Different quantization strategies are applied to each group: dense tokens retain full-channel computation, sparse tokens undergo channel pruning to reduce computational costs, and intermediate tokens are processed through both branches. The framework further employs branch-specific quantization transformations to improve low-bit accuracy. Extensive experiments on Qwen2.5-VL, Qwen3-VL, and LLaVA demonstrate strong performance under W4A4 and W3A3 quantization, substantially outperforming existing baselines in ultra-low-bit settings while reducing computational overhead.
Critical strengths
- The paper identifies meaningful intra-modality heterogeneity in VLM activation distributions and proposes a principled approach to grouping tokens based on their channel-wise energy concentration. By combining token-specific routing with branch-specific quantization and selective channel pruning, TRPQuant addresses both quantization accuracy and computational redundancy within a unified framework. This is a key strength because it provides a clear motivation for moving beyond conventional modality-aware quantization, supported by activation analyses and component ablations.
- TRPQuant demonstrates substantial improvements over the evaluated baselines, particularly under W3A3 quantization, across six VLMs from three model families. The ablation studies further support the effectiveness of token rearrangement and show that channel pruning reduces computational costs with minimal accuracy degradation. Importantly, the paper also reports actual GPU inference measurements rather than relying solely on theoretical Bit-FLOPs. These results provide compelling evidence of the framework's practical potential and constitute a strong empirical argument for acceptance.
Critical weaknesses
- Although the paper discusses several closely related approaches, including QIG, SplitQ, and Quant Experts, these methods are absent from the experimental comparisons. This is particularly concerning because some of them also exploit token-level characteristics or adaptive quantization strategies, making them directly relevant to TRPQuant's claimed contributions. Without comparisons against these methods, it is difficult to establish whether the proposed framework represents a meaningful advance over the current state of the art, despite its strong results against the selected baselines. This limitation directly affects the assessment of both novelty and empirical significance.
- Although the paper reports substantial theoretical Bit-FLOPs reductions and GPU acceleration, the measured model-level prefill speedup over FlatQuant is relatively modest. For example, on Qwen2.5-VL-7B, TRPQuant improves prefill latency from 84.23 ms (FlatQuant) to 73.72 ms, corresponding to only a 1.14× speedup, while slightly increasing GPU memory consumption. Furthermore, although the paper includes single-kernel benchmarks, its model-level latency evaluation is limited to W4A4 prefill on an RTX 4090, without examining autoregressive decoding, varying batch sizes, or resource-constrained hardware. Since computational efficiency and edge deployment are central motivations of the paper, these limitations weaken the practical significance of the proposed channel pruning and token routing mechanisms.
Questions for authors
- The paper discusses recent VLM quantization methods such as QIG, SplitQ, and Quant Experts but does not include them in the experiments. Could the authors provide direct comparisons with these methods under consistent quantization settings, particularly W3A3? If such comparisons are infeasible, please clarify the reasons and explain what advantages TRPQuant offers over these closely related approaches.
- While TRPQuant substantially reduces theoretical Bit-FLOPs, its measured prefill speedup over FlatQuant is relatively modest (1.14×). Could the authors provide a breakdown of the computational overhead introduced by token classification, rearrangement, and dual-branch processing? Additionally, how does TRPQuant perform during autoregressive decoding, where the small number of tokens processed per step may limit the benefits of token rearrangement and channel pruning?
- TRPQuant relies on a fixed layer-wise active-channel set identified from calibration data. While the paper demonstrates stability across different calibration set sizes, it remains unclear whether this set generalizes to unseen activation distributions. Could the authors evaluate the method under calibration–evaluation domain shifts or analyze how consistently the identified active channels capture the dominant activation energy of unseen sparse tokens? This would help establish whether the proposed token routing and pruning strategy is robust beyond the calibration distribution.