- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Vodka, a novel VQ-QAT algorithm via differentiable kernel approximation, to address the tradeoff between expressivity and efficiency. The authors identify the key expressivity gap as the mapping function and construct the Codebook Spectrum by ordering curvature, with affine and arbitrary maps. The method stays search-free and admits an exact codebook gradient at fixed indices. The authors jointly optimize the implicit codebook with the weights and propose Shakernel, a fused GPU kernel that reduces training cost and memory. Evaluations on Qwen3.6-35B-A3B and Llama3.1-8B across seven benchmarks demonstrate improvements over SOTA low-bit methods while maintaining efficiency.
Critical strengths
The proposed Codebook Spectrum provides a principled way to bridge affine quantization and unconstrained vector quantization. It improves codebook expressivity while preserving search-free index assignment.
The joint optimization of codebook parameters and weights enables both to adapt to the task loss. Experiments show substantially faster convergence than fixed-codebook VQ-QAT methods.
The proposed Shakernel significantly reduces the computational and memory overhead when training the more expressive quantizer.
Critical weaknesses
Figure 6 compares a fixed affine codebook at m=0 with trainable nonlinear codebooks at m>0, making it difficult to isolate the benefit of nonlinear codebook.
In figure 5, the accuracy improvements over existing methods are relatively small. Since LC-QAT has not converged, it is unclear whether Vodka maintains its advantage after both methods fully converge.
The inference efficiency is not sufficiently evaluated. No actual GPU inference latency or throughput results are provided.
Questions for authors
Can the authors provide an ablation that independently evaluates the benefits of nonlinear codebook and joint codebook-weight optimization?
How does Vodka compare with LC-QAT when both methods are trained to convergence?
What are the actual inference latency and throughput of Vodka on GPUs compared with existing VQ-QAT methods?