- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes JustQuant, a quantization framework based on Theseus QAD, which progressively expands knowledge distillation from local blocks to the full model. The method aims to enable W4A4 and W1.58A4 quantization without additional inference operators such as smoothing, SVD, or rotation. Experiments on DiT, FLUX, and diffusion language models demonstrate improvements over conventional QAT/QAD and competitive generation quality.
Critical strengths
- Simple and deployment-friendly quantization. The method avoids additional inference-time transformations, preserving a relatively simple low-bit computation graph. This is practically valuable for efficient hardware deployment.
- Promising empirical results. The experiments demonstrate substantial improvements over direct QAT/QAD, particularly under aggressive W4A4 and W1.58A4 settings. The extensive ablations on distillation granularity and training schedules also provide useful evidence for the effectiveness of the proposed approach.
Critical weaknesses
- Limited novelty specific to activation quantization. The core method is essentially progressive block-wise knowledge distillation, without an explicit design tailored to 4-bit activations or the unique characteristics of DiT/dLLM. Similar improvements are also observed for weight-only quantization (W1.58A32), raising questions about what makes the proposed approach particularly relevant to activation quantization.
- Insufficient validation of the claimed generality. Despite presenting a seemingly general framework for 4-bit activation quantization, the experiments are restricted to DiT and diffusion language models. The absence of mainstream autoregressive LLM evaluations makes it difficult to assess the method's general applicability and competitiveness against established quantization approaches, particularly strong rotation-based methods.
Questions for authors
- What specifically motivates the design of Theseus QAD for 4-bit activation quantization or diffusion-based models? Can the authors provide stronger theoretical or empirical evidence connecting the proposed progressive distillation strategy to the distinctive quantization challenges of DiT/dLLM, rather than generic low-bit optimization?
- Can the authors provide W4A4 experiments on mainstream autoregressive LLMs (e.g., Llama or Qwen), preferably comparing against competitive QAT/QAD and rotation-based baselines? Such evidence would substantially strengthen the paper's generality claims and could positively influence my assessment.