- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
EdgeRazor combines structural mixed-precision quantization with feature- and logit-level distillation for low-bit LLM deployment. It reports strong sub-2-bit results on 0.35–1.7B models and demonstrates a compact Qwen3-0.6B deployment on an Apple M4 Pro. However, its scalability and computational efficiency (the central premises of a lightweight LLM compression method) are not adequately demonstrated.
Critical strengths
- Strong low-bit results on small edge models. EdgeRazor achieves substantial accuracy improvements over existing low-bit baselines on <2B models. The deployment of a 0.19GB Qwen3-0.6B model through llama.cpp also demonstrates practical benefits in a highly resource-constrained setting.
Critical weaknesses
-
Scalability beyond small models is unsubstantiated. All sub-3-bit text-model experiments are limited to MobileLLM-350M, Qwen3-0.6B, and Qwen3-1.7B. The only 7B experiment uses 4-bit weights on a multimodal model. EdgeRazor additionally requires full-teacher feature and logit distillation over multiple training epochs, making its feasibility for standard 7B–70B text LLMs unclear. The current evidence supports small task-adapted edge models rather than general-purpose LLM compression.
-
The lightweight and efficiency claims lack transparent, matched-budget evidence. The models are trained on eight A100-80GB GPUs, but the paper does not report important details such as total training time (in wall-clock and/or GPU-hours), peak training memory, teacher inference cost, data-generation cost, etc. The efficiency claim relies primarily on the number of training tokens used for MobileLLM-350M, although feature- and logit-level distillation can have substantially higher per-token costs than competing methods. It is thus unclear whether EdgeRazor is Pareto-optimal in model quality versus total compression cost under matched training conditions. Therefore, an efficient compression method should be evaluated for both the cost of producing the compressed model and the efficiency of deploying it, rather than only the latter.
Questions for authors
-
Larger-model scalability. Can the authors demonstrate sub-3-bit quantization on standard 7B-70B text LLM settings and report how training memory and computation scale with model size?
-
End-to-end compression cost. What are the wall-clock time, total GPU-hours, peak memory, training-token count, and teacher or data-generation costs for each model and bit width (Please provide all details of the setting and results for transparency and reproducibility)? How does the resulting quality-cost Pareto frontier compare with EfficientQAT, ParetoQ, BitDistiller, and more recent methods (if they exist) under matched (training budget and output model) conditions?
-
Compression versus task-specific training. How much of the reported improvement comes from the quantization method rather than supervised task adaptation? Could the authors compare against a BF16 model and the main QAT/QAD baselines trained with the same data, teacher signals, and compute budget?
My rating is 1 because results limited to models of at most 1.7B parameters do not establish a practically useful LLM quantization method. A scalable method should (at least) handle standard 7B–70B models at a clearly reported and usable cost: EfficientQAT, for example, quantizes 70B models in tens of GPU-hours, whereas EdgeRazor applies extensive teacher-feature and logit distillation only to small text models and does not report its GPU-hour cost at all. This fails the minimum evidence needed to support the paper’s central claim of lightweight and scalable LLM quantization.