- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Ternova combines Hessian-aware ternary factorization, sensitivity-based mixed-rank allocation, and task-aligned refinement to compress pretrained LLMs without a dense high-precision residual branch. It evaluates model quality and efficiency across several families, reporting approximately 1.59 bits per quantized linear weight.
Critical strengths
The three stages address plausible sources of compression error within a coherent framework. The evaluation covers several model families, and Table 6 provides a useful storage-matched DBF+PV comparison with improvements on various models.
Critical weaknesses
- DBF (https://arxiv.org/html/2505.11076v4) already uses scaled discrete factors, nonuniform intermediate dimensions, and global refinement. Appendix A.4.4 explains Ternova's differences, Table 6 matches storage, and Tables 3, 4, and 7 provide component evidence. However, matched optimization costs for DBF+PV are not established, and Table 3 does not report each variant's realized storage and compute budget despite the use of a higher-capacity probe. These omissions limit attribution of the improvements to the proposed mechanisms rather than additional capacity or optimization.
- Section 4.4 names the GPU and llama.cpp but does not specify the inference batch size, prompt/generation lengths, prefill/decode distinction, or implementation of the two ternary factors.
- The QAT timing comparison also uses different storage budgets: approximately 1.03–1.04 BPW versus Ternova's 1.59. Table 2 lacks sufficient generation/scoring settings, while HumanEval drops from 78.05 to 49.36, qualifying the broad capability-retention claim.
Questions for authors
- Under matched data, realized storage, and optimization cost, what gains remain over DBF+PV and a direct ternary extension? Please specify the budgets for the existing component ablations.
- What exact implementation and workloads produced Figure 3, and were the timing comparisons measured on comparable hardware with all preprocessing included?
- Can you provide Table 2's generation/scoring settings, per-model packed storage, and a correction or explanation for Table 5's average?