- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes CHAMP, a post-training mixed-precision framework that hierarchically distributes a global FP8 budget across modules, Transformer layers, and channel-groups using calibration-based sensitivity estimates. It constructs matched weight–activation precision masks and combines module-level allocation, layer-level adjustment, and local–global channel-group selection. The authors compare CHAMP with FGMP on Llama-3.1-8B, Wan 2.2, and HunyuanVideo, reporting improved LLM accuracy at low FP8 budgets and better video reconstruction fidelity relative to BF16. Ablations support the contributions of the allocation components. The efficiency results are based on the compute-weighted MXFP8 ratio and an idealized throughput model, rather than measurements with actual low-precision kernels.
Critical strengths
-
The structured allocation of a limited high-precision budget is supported by concrete diagnostics. The oracle budget-swap diagnostics in the appendix show that allocations induced by global sensitivity ranking can be improved without changing the total budget. The leave-one-component-out ablation in Table 12 further supports module-level allocation, layer-level adjustment, and local–global selection. This provides stronger motivation than an allocation heuristic alone.
-
There are identifiable quality gains in the low-budget regime. At r = 0.1, CHAMP improves average accuracy over FGMP by 1.07 percentage points, while video fidelity improves on two backbones. These results suggest practical potential when the high-precision budget is particularly constrained.
Critical weaknesses
-
Deployment efficiency remains unvalidated, and theoretical throughput does not establish a practical advantage. Section 5.3 and Appendix B.4 explicitly use fake quantization, dequantizing tensors before computation. The throughput model assumes that halving bit-width doubles arithmetic throughput and omits the costs of channel-group execution, memory access, precision conversion, and other operators. The idealized gains over FGMP in Table 4 are already small—for example, from 3.10T to 3.20T on Llama—and actual execution overhead could change the conclusion. The current evidence therefore primarily supports precision-allocation quality rather than a demonstrated deployment acceleration.
-
The generalization and statistical evidence for the quality gains are limited. The LLM evaluation covers only one 8B model. All quantitative and qualitative video results use eight Sora demonstration prompts and a fixed random seed of 997. At r = 0.2, the LLM improvement is only 0.44 percentage points, and the average VBench score on Wan is slightly below FGMP. Greater pixel-level similarity to BF16 does not directly establish better video generation quality. These results are insufficient to support broad advantages across models and workloads.
Questions for authors
-
Deployment verification: Could you compare CHAMP and FGMP using at least one real kernel supporting the target numerical formats, reporting throughput, latency, peak GPU memory, and precision-conversion overhead? If this is not currently feasible, should the main claim be explicitly limited to allocation quality under a precision budget?
-
Generalization verification: Do the main gains persist on a larger, independent video prompt set, across multiple random seeds, and on another LLM architecture? What are the confidence intervals for the differences observed with the eight-prompt evaluation?
-
Mechanism clarification: After local–global channel-group selection redistributes the remaining budget across layers, how much of the original layer-level allocation is preserved in the final mask? Could you report the realized FP8 ratio, the compute-weighted MXFP8 ratio, and a controlled comparison against global ranking at the same granularity?
We thank the reviewer for the careful assessment and for recognizing the experimental motivation behind our method, the value of our ablations, and the improvements of CHAMP. We give our response as follows:
Deployment verification and actual inference efficiency We agree that the theoretical throughput estimates should not be interpreted as demonstrated end-to-end inference acceleration. However, the theoretical analysis can serve a useful purpose: it can provide a consistent basis for comparing the compute-weighted high-precision budgets of different methods. For example, when r=0.2 on Llama-3.1-8B, CHAMP and FGMP achieves r^=0.25 and r^=0.29, respectively, corresponding to a better theoretical throughput estimates. These results show that our method can r towards a different mix of computationally costly and cheap operations while maintaining improved model accuracy. However, understanding the theoretical benefits requires implementing specialized hardware kernels that would involve substantial engineering effort beyond the current scope of the work.
We will strengthen this clarification in the abstract, experimental discussion, and limitations section of the final manuscript.
Generalization, additional prompts, and statistical confidence we would like to highlight that our evaluation spans 7 language-understanding benchmarks (with Llama-3.1-8B), and two different video generation models (Wan 2.2-T2V-A14B and HunyuanVideo). We believe that these evaluations show the suitability of CHAMP in both autoregressive language models and diffusion-based video generation models.
We agree that improved frame-level similarity to BF16 does not automatically imply improved perceptual generation quality. These metrics measure different aspects of performance, and we intentionally report both. Regarding VBench, we would like to clarify that some of its dimensions have limited sensitivity. For example, motion smoothness scores are consistently close to 99, leaving little room to distinguish between methods. Therefore, we argue that small VBench differences should be interpreted cautiously, as they do not necessarily indicate meaningful differences. This, indeed, motivates using frame-level fidelity metrics to directly evaluate how our method compares against baselines in preserving BF16 outputs.
Preservation of layer-level allocation after local–global selection We thank the reviewer for highlighting an important aspect of the allocation mechanism. We clarify that for a layer initially allocated K FP8 channel-groups, the local selection stage reserves [K/2] groups for the most sensitive channel-groups within that layer. The remaining budget is then pooled across layers belonging to the same module and redistributed according to the channel-group sensitivity. This design choice (1) preserves a minimum local allocation, and (2) allows the rest of the budget to prioritize more sensitive channel-groups across layers of that module. Thus, the original layer-level allocations are not strictly preserved. Furthermore, Table 12 shows the value of combining local and global selection: comparing with full CHAMP, only local selection reduces accuracy by 0.27%, and using global selection reduces it by 0.50%.
Regarding realized FP8 ratios, Table 4 already shows the compute-weighted MXFP8 ratios used in our theoretical efficiency analysis. Similarly, our oracle experiments in Appendix B.5 investigate global ranking at the channel-group granularity and demonstrate that its allocation can be improved through even small budget reallocation.