- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes a sufficient-state framework for efficient nonlinear computing in scaled FP8 pipelines. It develops methods for SwiGLU, RMSNorm, and softmax attention to preserve final FP8 quantization decisions. The methods are evaluated through numerical verification, hardware implementation estimates, and LLM task accuracy experiments.
Critical strengths
- Preserving final FP8 quantization results while reducing intermediate computation costs is an interesting idea.
- The mathematical arguments appear sound.
Critical weaknesses
- The paper is difficult to follow. The introduction is too long, and there is no dedicated related work section. The presentation is overly technical and lacks sufficient intuition. Some figure text is too small, and some table captions appear to lack the required spacing below them.
- The contributions mainly concern numerical computing and hardware architecture, and their significance for machine learning is not sufficiently demonstrated. There are no model accuracy comparisons against conventional FP8 inference baselines or end-to-end inference latency or throughput measurements.
- The supplementary artifact referenced in the Reproducibility Statement does not appear to be available with the submission, limiting reproducibility.
Questions for authors
- Some GPQA-Diamond accuracies in Table 4 (e.g., 81.43% and 91.43%) are inconsistent with the stated evaluation set of 198 questions. These percentages cannot be obtained from an integer number of correct answers. Could the authors clarify the evaluation denominators and provide the raw numbers?
4 comments
W3: Reproducibility and Supplementary Artifact
We will provide an anonymized numerical verification and implementation package in the revised supplementary materials.
Table R6: Reproducibility package.
| Component | Verification material |
|---|---|
| Mathematical certification | Directed-MPFR boundary checks and decision-cell verification |
| State construction | Canonical inputs, G11 target mapping, producer verification |
| State compression | Compact16/Directed15 encoding and suffix-response replay |
| Format portability | E5M2 numerical certification and group-size validation |
| Hardware evaluation | RTL configurations, activity protocols, area/energy accounting |
The package will include execution instructions, environment specifications, and expected verification outputs. We will update the Reproducibility Statement with the anonymous artifact entry point.
Q1: GPQA-Diamond Accuracy Reporting
We confirmed transcription and rounding errors during result registration and recalculated the GPQA-Diamond accuracies from integer correct-answer counts over 198 questions.
Table R7: Corrected GPQA-Diamond results.
| Model | Method | Original (%) | Correct answers | Corrected (%) |
|---|---|---|---|---|
| DeepSeek-V4-Flash-0731 | Native | 89.89 | 178/198 | 89.90 |
| Replacement | 90.90 | 180/198 | 90.91 | |
| Qwen3.5-9B | Native | 81.43 | 161/198 | 81.31 |
| Replacement | 80.30 | 159/198 | 80.30 | |
| Qwen3.8-27B | Native | 90.91 | 180/198 | 90.91 |
| Replacement | 91.43 | 181/198 | 91.41 |
All corrected percentages use , rounded consistently to two decimal places.
We will update Table 4 and Appendix F.1 with the corrected percentages, raw counts, and rounding convention. The updated results preserve the overall finding of comparable task performance between joint replacement and native inference.
Thank you for the reply. Weakness 3 is not resolved because the supplementary material is still missing. The updated manuscript contains substantial edits and I recommend a resubmission to check it properly. I maintain my rating.
Thank you for pointing this out. We have now uploaded the supplementary material to OpenReview. Thank you for taking the time to review our response and the updated materials.
Dear Reviewer Qiiu,
Thank you for your feedback. We expanded model and hardware evaluations, verified cross-format portability, and corrected GPQA-Diamond reporting errors.
W1: Presentation, Technical Intuition, and Related Work
We will organize the manuscript around one question: What information must nonlinear computation preserve to guarantee final FP8 codes and shared scales?
The revised presentation will cover:
We will add a dedicated Related Work section, improve intuitive examples, enlarge figure labels, and fix table-caption spacing.
W2: Machine-Learning Relevance and Conventional Inference Baselines
(1) Model-Level Evaluation with Matched Quantization Interfaces
We evaluated Matched-native, which retains framework arithmetic under the same quantization interfaces as Replacement.
Table R1: Model evaluation under matched quantization interfaces (%).
Replacement achieves higher scores in all nine matched-interface comparisons, demonstrating model-level effectiveness in quantized LLM inference.
(2) Conventional Hardware Baselines
We implemented a matched BF16 LUT baseline for SwiGLU and conventional FP32 endpoints for RMSNorm and attention.
Table R2: Hardware efficiency against conventional implementations.
Table R3: Complete-row hardware and numerical comparison.
Across 1,024 RMSNorm rows (2,621,440 outputs), our implementation achieves zero code/scale mismatches, 33.20% fewer cycles, and 13.83% lower total energy.
For attention, all 31,744 committed FP8 codes and 992 scale bytes match the mathematical reference. Conventional FP32 produces 24 code mismatches across three constructed boundary rows. Our design reduces row energy by 63.22%, using 1.99× cell area.
These results establish final-quantization accuracy and hardware efficiency against conventional finite-precision implementations.
(3) Cross-Format Validation and Practical GPU Implementation
We extended SwiGLU to E5M2, multiple group sizes, and NVFP4.
Table R4: Cross-format numerical validation.
For NVFP4, the optimized Wide-State path reduces code mismatches from 8,296 to 1 and scale mismatches from 1,862 to 0.
Table R5: NVFP4 CUDA endpoint latency on RTX 6000D, N=9,216.
Cooperative data layout and optimized division processing deliver 2.52–7.28× CUDA endpoint speedup.
Together, these experiments demonstrate model-level compatibility, numerical portability, and hardware efficiency. We will incorporate the results in the revised manuscript.