- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CARE-Attn analyzes residual FP4 attention errors after QAT and applies separate corrections to QK, probabilities P, and values V. It reports improved VBench quality for Wan2.1, lower perplexity on four language models, and reduced training overhead from removing auxiliary probability accumulation.
Critical strengths
The component-wise analysis motivates distinct corrections, and the paper includes ablations, comparisons against the released Attn-QAT checkpoint and after-training corrections, plus training and attention-kernel throughput measurements.
Critical weaknesses
-
P-exact QAT leaves probabilities unquantized during training but applies two-level FP4 quantization at inference. This deliberate train–test mismatch is supported by the reported results, but the paper needs to distinguish gains from P-exact training from gains due to changing the inference probability quantizer.
-
The full method's efficiency evidence is split between training-step time and attention-kernel throughput; it does not report end-to-end video-generation latency including online tile selection. The evidence therefore supports low kernel-level overhead more directly than faster generation.
Questions for authors
- In the main comparison and ablations, do Attn-QAT and CARE-Attn use the same two-level P quantizer at inference, with P-exact training as the only difference? Please identify the exact baseline configuration.
- Does the reported attention throughput include CARE-Attn's online tile scoring and selection, or only the attention kernel after the correction mask has been generated?
4 comments
Response to Reviewer FmV3 (2/2)
Q2 / W2. Online selection and end-to-end efficiency
We confirm that the throughput measurements in Fig. 4(d) have already included the complete CARE-Attn operator, with online tile scoring, top-k selection, FP4 quantization, sparse BF16 recomputation, and result merging. No selection masks or indices are precomputed. The original benchmark was conducted at 24k tokens with a uniform 1% QK correction budget.
To further assess deployment efficiency, we measured the complete operator at Wan2.1-14B's deployed input shape (1280×720, 81 frames, 75,600 tokens) using the same uniform 1% budget as Fig. 4(d). Including online selection, CARE-Attn retains 94.7%, 94.5%, and 94.4% of Attn-QAT's attention throughput at SP=1, 4, and 8, respectively, and matches SageAttention3.
We also measured the complete video-generation latency, including text encoding, 50 denoising steps with classifier-free guidance, online selection, and VAE decoding.
Table R3. End-to-end Wan2.1-14B generation latency (s/video; lower is better).
| Method | SP=4 | SP=8 |
|---|---|---|
| BF16 (FlashAttention-2) | 1013.8 | 515.0 |
| Attn-QAT | 775.5 | 395.7 |
| CARE-Attn | 795.9 | 407.5 |
All measurements use one eight-GPU RTX PRO 5000 node. SP=4 runs two concurrent requests, with per-request latency reported, while SP=8 runs one request.
CARE-Attn adds only 2.6%–3.0% end-to-end latency relative to Attn-QAT, while maintaining a 1.26–1.27× speedup over BF16. These results confirm that online selection is included in our efficiency measurements and that the additional correction cost remains modest in the complete video-generation pipeline.
We have clarified the benchmark configuration in Fig. 4(d) and added the deployment-resolution operator measurements and end-to-end latency results to Table 16 in the revised PDF, with the main findings summarized in Sec. 5.4.
We sincerely thank the reviewer for the careful evaluation and constructive feedback. We hope the new controlled experiments have clarified the respective contributions of P-exact training and two-level P inference, while the end-to-end measurements have addressed the concerns about online selection and practical efficiency. We would be grateful if the reviewer would reconsider the overall assessment in light of these additional results.
Thank the authors for the additional experiments, which resolve my questions on attribution and online tile selection. However, against the reproduced Attn-QAT baseline, the VBench gain is smaller than the gap between the released and reproduced baselines themselves, so much of the reported improvement reflects the training recipe rather than the method. Moreover, training with two-level fake-quantized P matches P-exact QAT in quality, so P-exact mainly reduces training cost; I therefore maintain my original score.
We thank the reviewer for the follow-up and for acknowledging that our additional experiments resolve the attribution and online-selection concerns.
We would like to clarify that both observations regarding the baseline recipe difference and the comparable quality of two-level P fake-quantized training were already reported in our original submission (Table 11 in Appendix B.3 and Sec. 5.3, respectively).
Importantly, we reproduced Attn-QAT precisely to ensure a reliable and controlled comparison. Since the exact training data and conditions underlying the released checkpoint of Attn-QAT were not fully available to us, comparing against it alone could conflate differences in the training recipe with the contributions of our method. We therefore trained Attn-QAT using the same data, training recipe, GPU count and topology, and batch configuration as CARE-Attn, enabling a more controlled comparison. Under these matched conditions, CARE-Attn still improves weighted VBench from 0.8435 to 0.8472. Although this gain is more modest than the comparison against the released checkpoint, it is the more credible estimate of the improvement attributable to CARE-Attn, precisely because the major training-related confounding factors have been controlled.
We would also appreciate the reviewer considering the complementary fidelity results in Table 12 of the original submission. Against the reproduced Attn-QAT baseline, CARE-Attn improves PSNR from 12.60 to 15.76 dB, SSIM from 0.4913 to 0.6162, and LPIPS from 0.4807 to 0.3147. These matched-baseline results provide further evidence that the benefits of our method extend beyond what is captured by the weighted VBench score alone.
We agree that two-level P fake-quantized training achieves comparable quality to P-exact training, as already discussed in Sec. 5.3. The advantage of P-exact training is its lower training cost while maintaining comparable quality.
We appreciate the reviewer's careful consideration and feedback.
Response to Reviewer FmV3 (1/2)
We thank the reviewer for identifying two important ambiguities. We now separate P-exact training from two-level P inference with controlled experiments, and report end-to-end generation latency including online tile selection.
Q1 / W1. Baseline configuration and attribution
The reviewer is correct that Attn-QAT and CARE-Attn use different P quantizers during inference. Attn-QAT trains and infers with single-level P quantization, whereas CARE-Attn uses P-exact training followed by two-level P quantization at inference. Consequently, the original “P-exact QAT” row in Table 3 changed both the training procedure and the inference quantizer. To isolate their contributions, we conducted controlled experiments on Wan2.1-14B, Qwen3-8B, and Llama-3.1-8B.
Table R1. Effect of P training and inference quantization on Wan2.1-14B. Each entry reports weighted VBench / PSNR (dB) over the same 326 prompts and sampling seed. PSNR is measured against the BF16 fine-tuned reference. All configurations exclude V-centering and QK correction.
Using the reproduced Attn-QAT checkpoint as a controlled baseline, switching to two-level P inference alone barely changes PSNR (12.60 to 12.61 dB). In contrast, P-exact training improves PSNR by 0.72 dB under single-level inference and by 1.93 dB under two-level inference. These results show that the improvement cannot be attributed to the inference quantizer alone. Rather, P-exact training contributes independently, and its benefit is greater when paired with two-level inference.
We observe the same interaction on language models, where we additionally trained P-exact-only checkpoints without training-time QK correction. All results below use the same three-tile QK correction at inference, including the Attn-QAT checkpoints, so the training configurations can be compared under identical inference settings.
Table R2. Controlled P-pathway ablation on LLMs (C4 perplexity at 8,192 tokens; lower is better).
Simply applying two-level inference to Attn-QAT weights increases perplexity on both models, whereas it reduces perplexity for P-exact weights. With inference held fixed at two-level P, P-exact training improves perplexity from 11.4216 to 11.3401 on Qwen3-8B and from 9.4549 to 9.3371 on Llama-3.1-8B. The Llama result is reproduced with a second independent training seed (9.4602 to 9.3389). This confirms that P-exact training and two-level inference are complementary, rather than either being sufficient on its own.
We also evaluated training directly with two-level P fake quantization. This achieves comparable video quality to P-exact training (VBench 0.8460 vs. 0.8464) and slightly higher PSNR (14.87 vs. 14.53 dB), but increases training time by 28% (93.4 vs. 72.8 s/step) due to probability fake quantization and auxiliary output accumulation. P-exact therefore provides a more favorable quality–training-cost trade-off, without requiring the more expensive training path.
We have clarified the baseline configurations and renamed the original Table 3 row to “P-exact training + two-level P inference” in the revised PDF. The controlled LLM results are included in Table 7 (Appendix B.2), and the additional Wan2.1-14B comparisons are reported in Tables 11–12 (Appendix B.3).