- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Flow-Q, an auxiliary objective for Hessian-based block-wise reconstruction in post-training quantization of Vision Transformers. It aims to preserve full-precision representation quality, which the authors argue sensitivity-weighted reconstruction alone fails to do. Flow-Q builds a sensitivity-weighted Gram matrix from FP feature evolution and takes its top-µ SVD subspace. It then aligns the quantized and FP feature evolutions in this subspace with a direction loss and a magnitude loss. Integrated into APHQ-ViT and LS-ViT, it shows consistent gains on quantized DINOv3 across dense downstream tasks, on ImageNet classification with ViT/DeiT/Swin, and on mobile-efficient models at low bit-widths.
Critical strengths
- A novel perspective on the quality of quantized feature representations. This work raises a question about the quality of the quantized feature representation itself, offering a new perspective to existing quantization research.
- Extensive experiments demonstrating generalizable performance. This work achieves strong and generalizable performance across 9 tasks and 12 benchmarks that use vision transformers as the backbone.
Critical weaknesses
Incomplete description of the method. The paper does not describe how and are specifically used in the reconstruction. By examining the supplementary material, I confirmed that Eq. (5) and Eq. (6) are added to the reconstruction loss of the baseline method into which Flow-Q is integrated (e.g., LS-ViT in the provided code). I suggest explicitly stating this in the Geometric Rectification paragraph of the paper.
Minor Weaknesses (These points do not affect my evaluation of the paper, but are suggested to improve its overall quality.)
-
In the Implementation Details paragraph of Section 4, the paper states: "The learning rates for weight and activation quantization are set to 1e-3 and 4e-5, respectively." - This expression is inaccurate. These are not learning rates for quantization itself, but learning rates used in the reconstruction for optimizing the quantization parameters.
-
In Tables 1 and 2, the evaluation metrics for each task are described in the Evaluation Metrics paragraph rather than in the tables themselves, which reduces readability. If the paper is accepted and additional space becomes available in the camera-ready version, I suggest indicating the metric for each task directly in the tables.
Questions for authors
Q1. Why was Flow-Q not applied to FIMA-Q? Can the authors apply Flow-Q to FIMA-Q as well, to demonstrate its generalizability?
Q2. Section 3.1 states that sensitivity does not reflect feature evolution well, whereas Section 3.2 states that sensitivity indicates the importance of the corresponding evolution information. These two statements seem contradictory to me, though I may have misunderstood. Please clarify how they are consistent.
Q3. From the provided code, it appears that attention quantization is not applied to DINOv3, whereas prior ViT PTQ works typically quantize the attention modules. Is there a reason the authors did not apply attention quantization to DINOv3? Were the compared methods also evaluated under the same quantization setting?
Q4. From the provided code, flow_loss appears to be added to the optimization objective of LS-ViT. Could the authors provide an ablation on the coefficient multiplied by flow_loss to show the sensitivity to this value?
Q5. Could the authors provide an ablation study on the size of the optimization set?
4 comments
Official Comment by Authors (2/2)
Q7: Sensitivity analysis of the loss coefficient.
A7: Thank you for spotting this. We have added an ablation study on the loss coefficient in of the paper. The experimental results show that the model achieves its best performance when is set to 0.01.
| 0.001 | 0.005 | 0.01 | 0.02 | 0.05 | |
|---|---|---|---|---|---|
| Accuracy | 54.41 | 54.77 | 54.89 | 54.81 | 54.60 |
Q8: Ablation study on the size of the optimization set.
A8: Apologies for this missing ablation study. We define the size of the optimization set as and add an ablation study on different optimization set sizes under the W2A3 quantization setting on DeiT-S, together with the corresponding experimental analysis, in . For a fair comparison, we follow the settings of FIMA-Q, APHQ-ViT, and LS-ViT, and set the size of the optimization set to 1024.
| 768 | 896 | 1024 | 1152 | 1280 | |
|---|---|---|---|---|---|
| Baseline | 52.69 | 53.00 | 53.86 | 54.05 | 54.63 |
| + Ours | 53.92 | 54.27 | 54.89 | 55.24 | 55.60 |
Thank you for your valuable time and constructive feedback. We have revised the manuscript with more discussions and experimental results according to your comments. It is welcome if there are any further questions or suggestions.
I have carefully read the authors’ response, and all of my concerns have been addressed. The revised manuscript is more complete than the original submission. I believe this paper deserves acceptance, as it offers a new perspective on post-training quantization. I will maintain my score and wish the authors the best of luck.
Official Comment by Authors
Thank you very much for your acknowledgment of our work. Your careful review has greatly helped improve the quality of our work. We sincerely appreciate your support and encouragement.
Official Comment by Authors (1/2)
We thank reviewer YG9o for the valuable time and constructive feedback. Since the issues in "Critical Weaknesses" and "Questions for Authors" are different, we renumber the questions sequentially from Q1 to Q8, followed by our point-by-point responses below.
Q1: Incomplete description of the method in the Geometric Rectification paragraph.
A1: Thank you for your insightful question. We have added an explanation of how Ldirect and Lmagn are used in reconstruction to the Geometric Rectification paragraph (§3.3). For ease of review, we present it as follows:
Q2: An expression error in the implementation details paragraph.
A2: Thank you for pointing this out. We have corrected this in the implementation details paragraph (§4). For ease of review, we present it as follows:
Q3: Adding the metrics to Tables 1 and 2 improves readability.
A3: We fully agree with your point. The metrics for each task have been added to Tables 1 and 2 to improve readability.
Q4: Applied to FIMA-Q to demonstrate generalization.
A4: Thank you for your question. We integrated our method into the FIMA-Q framework and validated its generalization on DINOv3 and conventional ViTs under specific quantization settings. The experimental results are shown in the table below:
The above experimental results have been added to §4.1 of the revision and highlighted for clarity. Due to the limited time available for the current experiments, we plan to conduct a more comprehensive evaluation in future work by integrating our method into the FIMA-Q framework under various quantization settings on DINOv3, conventional ViTs, and mobile-efficient networks.
Q5: Inaccurate description.
A5: We apologize for the confusion. This was an inaccurate statement made during the writing process. We have corrected it in §3.2 of the paper, with the changes highlighted in blue. For ease of review, we present it as follows:
Q6: Attention quantization and experimental settings.
A6: We thank the reviewer for this reminder. First, we ensure consistent quantization settings across all methods on quantized DINOv3. Second, our public code applies partial attention quantization to the QKV and output projections (weights and input activations), while leaving the Q–K multiplication, softmax, and attention–V multiplication unquantized. The reasons are as follows:
All methods share the same DINOv3 quantization settings for fairness.