- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes DropIS, a data-free quantization DFQ method for ViTs. Instead of synthesizing calibration images against a single deterministic full-precision teacher, it optimizes them over a family of structurally perturbed teachers (attention-head dropout and block-wise spatial token dropout), with SSIM/entropy attention regularizers borrowed from MimiQ.
Critical strengths
I think it is a plug-in idea with consistent gains across backbones and bit-widths. Even K=1 gives about 4 points at 1.009 times total cost, which is the strongest evidence. The broad empirical coverage, ie. component, probe-sampling, K, r, lamda, sample-count, 2-bit and dense-task studies.
Critical weaknesses
It seems the main multi-probe gain is confounded with compute. K=4 costs about 5 times synthesis time (3.34 times total), yet no compute-matched baseline is given. In Fig 1b, the motivating margin-variance evidence is measured under the same dropout probes DropIS optimizes, so it is circular. Moreover, it looks like prop 2.1 is a triangle inequality that never shows DropIS reduces the sensitivity term? Also the prop 2.2's lower bound vanishes at large margins?
Questions for authors
how were the hyper parameters chosen in a data-free setting, expecially p_h, p_s, r, lamda? Why does the Deterministic row in Table 2c (31.32) differ from reproduced MimiQ (27.39) on DeiT-S 3w3a? How do r=3 and p_s=0.2 relate, and how are zeroed heads handled in head-wise SSIM? It would be good if the author can report the seeds and variances for each experiment results.
Thank you for the clarification.
The reviewer appreciates the authors' efforts to clarify my concerns. The reviewer maintains "3: weak accept" score for the value of simplicity of the method and theoretical justification as well as empirical gains.