- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduces QuIL, an entropy-based measure of posterior structure removed by quantization, and extends BitVI to floating-point representations. It evaluates whether this measure predicts precision sensitivity in Bayesian neural networks and pretrained transformers.
Critical strengths
- The mass-preserving coarsening formulation enables exact entropy calculations for the fitted posterior across multiple precisions without refitting.
- The evaluation separates posterior-approximation errors in a tractable example and includes deterministic quantization comparisons, with strong sensitivity rankings at three bits.
Critical weaknesses
-
Appendix A.1 identifies the loss with the conditional entropy of discarded bits, whereas QuIL measures the differential-entropy increase from within-cell uniformization. For equal-width cells, a conditionally uniform discarded suffix has maximal conditional entropy but zero QuIL. The paper does not reconcile these different information quantities, weakening the central theoretical interpretation.
-
In Table A9, its four-bit attention correlation with accuracy change is +0.57, reversing the expected negative direction, while empirical-Fisher trace reaches -0.78. Its five-bit block correlation is also nearly zero. These results do not support the claim that QuIL is the most consistent sensitivity predictor.
-
At two bits and correlation 0.9, Table A10 reports a mean-field error of 0.278 ± 0.250 nats, substantially above the approximately 0.10 stated in the main text. The error increases with aggressive coarsening and posterior dependence, so the reported cancellation is not reliably established in this critical regime.
-
Mean BoolQ accuracy declines around six bits before the pronounced entropy rise, although seed variability is large. On RESISC, two-bit coarsening improves NLPD and ECE relative to ten bits while reducing accuracy. With the selection threshold remaining hypothetical, these results do not establish a reliable mapping from QuIL to the onset or type of predictive harm.
Questions for authors
See weakness