- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a lightweight method based on Semantic-Aware Grouping (SAG) for CTR/CVR calibration. In addition, it introduces Logit-Cluster Calibration Error (LCCE) to decouple evaluation from training and align with the reliability term of proper scoring rules. For evaluation, the paper conducts experiments on three real-world datasets, and the proposed method achieves the best performance.
Critical strengths
- This paper proposes VR-SAG, a lightweight post-hoc layer applied on top of a frozen backbone, which is easy to deploy.
- Comprehensive experiments are conducted to validate the effectiveness of VR-SAG, showing that VR-SAG outperforms numerous baselines across various calibration metrics.
Critical weaknesses
- There is substantial content overlap between Lines 54–62 and Lines 63–71.
- The paper claims the method can be applied to CVR prediction, yet it neither describes how the method is adapted for this task nor reports its CVR prediction performance.
- The baselines are somewhat outdated. With the exception of SAG, all other baselines adopted in this work were proposed prior to 2017.
Questions for authors
- How is the stability of LCCE ensured? As LCCE uses k-means for sample partitioning, and the appropriate K varies across different backbones, how can the authors verify that the selected K is well-suited for the backbone adopted in this work?