- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 3
Read review
Paper summary
As I understand it, the paper is focusing on maximizing the fraction of model predictions that can be certified. It is focusing on the case where several subgroups of data are certified separately, but there might not be sufficient data to certify specific subgroups at a requested error level. It is therefore trying to predict whether subgroups at a specific granularity can be certified with a finite set of data to optimize the overall amount of predictions that can be certified.
Critical strengths
As I understand it, I really like the problem setting of explicitly trying to maximize the expected fraction of deployment traffic, and I think setting up and defining this problem is valuable in itself. The results are also described in a very nuanced way they include some negative results.
Critical weaknesses
===Readability=== I currently find the paper very difficult to follow, which makes it hard to properly judge the actual technical contributions. For example, the abstract in particular (but also the rest of the main text) contains a lot of unexplained jargon and evaluation metrics that are sometimes hard to put into context. The datasets used for evaluation are never motivated or explained in the main text. Overall, I think the paper would benefit from focusing more on readability, intuition-building, and exposition.
===References=== There seem to be a lot of references missing. The evaluation datasets are never referenced in the main text. Neither are used models such as ResNet or Qwen.
Questions for authors
None
Regarding no engagement
Dear Area Chair,
Thank you for your consistent efforts to elicit responses from reviewers to our replies. Despite your reminder and our response to most reviews more than a week ago, we have not received much engagement from a few reviewers. This is discouraging and does not align with the spirit of the rebuttal period. It is particularly concerning for authors when months of work are assessed with a low rating and no active subsequent engagement with our response.
We kindly ask that you take this into consideration when evaluating our work. The lack of follow-up has left us uncertain about the concerns underlying the rating and without a meaningful opportunity to address them.
Thank you again for your efforts to encourage a constructive discussion between authors and reviewers, and for your gentle reminders.
Sincerely, The authors