- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies the performance of Machine Learning models when there are missing labels in examples (X,y) that are considered hard or important to the performance of the model. The authors derive two theoretical results, 1) under a fixed covariates set (X_1,X_2,...,X_n) with an outcome depedent missing labels mechanism they derive the exact fixed-query asymptotic minimax power over adaptive queries, 2) they examine which risk measures have robust performance under label missingness.
Critical strengths
The authors study an interesting model from an applied perspective and develop theoretical results to quantify the importance of each input data point to the model's accuracy.
Critical weaknesses
As I wrote before, the paper is really hard to read and not even the model the authors work on is clear to me. I ask the authors to give a clear explanation of what is the model they are working on so as the reviewer can quantify the importance of their results. I strongly recommend the authors to write the main body from scratch.
Questions for authors
-
At the beginning of section 2) (line 97) "Condition on a frozen predictor" -- what is the inference task, what type of predictor is it?
-
What is the data generating process, are the covariates high dimensional vector arriving iid?
-
In line 107 what is the difinition of a "randomized kernel"?
26 hours remaining until score update is closed
Hello, this is a friendly reminder that we have only 26 hours until the score update is closed. Could you reflect your scores based on the renewed manuscript? Thank you.