- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper considers picking one of a fixed set of candidate predictors for a given task from a small sample of that task’s data. Rather than picking the predictor with the lowest sample loss as is, it proposes adding to the loss a learned control variate with zero mean, which can change the preferred predictor on a small sample. The paper develops theory on when such terms help and on learning them from past tasks, and illustrates it on simulated examples.
Critical strengths
The results appear correct.
Critical weaknesses
This paper offers nothing substantially new. It repackages textbook results for the selection rule and presents them as its theorems: a Chernoff bound, a representer theorem, a Jensen counterexample, the generalization bound of [1, Theorem 3.3], an Assouad lower bound and a union bound. The control-variate idea is already used in ranking and selection [2]. The experiments are all toy simulations.
[1] Balcan et al. How Much Data Is Sufficient to Learn High-Performing Algorithms? Generalization Guarantees for Data-Driven Algorithm Design. STOC, 2021.
[2] Tsai and Nelson. Combined Ranking and Selection with Control Variates. Winter Simulation Conference, 2006.
Questions for authors
None.