- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies fixed-confidence identification of optimal capacity-constrained treatment allocations using individual-level prognostic covariates. It derives an information-theoretic sample complexity lower bound, shows that observing prognostic covariates can reduce the required sample size, and proposes an adaptive procedure that learns unknown outcome regression functions while maintaining finite-sample error control. The authors claim they may attain the optimal first-order expected stopping time.
Critical strengths
The problem is practically relevant, and the paper provides a coherent connection between constrained experimental design, covariate adjustment, and fixed-confidence best-arm identification. Also, the use of predictable likelihoods and composite-null profiling may offer a principled way to maintain sequential error control while learning nuisance functions.
Critical weaknesses
-
The proposed framework largely combines existing ideas from covariate adjustment, best-arm identification, and sequential inference, without a clearly distinctive methodological contribution.
-
The theoretical guarantees rely on pretty restricted conditions such as Gaussian outcomes and known residual variances and covariate distributions. The deployment objective also requires clearer justification.
-
Experiments seem to be very limited to a single simple synthetic setting with only two feasible allocations, providing little evidence of general applicability. Further, the proposed method outperforms the benchmark only at extremely stringent confidence levels, leaving its advantages in realistic settings unclear.
-
The adaptive design is initialized at its oracle optimum and remains fixed, while comparisons rely on an oracle benchmark rather than practical competing methods.
Questions for authors
See the weakness section above.