- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper develops a conformal framework for top- selection with false discovery rate (FDR) control. It constructs exact rank-projection weights using auxiliary rank constructions and incorporates them into a BH-type selection procedure, avoiding the need to specify an additional outcome threshold. The paper theoretically characterizes the asymptotic FDR, power, and oracle ranking properties of crBH, while Certified-crBH provides a finite-sample FDR guarantee under joint exchangeability and prespecified parameters.
Critical strengths
-
The paper uses exact rank-projection probabilities as calibration weights to construct a conformal procedure for top- selection with FDR control. Unlike fixed-threshold conformal selection, this addresses an unknown cutoff determined jointly by the test outcomes.
-
The analysis establishes asymptotic FDR and power results, including conditions for attaining oracle power within the specified rule class. Certified-crBH additionally provides a finite-sample FDR guarantee under joint exchangeability.
Critical weaknesses
The main asymptotic analysis in Theorem 5 assumes , and therefore does not cover the sparse-selection regime , including fixed as . Although Certified-crBH provides finite-sample FDR control, its minimum nonempty selection size can severely restrict power when is small and calibration and test sizes are comparable. The paper does not provide a corresponding power characterization in this important regime.
Questions for authors
-
How does crBH differ from conformal FDR control after data-dependent selection [1] and FDR-controlled ranking inference under the BTL model [2]? Which differences in the hypotheses, observed data, and assumptions prevent their direct application here?
-
Appendix C.2 compares the exact weights with a mass-matched scaled-rank step approximation using the same information and selection procedure. Could the asymptotic FDR/power results and the oracle-attainment claim also hold with these simpler weights under the same assumptions? If not, which step of the proof specifically requires the exact projection?
-
Corollary 6 allows randomization at the oracle threshold, and Section 3 uses randomized score tie-breaking. However, the power bound in Appendix B.1 and Appendix K uses the strict sets and , which exclude boundary points. How does the bound account for candidates selected through randomization at these boundaries?
-
Lines 428 and 433 refer to "Appendix 4". Are these references intended to point to Figure 4 in Appendix C.1?
[1] Wang, X., Huo, Y., Peng, L., and Zou, C. Conformalized Multiple Testing after Data-dependent Selection. NeurIPS, 2024.
[2] Liu, Y., Fang, E. X., and Lu, J. Lagrangian Inference for Ranking Problems. Operations Research, 2022.