- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper considers conformal selection under partial conjunction targets, where a candidate is considered desirable if at least r of d conditions are satisfied. The proposed method combines a binary indicator of target satisfaction with order statistics of predicted signed margins to construct a clipped nonconformity score, followed by conformal calibration and the BH procedure. The authors establish finite-sample FDR control under conditional exchangeability and characterize asymptotic FDR and power. The authors evaluate the method against baselines through simulations and real data applications.
Critical strengths
The design of scores is tailored to the specific partial conjunction selection problems it considers. The global construction uses the r-th largest predicted signed margin to reflect the requirement that at least r conditions are satisfied, while the block-level construction uses nested order statistics to reflect requirements within and across groups. These choices provide an interpretable way to translate the selection targets into scalar scores that can be used within the existing conformal selection framework.
Critical weaknesses
The main weakness is the limited methodological and theoretical contribution relative to existing methods. The global and block-level partial conjunction rules define particular target regions R, which already fit the general formulation of mCS (Bai et al., 2025). The paper’s claim that these targets are beyond the scope of existing mCS is not clearly justified.
More importantly, the problems considered in both mCS and this paper can be reduced to the setting of Jin and Candès (2023) by defining the binary response with appropriately defined . Under this transformation, the proposed score is exactly the clipped score in Jin and Candès (2023, Eq. (9)), with a particular choice of prediction score. The finite-sample FDR guarantees and asymptotic FDR and power characterizations are essentially specializations of the existing results under the corresponding assumptions. The main remaining contribution is the use of order statistics of predicted signed margins to construct scores for the specified targets. This is a reasonable and interpretable design, but I do not see a substantial methodological or theoretical contribution beyond this problem-specific score choice.
[1] Bai, T., Zhao, Y., Yu, X., and Yang, A. Y. (2025). Multivariate Conformal Selection. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:2535–2559.
[2] Jin, Y., and Candès, E. J. (2023). Selection by Prediction with Conformal p-values. Journal of Machine Learning Research, 24(244):1–41.
Questions for authors
-
The main question is whether the problem can be extended to a setting that requires new methodology beyond the current score construction. I would be interested in a natural example that is not directly covered by existing conformal selection methods through a change of target indicator and prediction score. What additional statistical challenges would arise in such a setting? Given the relatively narrow scope of the conformal selection problem, a clear methodological contribution is important to justify a standalone paper.
-
In the experiments, BinCS is trained on the binary target indicator, while PCCS uses predictions of the original responses. How much does this difference in training contribute to the power gain? A comparison of different aggregation scores using the same fitted multivariate predictor would be helpful.
-
The ADMET evaluation uses imputed labels. How does this affect the interpretation of the reported FDR and power?
Thank you for the detailed response. I have raised my score to 3.