acceptodds
Under review as a conference paper at ICLR 2027

Partial Conjunction Conformal Selection

Abstract

Conformal selection (CS) provides finite-sample guarantees for False Discovery Rate (FDR) control when selecting a subset of high-quality candidates from a large pool. Existing multivariate conformal selection (MCS) methods are restricted to the “all-or-nothing” rule (i.e., samples are selected only if all criteria are satisfied) and do not address the practical need for -out-of- conjunction rules. In this study, we show that directly applying unadjusted partial conjunction combinations to componentwise conformal -values can fail to preserve FDR control in the CS setting. To address this, we propose Partial Conjunction Conformal Selection (PCCS), a unified framework that accommodates both the global -out-of- and block-level -out-of- conjunction rules. In particular, we introduce two types of novel nonconformity scores to construct conformal -values and then run the Benjamini–Hochberg (BH) procedure, thereby providing theoretical guarantees for finite-sample FDR control. Simulated and real-world experiments demonstrate that PCCS maintains valid FDR control while substantially improving selection power, establishing a versatile framework for selection tasks such as drug property prediction, LLM response filtering, and monocular depth estimation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Uncertainty Quantification, Conformal Inference, False Discovery Control

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_62fyOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper considers conformal selection under partial conjunction targets, where a candidate is considered desirable if at least r of d conditions are satisfied. The proposed method combines a binary indicator of target satisfaction with order statistics of predicted signed margins to construct a clipped nonconformity score, followed by conformal calibration and the BH procedure. The authors establish finite-sample FDR control under conditional exchangeability and characterize asymptotic FDR and power. The authors evaluate the method against baselines through simulations and real data applications.

Critical strengths

The design of scores is tailored to the specific partial conjunction selection problems it considers. The global construction uses the r-th largest predicted signed margin to reflect the requirement that at least r conditions are satisfied, while the block-level construction uses nested order statistics to reflect requirements within and across groups. These choices provide an interpretable way to translate the selection targets into scalar scores that can be used within the existing conformal selection framework.

Critical weaknesses

The main weakness is the limited methodological and theoretical contribution relative to existing methods. The global and block-level partial conjunction rules define particular target regions R, which already fit the general formulation of mCS (Bai et al., 2025). The paper’s claim that these targets are beyond the scope of existing mCS is not clearly justified.

More importantly, the problems considered in both mCS and this paper can be reduced to the setting of Jin and Candès (2023) by defining the binary response Y=IY∈RY=\mathbb{I} \\{ {\bf Y}\in R \\} with appropriately defined RR. Under this transformation, the proposed score is exactly the clipped score in Jin and Candès (2023, Eq. (9)), with a particular choice of prediction score. The finite-sample FDR guarantees and asymptotic FDR and power characterizations are essentially specializations of the existing results under the corresponding assumptions. The main remaining contribution is the use of order statistics of predicted signed margins to construct scores for the specified targets. This is a reasonable and interpretable design, but I do not see a substantial methodological or theoretical contribution beyond this problem-specific score choice.

[1] Bai, T., Zhao, Y., Yu, X., and Yang, A. Y. (2025). Multivariate Conformal Selection. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:2535–2559.

[2] Jin, Y., and Candès, E. J. (2023). Selection by Prediction with Conformal p-values. Journal of Machine Learning Research, 24(244):1–41.

Questions for authors
  1. The main question is whether the problem can be extended to a setting that requires new methodology beyond the current score construction. I would be interested in a natural example that is not directly covered by existing conformal selection methods through a change of target indicator and prediction score. What additional statistical challenges would arise in such a setting? Given the relatively narrow scope of the conformal selection problem, a clear methodological contribution is important to justify a standalone paper.

  2. In the experiments, BinCS is trained on the binary target indicator, while PCCS uses predictions of the original responses. How much does this difference in training contribute to the power gain? A comparison of different aggregation scores using the same fitted multivariate predictor would be helpful.

  3. The ADMET evaluation uses imputed labels. How does this affect the interpretation of the reported FDR and power?

1 comment

Thank you for the detailed response. I have raised my score to 3.

Reviewer_g7xqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper purposes Partial Conjunction Conformal Selection (PCCS), a method that extends multivariate conformal selection from full conjunction requirements to more flexible partial conjunction rules, where a selected candidate should satisfy at least r out of d criteria. A more complicated case of block-level conjunction rules is also considered. The purposed method constructs conjunction-based nonconformity scores using the order statistics of coordinate-wise positive margins, followed by the standard conformal p-value and BH construction. The paper gives finite-sample FDR control under exchangeability, and also characterize the asymptotic FDR and power. Experiments on synthetic datasets and several real-world applications are presented, demonstrating FDR control and improved selection power over the baselines.

Critical strengths
  • The problem is practically relevant. Partial conjunctions provide a natural relaxation of the full conjunction criteria.
  • The paper is generally clearly written.
  • The proposed method shows empirical improvements over binarized CS and partial conjunction testing baselines.
Critical weaknesses
  • The paper claims that the existing multivariate CS methods mainly consider the full conjunction case. However, the mCS work by Bai et al. already allows general target regions, including partial conjunctions. The proposed method mainly introduces a specific choice of nonconformity score, rather than a fundamentally new approach.
  • Related to the first point, the paper should also directly compare against the original mCS methods, applied to the same settings.
  • The paper would benefit from a brief explanation of how the Bonferroni and omnibus baselines combine and adjust the coordinate-wise conformal p-values, particularly to help understand their different FDR behaviors across conjunction thresholds.
Questions for authors

Is the proposed order-statistic score closely related to a distance-based mCS score applied to the same partial conjunction region? Is there a theoretical or computational advantage that distinguishes the proposed method from a direct application of mCS?

Reviewer_rqi6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends conformal selection (CS) from the univariate and “all-criteria” multivariate (MCS) settings to partial conjunction targets: a global rr-out-of-dd rule and a block-level r0r_0-out-of-LL rule. It shows that standard partial-conjunction combinations of componentwise conformal pp-values can violate FDR control. It then proposes PCCS, which applies BH to conformal pp-values built from the clipped score M⋅1{target holds}−D[r]↓(x)M\cdot\mathbb{1}\{\text{target holds}\} - D^{\downarrow}_{[r]}(x), where D[r]↓(x)D^{\downarrow}_{[r]}(x) is the rr-th largest signed margin of the coordinate predictions. Finite-sample FDR control is established, and experiments cover synthetic data, ADMET, HelpSteer2, and NYU Depth V2.

Critical strengths
  1. The problem is well motivated. Partial and block-structured (hard/soft) conjunction rules arise naturally in compound screening and data curation, and existing MCS methods cannot handle them.

  2. The method is simple and correct. It requires no training beyond the coordinate-wise predictors, applies to any coordinate-wise region with a signed margin, and has finite-sample FDR guarantees.

  3. The evaluation is broad: four data-generating processes with Gaussian and tt noise, dimensions d∈{5,10,30}d \in \{5, 10, 30\}, four region types, several nominal levels, and three real-data domains.

Critical weaknesses
  1. Limited methodological novelty. Written as the binary event {r∗(y)≥r}\{r^*(y) \ge r\}, PCCS is essentially the clipped-score cfBH of Jin & Candès (2023), and the FDR proofs largely carry over from that work. The new ingredient, the ranking statistic D[r]↓(x)D^{\downarrow}_{[r]}(x), is heuristic and comes without a power analysis.

  2. Weak baselines. PCCS and BinCS apply cfBH to the same binary indicator and differ essentially only in the ranking statistic. Yet BinCS fits "the same model family" (A.3) to a 0/10/1 target, which likely explains its near-zero power for union and complex regions (Tables 16–17) and for interval regions at r≥3r \ge 3 (Table 15). Comparisons with a flexible classifier and with existing MCS methods at r=dr = d are missing.

  3. Scale sensitivity. D[r]↓(x)D^{\downarrow}_{[r]}(x) compares raw signed margins across coordinates and ignores predictive uncertainty. This matters for the unstandardized block-level experiments (A.3) and for ADMET, whose responses have heterogeneous units and cutoffs (Table 18).

  4. Limited scope of Proposition 1. For r<dr < d, it analyzes the unadjusted p(r)p_{(r)} rather than the Bonferroni-adjusted pp-value (d−r+1) p(r)(d - r + 1)\,p_{(r)} of Benjamini & Heller (2008). Its r=1r = 1 case reflects only a missing multiplicity correction. The failure in Table 1 (r=5r = 5: FDR 0.7010.701, power 1.0001.000) suggests that componentwise conformal pp-values are capped near the per-coordinate null proportion, but this mechanism is not formalized.

  5. Real-data interpretability. Base rates and selection counts are not reported, so some low-rr ADMET settings may be trivial: at q=0.1q = 0.1, selecting every compound controls FDR if more than 90%90\% satisfy the rule. Power is compared across rr values whose denominators differ (0.9770.977 at r=11r = 11 vs. 0.0120.012 at r=15r = 15; Sections 1 and 6).

Questions for authors
  1. Can you justify the power of D[r]↓(x)D^{\downarrow}_{[r]}(x), for example with an optimality or power-ordering result under a model? How does it compare with scale-normalized margins and with an estimated probability P^(r∗(y)≥r∣x)\hat{P}(r^*(y) \ge r \mid x)?

  2. How does PCCS compare with BinCS using a flexible classifier, such as gradient boosting or an MLP? At r=dr = d, how does it compare with Bai et al. (2025) and Hao et al. (2026)?

  3. Can Proposition 1 be extended to the Bonferroni-adjusted and Omnibus pp-values computed from actual conformal pp-values? For the real data, please report P(r∗(y)≥r)P(r^*(y) \ge r) for each rr, the number of selections and true discoveries, results for a “select all” reference, and the fraction of imputed ADMET labels.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.