acceptodds
Under review as a conference paper at ICLR 2027

Certifying Query-Local Calibration Contamination for Safe-Pair Prediction under Interventions

Abstract

This paper aims to construct reliable prediction intervals for intervention–outcome pairs asserted to be causally unaffected. Although working graphs identify candidate negative controls, graph errors can contaminate query-local calibration pools, making global error corrections insufficient. To address this problem, we propose Certified Graph-Local Conformal Prediction (CGLCP), which couples limited random verification with query-specific conformal calibration. For each fixed safe query, we invert the exact hypergeometric law to upper-bound the selected pool's unsafe count and translate this certificate into a conformal rank. Under safe-score exchangeability and exact independent verification, the procedure guarantees finite-sample marginal coverage and uses the smallest certificate-valid rank against arbitrary contaminating scores. We further quantify the efficiency loss relative to an oracle that knows the local contamination count. Simulations and a K562 Perturb-seq study illustrate the resulting trade-offs among coverage, interval width, finite-output availability, and verification cost, supporting the use of calibrated intervals as negative-control diagnostics under interventions.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Conformal prediction, Causal graphs, Calibration contamination, Finite-sample coverage

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pf4rOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces CGLCP, a method to create reliable prediction intervals for variables that aren't supposed to causally affect each other. Since our assumptions about these relationships are sometimes wrong, the method uses random spot-checks and statistics to estimate the errors and adjust the predictions accordingly.

Critical strengths
  • It tackles a real-world problem where global estimates don't capture local errors.
  • The math is solid and comes with strong theoretical guarantees.
  • The experiments are thorough, especially because they honestly report when the method has to give "infinite" (or unhelpful) intervals just to play it safe.
Critical weaknesses
  • An earlier paper https://openreview.net/forum?id=qdrkePJZCW did something very similar. The authors need to clearly explain why their new math is a major step forward compared to that older work.
  • The mathematical assumptions are pretty strict (like assuming perfect spot-checks) and are hard to guarantee with real-world biology data.
  • When tested on real data (K562), the method gives infinite intervals nearly 40% of the time, which makes you wonder how practical it actually is for daily use.
Questions for authors
  1. How exactly does your math differ from the previous Asiaee paper?
  2. What happens to the method's accuracy if the spot-checks are wrong (specifically, false negatives)?
  3. In practice, when is the extra effort and cost of running these checks actually worth it?
  4. Does the real-world data you tested actually fit your strict mathematical assumptions?
4 comments

We thank Reviewer pf4r for supporting our local-risk motivation and transparent reporting. We have addressed all questions regarding methodological novelty, verification noise, and practical applicability.

Below are our responses to each weakness and question.

W1 & Q1

Corresponding to Sections 1, 2, and 4 (explicit attribution), Section 5 and Algorithm 1 (local certificate and error budget), Appendix F.6, and Tables 1 and 5 in the revised manuscript.

We build upon the sharp oracle contamination inequality of Asiaee et al. (2026, Appendix C), which is explicitly credited in Sections 2 and 4. Our mathematical contribution lies in making this bound operational when safety labels are partially observed rather than fully known.

Specifically, Asiaee et al. require exact safe/unsafe labels for the selected calibration pool. Addressing the setting where labels are hidden and costly to verify, we construct an exact finite-population hypergeometric certificate from a limited random audit, couple it with a split error budget (ρ\rho for certificate validity and β\beta for conformal calibration) to derive the minimal certificate-valid rank (Proposition 1), and establish an exact oracle-relative efficiency decomposition separating certificate-validity cost from over-certification error (Eq. 12). For example, with n=60n=60, certified unsafe count 1212, α=0.10\alpha=0.10, and ρ=0.02\rho=0.02, our certified rank is 57, whereas an additive certified rule exhausts the error budget and returns +∞+\infty (0% finite output vs. 61.61% on K562 under matched information; Appendix F.6).

W2 & Q2

Corresponding to Section 3.3 (Remark 1), Section 6.1 (Verification errors and score placement), Appendix E.6, Table 3, Figure 13, and Section 7 in the revised manuscript (complementing Appendix E.5).

Table R3.1: Impact of Noisy Verification and Unsafe-Score Placement Settings: n=60n=60, true unsafe B=24B=24, nominal target 90%. "Low" places unsafe scores near zero (theoretical worst-case); "High" places them in the right tail (large-residual regime). Finite output relies solely on the audit count, independent of score placement.

Error / scoresAudit ssError RateCertificate failureCoverageFinite
FN; low600%0.00000.92011.0000
FN; low601%0.21950.92011.0000
FN; low6010%0.92270.92011.0000
FN; low6060%1.00000.89521.0000
FN; high6060%1.00001.00001.0000
FP; low3040%0.00000.98030.7049
FP; low3060%0.00000.99570.1474
FP; high3060%0.00001.00000.1474

With unsafe members treated as the positive class, verification false negatives (FN) and false positives (FP) affect the certificate asymmetrically, interacting with the score placement of contaminants (Appendix E.6):

  1. False Positives (FP) are conservatively safe: FP errors inflate B‾\overline{B} regardless of score placement, widening fixed-pool intervals relative to exact verification. As shown in Table R3.1, this preserves nominal coverage under valid safe-score exchangeability (exceeding 98% empirically here) at the expense of finite availability (dropping to 14.74% at 60% FP).

  2. False Negatives (FN) decouple certificate validity from empirical miscoverage: FN errors under-certify the unsafe count, yet empirical coverage degrades much more gradually than the certificate itself:

    • Under worst-case small residuals (FN; low), discrete rank buffering absorbs moderate FN rates: at 1% and 10% FN, certificate failure reaches 21.95% and 92.27%, yet empirical coverage remains intact at 0.9201; undercoverage appears only under severe noise (0.8952 at 60% FN).
    • When missed unsafe pairs exhibit large residuals (FN; high), they sit in the upper tail and push order statistics upward, preserving 1.0000 empirical coverage even at 60% FN.

Together, Remark 1 (giving the degraded lower bound (1−pfail)(1−β)(1-p_{\text{fail}})(1-\beta)) and Appendix E.5 outline how an external upper bound on missed unsafe labels can be incorporated into a noise-aware union-bound certificate, while Table R3.1 clarifies that practical undercoverage requires both substantial FN rates and small contaminating residuals.

W3, Q3 & Q4

Corresponding to Section 3.3, Section 6.2 (Results and sensitivity analyses), Tables 1 and 8, Appendices F.6–F.9, Appendix B, and Section 7 in the revised manuscript.

Table R3.2: Coverage and Finite Output Under Upstream Effect Screening

Calibration settingPooled CP (label-free)GraphLCP (label-free)CGLCP (certified)CGLCP finite output
Original pools90.95%91.09%97.68%61.61%
Effect screening88.97%88.97%96.72%61.70%

We appreciate the reviewer’s practical perspective on assumption fit, infinite intervals, and cost-benefit trade-offs.

1. How K562 fits our assumptions (Q4): We distinguish design-enforced conditions from biological idealizations (Appendix F.9). Non-score pool construction (Assumption 1) and uniform score-independent sampling (Assumption 3's sampling clause) hold by construction via sealed fold splitting and randomized auditing. By contrast, assay-based equivalence intervals (Δ=0.50\Delta=0.50) serve as operational proxies rather than exact causal memberships—with unresolved pairs driving much of the conservative 38.7% abstention rate (optimistic handling raises finite output from 61.61% to 77.55%, Table R1.2)—and cell-state or scale differences can perturb exact safe-score exchangeability (addressed via tail diagnostics in Appendix F.8 and total-variation bounds in Appendix B).

2. When verification checks and abstentions are worth it (W3 & Q3): Infinite intervals in CGLCP act as explicit calibrated abstentions rather than silent failures: returning +∞+\infty signals that local contamination is too high (or pool size too small) to certify a finite 90% interval at the current audit budget, prompting targeted follow-up or a larger global pool (which raises finite output to 68.6%–82.1%, Appendix F.6).

Verification is most valuable when upstream filtering removes obvious large responses, leaving subtle, small-residual contaminants where label-free methods lose their natural conservative cushion. We modeled this on K562 by retaining the bottom 80% of calibration candidates by absolute reference-fold effect magnitude (Table R3.2). Under this screening, label-free Pooled CP and GraphLCP drop below the 90% target in point estimate (to 88.97%; 95% CIs spanning 90%), whereas CGLCP sustains 96.72% coverage and 61.70% finite output using ~14 audited labels per query. In high-stakes perturbation or safety screens where false-safe assurance is costly and upstream QC filters out large effects, this targeted verification cost buys reliable protection and explicit abstention flags when local evidence is insufficient.

Thank you for addressing my questions. The clarification of the contribution beyond Asiaee et al. and the new analyses of verification errors and practical trade-offs strengthen the paper and exact verification and the high rate of infinite intervals remain concerns, I will maintain my rating since it justify the current state of the paper.

I again thank you for your time and good luck to you!!

Thank you for your time and valuable feedback on our work. Your comments and suggestions have provided strong help in further improving our paper and reflecting on its value (including the clarification beyond Asiaeet al., verification errors, and practical trade-offs). We appreciate the concerns you raised regarding exact verification and the high rate of infinite intervals, and we will continue to address them in our future revisions. Your input has been truly beneficial to our project. Thank you again for the time you invested in our work, and good luck with your future research!

Reviewer_eT6jOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Certified Graph-Local Conformal Prediction (CGLCP), a method for predicting outcomes related to intervention-target pairs that are deemed causally safe. It identifies a local calibration pool using a working graph, randomly verifies a subset of safety labels, and transforms a hypergeometric upper bound on contamination into an adjusted conformal rank. The study establishes the coverage guarantee under the conditions of conditional safe-score exchangeability and exact independent verification. Additionally, it evaluates the resulting prediction intervals through simulations and a K562 Perturb-seq study.

Critical strengths

The paper addresses an interesting problem in intervention studies: providing reliable prediction intervals for outcomes that are expected to remain unaffected by an intervention, despite possible errors in the available causal graph structure, while allowing for further verification.

Critical weaknesses
  1. The paper offers limited novelty and makes a small contribution beyond the existing literature. The concepts of safe-pair prediction and false-safe calibration contamination have already been explored by Asiaee et al. (2026). Conformal inference is also a well-established tool. The manuscript fails to convincingly demonstrate any significant methodological or theoretical advancement.
  2. The advantages of the chosen calibration pool and use of verification are not convincingly demonstrated in this paper. The experimental comparisons do not adequately substantiate the benefits of the proposed calibration and label-use strategy. Moreover, the organization of the paper is not intuitive. Many symbols are not clearly defined before being used, and technical assumptions appear before the corresponding procedure and data objects are sufficiently explained.
Questions for authors
  1. The problem formulation requires exchangeability (Assumption 2). This assumption does not seem particularly realistic, and it is not intuitively clear why it should hold in practice. Why should the resulting safe calibration and test residuals be conditionally exchangeable after fixing the query and selecting the local pool? Is there an intuitive explanation for why this should hold?
  2. The paper emphasizes the use of local information. Why not use the full global data as the calibration pool and consider verifying a subset of it? This allows more samples to be used for calibration and should be included as an experimental baseline.
  3. In the setting considered by the paper, verification is allowed with its results assumed to be correct. However, observations confirmed to be unsafe remain in the calibration pool. Given that some samples have already been verified, why not remove those identified as unsafe from calibration, or consider alternative constructions to make better use of this information?
  4. The experiments do not include an ordinary conformal prediction baseline calibrated only on verified-safe samples. This comparison is needed to establish the benefit of incorporating the unverified scores. Moreover, the real-data results are quite conservative, with many infinite prediction intervals, without a clear practical advantage of the proposed method.
4 comments

We appreciate the reviewer-eT6j’s recognition of the problem's relevance. We have thoroughly addressed the requests regarding exchangeability rationale, global vs. local certification costs, manuscript organization, and the use of unverified labels.

Below are our responses to each weakness and question.

W1 (Methodological Contribution)

Corresponding to Sections 1, 2, and 4 (attribution), Section 5 and Algorithm 1 (local certification procedure), and Appendix F.6 in the revised manuscript.

While safe-pair prediction and the sharp oracle contamination penalty originate in Asiaee et al. (2026), their framework assumes that exact descendant memberships are known across the selected calibration pool. Our methodological contribution addresses the realistic regime where safety labels are costly and largely unobserved: we couple finite-population hypergeometric auditing with query-specific rank calibration, establish an end-to-end finite-sample coverage guarantee that jointly accounts for audit-certificate failure (ρ\rho) and conditional miscoverage (β\beta), and derive the exact efficiency decomposition relative to a local oracle (Eq. 12). In Appendix F.6, we isolate this contribution under matched audit information, showing that pairing our local certificate with the sharper rank formula achieves 61.61% finite availability whereas an additive certified rule yields 0%.

Q1

Corresponding to Section 3.3 (Workflow and assumptions), Algorithm 1 in Section 5, Appendix B, Appendix F.8, and Section 7 in the revised manuscript.

The causal rationale rests on the principle that in a correctly specified causal model, intervening on an upstream variable leaves the distribution of any non-descendant target invariant. When paired with a shared measurement regime, homogeneous noise, and uniform score normalization from a frozen predictor, this causal invariance naturally motivates conditional exchangeability among truly safe calibration and test residuals.

The hard radius τ\tau defines a score-blind, discrete finite pool for the hypergeometric audit, providing a structural control over verification scope and local graph error. Because graph proximity alone cannot eliminate biological batch or cell-state shifts (as examined in R1Q5), we state conditional safe-score invariance explicitly as Assumption 2 to separate membership certification from distributional shift, and provide a formal total-variation sensitivity extension in Appendix B to quantify potential coverage degradation under score mismatch.

Q2

Corresponding to Section 6.2 (sensitivity analyses), Appendix D, Appendix F.6, Table 4, and Section 7 in the revised manuscript.

Table R2.1: Verification Costs and Performance: Local vs. Global Pools

MethodVerification CostCoverageFiniteFinite width (log₂FC)
CGLCP (local pool, τ=3\tau=3)Matched count (~24.67 labels)0.97680.61611.1064
Global CGLCP (global pool)30% own pool (~27.45 labels)0.96800.82091.1256
Global CGLCP (global pool)Matched count (~24.67 labels)0.97480.68601.1287

We have added "Global CGLCP" as an experimental baseline to test this directly. Global CGLCP removes the radius restriction, auditing from the entire global working-graph-predicted-safe pool for the target gene.

As Table R2.1 shows, at a fixed 30% verification fraction, Global CGLCP requires a higher per-query verification budget (27.45 vs. 24.67 labels). While the larger pool increases finite availability (0.8209 vs. 0.6161), it raises per-query assay demand. Under strictly matched label counts (24.67 labels per query), Global CGLCP still yields higher finite availability (0.6860 vs. 0.6161) with slightly wider finite intervals (1.1287 vs. 1.1064 log⁡2\log_2FC).

We therefore frame locality (τ=3\tau=3) as a design choice that bounds per-query verification demand, tightens interval precision, and targets concentrated local graph errors (as demonstrated in Figure 1a), while noting that global pooling is an effective alternative when maximizing finite availability is the primary objective.

W2, Q3 & Q4

Corresponding to Section 3 and Section 5 / Algorithm 1 (reorganized presentation), Section 6.2, Tables 1 and 5, Appendix D, and Appendix F.6 in the revised manuscript.

Table R2.2: Value of Unverified Scores and Verified-Unsafe Removal

MethodUse of verificationCoverageFiniteFinite width (log₂FC)
Verified-safe-only CPIdentify verified-safe members; discard all unverified0.9600.4530.752
CGLCP (fixed pool)Bound unsafe count in the whole mixed pool0.9780.6131.061
CGLCP removalRemove verified unsafe; bound remaining unsafe count0.9750.6131.042

We thank the reviewer for suggesting the Verified-safe-only baseline; it directly highlights the benefit of leveraging unverified scores.

As shown in Table R2.2, Verified-safe-only CP (which discards all unverified scores) achieves only a 45.3% finite-output rate, because a small audited safe subset frequently falls below the minimum sample size (9 safe scores) required for a finite 90% conformal quantile.

By contrast, CGLCP uses the random audit to certify a high-probability upper bound on contamination across the entire local pool. Safely retaining the unverified scores boosts finite availability by 16.0 percentage points (to 61.3%) under the exact same verification budget, trading interval width for substantially higher availability.

Regarding Q3: As detailed in R1Q4, removing verified-unsafe observations via (n−X,B‾−X,k−X)(n-X, \overline{B}-X, k-X) is a strictly valid efficiency refinement under exact labels. "CGLCP removal" preserves the 61.3% finite rate while narrowing the finite-width summary from 1.061 to 1.042 log⁡2\log_2FC. We originally retained fixed pools to make the monotonic conservatism of false-positive label errors transparent (Remark 1), and have now incorporated the removal variant into Table 1 and Appendix F.6.

Regarding manuscript organization (W2): We restructured Section 3 so that all data splits, query-specific pools, and audit objects (si⋆,Vi⋆,Xi⋆,B‾i⋆,ρ,βs_i^\star, V_i^\star, X_i^\star, \overline{B}_i^\star, \rho, \beta) are clearly defined alongside the end-to-end workflow (Section 3.3) before stating Assumptions 1–3, and we added a self-contained operational summary in Algorithm 1 (Section 5).

Thank you for the response. However, I still have two main concerns:

  1. The methodological and theoretical novelty remains limited. The paper mainly combines the safe-pair prediction and contamination correction results of Asiaee et al. (2026) with a finite-population contamination bound obtained through random verification.

  2. The experimental results do not sufficiently demonstrate the practical advantages of CGLCP over simpler baselines. CGLCP remains conservative, with a high proportion of infinite prediction intervals, leaving its practical utility unclear.

Therefore, I have decided to maintain my original rating.

Thank you for your further feedback and all your suggestions. They have helped us think more deeply about the value, novelty, and practical effectiveness of our work. We would like to clarify two points:

  1. Our contribution targets a narrower, verification-limited problem. We consider prediction for a prespecified safe query when an imperfect working graph selects a potentially contaminated calibration pool and reliable biological membership labels are costly to obtain. We explicitly credit Asiaee et al. (2026) for the sharp oracle contamination inequality. Our contribution makes this bound operational with partially observed labels: a query-local audit certificate, an end-to-end error-budget allocation, and a minimal certificate-valid rank with an oracle-relative efficiency analysis. This formulation addresses how much reliable prediction can be obtained from limited verification.

  2. The revised K562 experiments identify concrete benefits and their costs. Under upstream effect screening (Appendix F.9, Table 8), Pooled CP and GraphLCP fall below the 90% target in point estimate (88.97%), while CGLCP achieves 96.72% coverage with 61.70% finite output using approximately 14 verification labels per query. The baseline confidence intervals include 90%, so we present this as sensitivity evidence rather than statistically established undercoverage. Moreover, at the same audit budget, CGLCP increases finite availability from 45.3% for verified-safe-only CP to 61.3% (Appendix F.6). Local pooling also produces slightly narrower intervals than global pooling while bounding verification demand.

We acknowledge the trade-off highlighted by the reviewer: protection against adverse contamination comes with wider intervals and more abstentions. Sections 6.2 and 7 explicitly discuss this limitation; our goal is reliable prediction under limited verification, with guarantees conditional on the stated assumptions.

We hope these clarifications address your concerns, and we would sincerely appreciate your reconsideration of the assessment in light of the revised scope and evidence. Thank you again for helping us improve this work.

Reviewer_nHRJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses safe-pair prediction: prediction intervals for Yi(a⋆)Y_i^{(a^\star)} when a fallible working graph asserts that intervention a⋆a^\star does not affect target ii. Because the query-local calibration pool (predicted-safe interventions within graph distance τ\tau) may contain truly unsafe members whose scores are not exchangeable with the test score, and a global graph-error rate does not bound local contamination (Eq. 4), the authors propose CGLCP: verify a uniform random subset of size ss of the local pool with an exact oracle, invert the hypergeometric CDF to get an upper bound Bˉ\bar B on the unsafe count at level ρ\rho, and use the calibration order statistic at rank k=Bˉ+⌈(1−β)(n−Bˉ+1)⌉k = \bar B + \lceil(1-\beta)(n-\bar B+1)\rceil, β=(α−ρ)/(1−ρ)\beta=(\alpha-\rho)/(1-\rho). Simulations (adversarially low-rank unsafe scores) show CGLCP keeps coverage where global-rate corrections under-cover and PCS corrections abstain. On K562 Perturb-seq (5,994 reference-safe test pairs, 30% verification), CGLCP covers 0.978 with 61% finite output and width 1.061, vs. Pooled CP at 0.909 / 100% / 0.780.

Critical strengths

S1. Sharp, exact, and simple theory. The query-locality framing (Sec. 3.4) is a useful insight, and the rank in Eq. 10 is strictly less conservative than the additive PCS correction α′=α−B/(n−B+1)\alpha' = \alpha - B/(n-B+1): with BB known it is ≈(1−α)(n+1)+αB\approx(1-\alpha)(n+1)+\alpha B vs. ≈(1−α)(n+1)+(n+1)B/(n−B+1)\approx(1-\alpha)(n+1)+(n+1)B/(n-B+1) for PCS. Reducing the contamination penalty from B/(M+1)B/(M+1) to αB/(M+1)\alpha B/(M+1) is the most consequential technical point and is what makes the method usable at realistic contamination levels.

S2. Honest, thorough reporting. Coverage is always paired with finite-output rate; negative results are included (FN oracle noise breaks the certificate while coverage looks fine, Table 2; score mismatch is outside the theorem, Fig. 10; weighting gives no gain, Fig. 13; GraphLCP ranks better at matched FPR, Fig. 19). The K562 pipeline uses sealed, nonoverlapping folds.

Critical weaknesses

W1. The real-data study does not show the problem occurring, and the method is expensive where applied. On K562, label-free Pooled CP / GraphLCP already cover 0.909 / 0.911 with 100% finite output and width 0.780; CGLCP over-covers at 0.978, abstains on 39% of queries, and is ~36% wider, after exact labels on ~33% of each pool. CGLCP's advantage appears only in simulations where unsafe scores are placed adversarially at the lowest ranks (∣N(0,0.12)∣|N(0,0.1^2)|). In the motivating application, truly affected pairs should generically yield larger residuals, which the paper concedes only make the correction conservative. No evidence is given that low-rank contamination is realistic.

W2. Limited novelty. The "drop BB worst, run CP on the rest" bound, hypergeometric upper confidence limits, and the ρ+(1−ρ)β\rho + (1-\rho)\beta error split are all standard; Appendix B is an application of Barber et al. (2023). The delta over Asiaee et al. (2026) + a sampling audit is thin and not clearly articulated (see Q4).

  1. Barber, R. F., Candès, E. J., Ramdas, A., & Tibshirani, R. J. (2023). Conformal prediction beyond exchangeability. The Annals of Statistics, 51(2), 816–845.
  2. Asiaee, A., Aryan, K., & Long, J. P. (2026). Partial causal structure learning for valid selective conformal inference under interventions. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337, 161–184.
Questions for authors

Q1. Can you show any real-data stratum or working graph where label-free baselines under-cover and CGLCP restores coverage with non-trivial finite output?

Q2. What fraction of the 0.254 global non-safe rate is "unresolved" vs. "definitively unsafe", and how do CGLCP's width and finite rate change if unresolved pairs are treated as safe or Δ\Delta is varied?

Q3. Which inequality is new relative to Asiaee et al. (2026)? Is the sharp αB/(M+1)\alpha B/(M+1) penalty (Eq. 6) already there, and is the PCS baseline implemented as in that paper?

Q4. Why not remove verified-unsafe members from the pool and certify Bˉ−X\bar B - X over n−Xn - X members (rank k−Xk - X)? Under Assumptions 1–3 this appears never worse.

Q5. (Clarification) Any empirical diagnostic supporting safe-score exchangeability (Assumption 2) on K562 at τ=3\tau=3?

  1. Asiaee, A., Aryan, K., & Long, J. P. (2026). Partial causal structure learning for valid selective conformal inference under interventions. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337, 161–184.
3 comments

We thank Reviewer nHRJ for valuing our local-risk framing and honest reporting. We have expanded our empirical evaluations and sensitivity analyses based on your feedback, and incorporated all new experiments into the manuscript appendices, with key comparisons and findings summarized in the main text.

Below are our responses to each weakness and question.

W1 & Q1

Corresponding to Section 6.2 (sensitivity analyses), Appendix F.9, and Table 8 in the revised manuscript.

Table R1.1: Coverage and Finite Output Under Upstream Effect Screening

Calibration settingPooled CP coverageGraphLCP coverageCGLCP coverageCGLCP finite output
Original calibration pools90.95%91.09%97.68%61.61%
Reference-fold effect screening88.97%88.97%96.72%61.70%

In practice, researchers often prioritize excluding large, non-specific measured responses during upstream screening to define reliable empirical negative controls and avoid confounding local network analyses (Dixit et al., 2016; Replogle et al., 2022). For instance, Mellis et al. (2024) pair an absolute log⁡2\log_2 fold-change cutoff of 0.5 with an adjusted pp-value criterion to filter differential expression, motivating an evaluation of how upstream effect filtering interacts with downstream calibration.

To examine a realistic scenario where label-free methods struggle, we evaluated a calibration screening rule on the K562 test pairs by retaining the 80% of calibration candidates with the smallest absolute effect estimates from separate reference folds. This screening rule operates independently of the final calibration residuals and categorical reference labels (relying only on continuous effect estimates from the reference folds).

As shown in Table R1.1, under this screening, both baselines had coverage point estimates below the 90% target (bootstrap 95% CIs: 86.07%–92.10% for Pooled CP, 86.08%–92.02% for GraphLCP). By contrast, CGLCP maintains 96.72% empirical coverage with a 61.70% finite-output rate (at a cost of ~14 verification labels per query and wider intervals). While the baseline confidence intervals span 90%, this sensitivity analysis illustrates a realistic upstream-filtering regime where label-free coverage erodes in point estimate, whereas CGLCP preserves above-nominal coverage alongside substantial finite output.

Q2

Corresponding to Section 6.2 (sensitivity analyses), Appendix F.7, Table 6, and Section 7 (Discussion) in the revised manuscript.

Table R1.2: Sensitivity to Equivalence Margin (Δ\Delta) and Unresolved Pair Handling

Δ\Delta (log₂FC)Reference safe : unsafe : unresolvedCoverage, conservative → optimisticFinite output, conservative → optimisticFinite width (log₂FC), conservative → optimistic
0.3537.75% : 0.62% : 61.63%99.39% → 96.78%34.08% → 77.49%1.1321 → 1.1021
0.4053.00% : 0.51% : 46.48%98.84% → 96.77%44.74% → 77.52%1.1085 → 1.1019
0.5074.59% : 0.36% : 25.04%97.68% → 96.76%61.61% → 77.55%1.1064 → 1.1017
0.6086.43% : 0.24% : 13.33%97.16% → 96.76%70.87% → 77.56%1.1359 → 1.1016
0.7594.34% : 0.16% : 5.50%96.89% → 96.76%76.19% → 77.60%1.1178 → 1.1014

At our default practical margin of Δ=0.50\Delta = 0.50 (matching the log⁡2\log_2 fold-change magnitude used by Mellis et al., 2024 for perturbation response filtering), 25.41% of working-graph-predicted-safe pairs were classified as reference non-safe. Of these, 98.57% were "unresolved" (confidence interval straddles the margin) and 1.43% were "definitively unsafe".

We treat unresolved pairs as unsafe because they may contain genuine causal effects hidden by statistical noise; assigning them to the safe pool could compromise conformal validity. To quantify this, we compared our conservative policy (unresolved = unsafe) to an optimistic policy (unresolved = safe) in Table R1.2. At Δ=0.50\Delta = 0.50, treating unresolved pairs as safe raises finite output by 15.94 percentage points while shifting empirical coverage from 97.68% to 96.76% (both above the 90% target). We have added this sensitivity analysis to clarify how unresolved assay labels drive the trade-off between finite availability and conservative protection.

W2 & Q3

Corresponding to Sections 1, 2, and 4 (attribution), Section 5 and Algorithm 1 (certification procedure), Appendix D (comparator definitions), Appendix F.6 (matched-information correction diagnostic), and Table 5 (audit-label use) in the revised manuscript.

We explicitly attribute the sharper inequality (αB/(M+1)\alpha B / (M+1)) to Asiaee et al. (2026, Appendix C) in Sections 2 and 4.

Our methodological contribution lies in bridging this oracle inequality with limited local verification. We couple exact finite-population hypergeometric auditing with the sharp contaminated-rank rule by allocating the total miscoverage budget α\alpha between certificate failure (ρ\rho) and conditional calibration (β\beta), and we characterize the resulting minimal certificate-valid rank and oracle-relative rank overhead.

Regarding the PCS baseline: our submitted PCS comparison applies the additive contamination correction in its stated selection setting (detailed in Appendix D), rather than rerunning external software conventions. To isolate the impact of the rank formula itself, we added a matched-information diagnostic in Appendix F.6 that holds the local pool, audit sample, certificate B‾\overline{B}, and conditional budget β\beta identical across both rules. Under this matched information, the additive rule yields 0% finite output versus 61.61% for the sharper rank, demonstrating why coupling our certificate to the sharper bound is essential for practical availability.

Q4

Corresponding to Section 6.2 (Results), Tables 1 and 5, Appendix D, Appendix F.6, and Section 7 in the revised manuscript.

Table R1.3: Efficiency Gains from Removing Verified-Unsafe Members

MethodCoverageFiniteFinite width (log₂FC)
Verified-safe-only0.9600.4530.752
CGLCP removal0.9750.6131.042
CGLCP (fixed pool)0.9780.6131.061

We originally retained verified-unsafe observations for analytical clarity: it cleanly separates graph-based pool selection from verification-based rank adjustment, making the conservative monotonicity of false positives transparent (Remark 1).

However, we fully agree that removing them is a clean efficiency refinement under exact verification. Replacing (n,B‾,k)(n, \overline{B}, k) with (n−X,B‾−X,k−X)(n-X, \overline{B}-X, k-X) preserves the valid-certificate event and finite availability while never increasing the threshold. As shown in Table R1.3, this removal variant narrows the finite-width summary from 1.061 to 1.042 log⁡2\log_2FC (and yields a paired width reduction of 0.0427 log⁡2\log_2FC across 30 audit draws on jointly finite instances) while preserving the exact 61.3% finite rate and 97.5% coverage. We have incorporated this refinement into Table 1 and Appendix F.6.

Q5

Corresponding to Section 3.3 (Workflow and assumptions), Section 6.2 (sensitivity analyses), Appendix F.8, Table 7, Appendix B (score-mismatch sensitivity), and Section 7 in the revised manuscript.

Table R1.4: 90% Empirical Tail Coverage Diagnostics (Target-Balanced)

ScalingReference-Safe PoolTest Coverage95% CIMean Safe Pool Size
OriginalLocal (τ=3\tau=3)0.9192(0.8893, 0.9439)41.52
OriginalGlobal Work0.8988(0.8704, 0.9245)48.86
OriginalRandom Matched0.9195(0.8915, 0.9436)41.71
SymmetricLocal (τ=3\tau=3)0.9206(0.8912, 0.9458)41.52
SymmetricRandom Matched0.9204(0.8926, 0.9455)41.71

To empirically probe safe-score alignment at τ=3\tau=3, we conducted a reference-safe tail diagnostic (Appendix F.8), restricting calibration pools to reference-safe pairs to separate score alignment from causal misclassification.

As shown in Table R1.4, the 90% empirical tail coverage for the local pool (0.9192) and a size-matched random pool (0.9195) closely align near the 90% nominal target, with overlapping 95% confidence intervals, and applying a symmetric target scale yields the same close agreement.

Consistent with empirical alignment on the K562 reference-safe cohort, these diagnostics clarify the division of roles in our framework: hard localization (τ=3\tau=3) defines a bounded query-specific population to control verification cost and local contamination risk, while conditional safe-score exchangeability remains a distinct structural assumption. To account for potential score shifts across measurement contexts, Appendix B provides an explicit total-variation sensitivity bound quantifying how safe-score mismatch propagates through the CGLCP guarantee.

References

Atray Dixit, Oren Parnas, Biyu Li, et al. Perturb-Seq: Dissecting Molecular Circuits with Scalable Single-Cell RNA Profiling of Pooled Genetic Screens. Cell, 167(7):1853–1866.e17, 2016. doi:10.1016/j.cell.2016.11.038.

Ian A. Mellis, Madeline E. Melzer, Nicholas Bodkin, and Yogesh Goyal. Prevalence of and gene regulatory constraints on transcriptional adaptation in single cells. Genome Biology, 25:217, 2024. doi:10.1186/s13059-024-03351-2.

Joseph M. Replogle, Reuben A. Saunders, Angela N. Pogson, et al. Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq. Cell, 185(14):2559–2575.e28, 2022. doi:10.1016/j.cell.2022.05.013.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.