acceptodds
Under review as a conference paper at ICLR 2027

Certify or Abstain: Auditing Thresholded Decisions Under Dependence

Abstract

Decisions to raise an alarm, withhold an answer, or refer a case for human review are commonly made by comparing a score against a calibrated cutoff. Claims about the resulting population operating rate are typically assessed under independence, although evaluation units may share a patient, time series, or prompt. We formulate a certify-or-abstain audit that identifies the independence conditions required for valid certification. Under -wise independence, symmetrisation reduces the analysis of any procedure to a randomised rule based on the total count. For each null rate, its maximal certification probability is a discrete moment problem. The corresponding primal and dual solutions yield exact rational certificates of feasibility and impossibility. This formulation gives exact minimum sample requirements for randomised rules, certified deterministic gates, and a scaling law across independence orders. For independent clusters, we characterise a strong criterion with unrestricted intracluster dependence and give finite-sample moment and Bernstein audits when average intracluster correlation is bounded. The -wise hierarchy is used as a conditional sensitivity analysis rather than as a fitted model of positive intracluster dependence. Applications to clinical referral, spacecraft telemetry, and language model reliability demonstrate how the framework separates conclusions warranted by the sampling design from those that rely on unjustified independence at the observation level.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.