Shapley Data Auditing Certificates: Bounding Downstream Fairness Without Model Access

Alignment auditing is organizationally separated from model development: those who assemble and curate data, those who train models, and third parties who assess the system for fairness and privacy compliance, rarely hold the same information. We take this role separation as our design constraint and ask what an auditor can certify using only the information its role grants. We formalize a staged pipeline in which each block consumes the output of the previous one, and attach to a data block a stage-local certificate: a Shapley attribution of downstream accuracy and group-fairness violation to that block's input features, computed from dependence measures over data variables alone, with no access to the model or its predictions. A tight one-sided guarantee makes the certificate meaningful under restricted access: the mutual-information and conditional-mutual-information certificates upper-bound the sensitive-attribute leakage—hence the statistical-parity and equalized-odds (EO) gaps—of every predictor built downstream, with equality exactly when the downstream model realizes all of that leakage. By benchmarking 26 measures on seven datasets, with robustness across seven downstream predictors, we quantify the pre-hoc attribution gap—the fidelity a data auditor forfeits by not holding model access—and trace it to whether a measure's scale moves with coalition size. A final study prices the audit itself: permutation sampling brings a 20-feature certificate whose exact evaluation costs 71–150 hours down to sub-1% error in 30–70 minutes, while declared causal constraints move fidelity non-monotonically, so the constraint set must be reported. When a biased upstream annotator emits a stage's supervision, a two-stage study on 26 U.S. states shows that label-dependent certificates register the upstream block, the statistical-parity certificate is blind to it, and an EO certificate on inherited labels does not decisively reveal whether downstream harm rises or falls. Our results yield construction guidelines indexed by access regime—rather than by objective, bounding the price of the information asymmetry that alignment governance is built on.