Certify What the Budget Can Reach: Peer Assessment in Federated Data Markets

In federated data markets, evaluating decentralized client contributions without central ground truth or public validation sets is a core challenge. Peer prediction addresses this by rewarding informative reports via peer cross-validation, yet its theoretical guarantees often become operationally unusable in practice. Because distributed supplier pairs share only scarce task overlap, the platform must ration a limited assessment budget across many candidate pairs. Existing mechanisms assume fixed task assignments, while sequential allocation methods merely reduce generic uncertainty; neither assesses whether the remaining budget and local overlap can actually complete statistically defensible incentive evidence. Consequently, cautious schedulers often exhaust the entire budget without certifying any supplier, even when multiple certificates were affordable. We formalize this missing objective as budget-reachable certification under adaptive sampling and heterogeneous finite pools. We propose BRCert, which pairs simultaneous finite-population confidence bounds with a lookahead estimator of each edge’s remaining certification cost. BRCert prunes unreachable pairs and concentrates purchases on the most affordable reachable certificates without modifying the underlying payment rule. Across five real-world markets, BRCert achieves the highest mean certified coverage with zero false certificates under exact audits, outperforming the strongest baseline by 1.6% on average in the main setting. The gain increases to 1.9% with 50 candidate edges, demonstrating that finite-batch incentive design must prioritize completed evidence over diffuse exploration.