ProxyBench: A Tabular Classification Benchmark that Mirrors Enterprise Data

Enterprises are increasingly deploying machine-learning models for tabular data. However, they face a structural evaluation gap: Very few proprietary enterprise datasets are publicly released, and public benchmarks often fail to capture enterprise deployment realities. Similarly, as model development within the scientific community relies on openly accessible benchmarks, it cannot directly address enterprise challenges and needs. To fill this gap, we introduce ProxyBench, an open-source enterprise-grade classification benchmark containing 134 tasks on 90 different tables. To ensure the similarity of ProxyBench with enterprise data without disclosing sensitive information, we first generate tasks from a large pool of public datasets. Then, we use a privacy-preserving data fingerprint to automatically identify the ones having the most similar characteristics to the tasks of an internal, proprietary benchmark. By comparing the resulting aggregated statistics, we show that ProxyBench is much more statistically similar to enterprise-grade data than existing open-source benchmarks. Furthermore, our experimental evaluation of eleven baseline models shows that ProxyBench offers a model ranking with a superior Spearman's correlation with the internal benchmark than open-source alternatives. ProxyBench enables reproducible and accessible evaluation of tabular models in an enterprise-grade context. It provides the research community with a tool to develop tabular learning methods that more closely reflect real-world enterprise data.