DensityFM: Foundation-Style Pretraining for Density Estimation

Estimating probability densities is at the heart of many data science and machine learning pipelines. However, existing estimators face a fundamental accuracy–efficiency trade-off: classic non-parametric methods are fast but struggle in high dimensions, while more expressive flow-based methods require costly training and tuning for each new dataset. In this work, we present DensityFM, a pretrained, foundation model-style density estimator achieving the best of both worlds. Pretrained once on large-scale and diverse distributions, DensityFM performs zero-shot density estimation on unseen datasets with frozen parameters at test time. Experiments across diverse synthetic distributions and multiple 64-dimensional real-world datasets demonstrate that DensityFM processes each dataset in 0.5 seconds, delivering a speedup up to $156\times$ over the fastest neural baseline, while achieving comparable or even superior accuracy, establishing a new foundation for scalable data science and knowledge discovery.