Learning with empirical measures: practical transformers with universality guarantees

We propose the *Quantile Transformer*, a transformer-based architecture for learning with empirical measures. This architecture alleviates the computational burden of existing transformer layers for this task, which either require all-to-all particle interactions or rely on expensive optimal transport subprocedures. For measures on $\mathbb{R}$, we encode each empirical measure as a finite-dimensional token derived from order statistics and establish universal approximation guarantees for continuous maps on compact classes. For measures on ${\mathbb{R}}^d$, we extend both the construction and the universality result to compact classes of elliptical distributions using finitely many one-dimensional projections. Preliminary synthetic experiments validate our approach in terms of both learning performance and computational efficiency.