Slow Prediction Concentration in Tabular Foundation Models

How do tabular foundation model predictions change with more labelled rows at inference? We measure prediction spread across contexts and the effect of deleting a single row for six released checkpoints from four model families. On disjoint higgs contexts of $1024$–$16,384$ rows, four checkpoints have fitted probability spread slopes between $-0.08$ and $-0.14$, compared with $-0.51$ for logistic regression. At $16,384$ rows, their accuracy is $8$–$10$ percentage points higher than logistic regression's. An exact-tree XGBoost reference refit on the same supports also concentrates faster. These slopes describe the observed range. They give no asymptotic guarantee. Deleting a support row can also alter the fingerprint features of retained rows. For TabPFN v2.5, we compare fingerprints on and off across two datasets, six support seeds, four sizes ($64$–$512$), and 50 fixed, unfiltered queries per dataset. Disabling fingerprints makes mean pairwise influence decay faster on both probability and log-odds scales. All four paired contrasts survive Holm correction. The log-odds exponent rises by $0.239$ on higgs and $0.550$ on magic. Faster influence decay does not by itself imply better prediction or a population-wide effect. Whether spread eventually concentrates faster at larger context sizes remains open.