GIST: Generator-state-Informed Supervision for Tabular Foundation Models

Tabular foundation models (TFMs) based on prior-data fitted networks typically use sampled hard query labels, leaving avoidable sampling variability in pretraining. A key observation is that intermediate generator states can reveal label uncertainty beyond a sampled outcome. We propose **G**enerator-state-**I**nformed **S**upervision for **T**FMs (**GIST**), which uses these states to average query losses over conditional label distributions. GIST couples this supervision with entropy-calibrated noise in selected feature-generation steps, making the required distributions tractable and consistent with the model input. We show that this averaging preserves the population objective and yields no larger gradient covariance at fixed parameters within each generating distribution. Under matched pretraining schedules, GIST continuously improves mean rank across three architectures on both TabArena and TALENT without changing architecture or inference. Across these benchmarks and architectures, GIST wins 62.1% of paired dataset-level comparisons against the corresponding hard-label controls. Code is available at https://anonymous.4open.science/r/GIST-anonymous-B54D.