Characterizing Tabular Foundation Models in Early Literacy

Tabular foundation models show strong performance across diverse domains and settings, yet few studies examine their use for early literacy prediction-a consequential domain with scarce longitudinal data and heterogeneous, incomplete real-world records. We ask whether this promise carries over to K-4 literacy prediction in realistic settings. To answer this question, we introduce RealLit, a large-scale, longitudinal multi-cohort benchmark comprising 4,582 students across 53 schools and six U.S. states. A large portion of the benchmark comes from real-world school-system data, preserving naturally occurring missingness and heterogeneity in assessment practice, while providing fine-grained student-level demographic attributes across cohorts. Our benchmarking surfaces several key findings while offering guidance for evaluating performance beyond aggregate metrics using student-level metadata. For instance, we find that model ensembling can have uneven effects across student groups, reducing performance for some. Newer model variants can also underperform earlier ones. Surprisingly, in our missing-data analysis, learned imputation also provides little consistent downstream benefit over simpler missing-data handling strategies. By making such behaviors measurable, RealLit provides a standardized testbed for understanding and advancing the reliable use of tabular foundation models in real-world early literacy prediction. Our code and data is available at https://github.com/anonymous-submission-site/real-lit.