KnowsTFM: Towards Knowledge-Grounded Tabular Foundation Models

Tabular foundation models have advanced deep learning for tabular data by achieving strong performance across many tasks. However, adapting these methods to data-scarce domains is challenging. Many such domains provide curated relational knowledge in the form of knowledge graphs and bases, but how to use this knowledge to improve and steer specialist tabular foundation models remains unclear. We address this problem through Knowledge-informed fine-tuning of Tabular Foundation Models (\modelname), which incorporates relational knowledge directly into feature attention. We evaluate the approach across controlled nanoscale TabPFN checkpoints and the frontier TabPFN-v3 model to study when external structure improves downstream adaptation. During fine-tuning, we use structural priors derived from knowledge graphs to guide feature-to-feature attention, encouraging interactions between features linked in the graph. Our results on 42 biomedical and 13 cross-domain datasets show that this approach yields meaningful gains over standard fine-tuning in domain-specific tasks, while maintaining competitive performance on general-domain tasks.Source code is available at https://anonymous.4open.science/r/kg_tuned_tfms-DBC5.