Toward Effective Relational Foundation Model: A Retrieval and Memory Based Approach

Relational Foundation Models (RFMs) aim to learn generalizable representations across different relational databases, whose critical information is distributed across tabular entity attributes, primary–foreign key connections, and multi-hop relational contexts. While existing methods have achieved strong performance on relational databases, our empirical experiments reveal three fundamental limitations: (1) Flattening relational databases obscures relational information, making primary–foreign key connections and relational context less explicit. (2) Graph-based methods process relations indiscriminately, introducing task-irrelevant relational context which contributes little to target tasks. (3) Existing pretraining objectives under-utilize relational information, limiting the models' ability to capture transferable relational information. To address these challenges, we propose RaMA (Retrieval and Memory Assistant), an RFM that utilizes relational knowledge from records stored in a dynamically evolving external relational memory. (1) RaMA preserves relational information through a relation-based model backbone and overall framework. (2) RaMA retrieves task-relevant relational information through a dynamically evolving external relational memory, where task-aware queries retrieve records for adaptive fusion with local context and reward-guided feedback constantly refines memory effectiveness. (3) RaMA explicitly learns relational information by jointly optimizing masked attribute reconstruction and masked foreign-key recovery. Extensive experiments on RelBench show that RaMA consistently outperforms baselines across classification, regression, and relation prediction, with up to 2.45% AUROC improvement and significantly lower MAE. It further improves zero-shot transfer by up to 3.24% on unseen databases, while retaining efficient adaptation with moderate time and GPU-memory costs.