Relational In-Context Learning via Synthetic Pre-training with Structural Prior

arXiv:2603.03805v2 Announce Type: replace-cross
Abstract: Relational Databases (RDBs) are the backbone of modern business, yet they lack foundation models comparable to those in text or vision. A key obstacle is that high-quality RDBs are private, scarce and structurally heterogeneous, making internet-scale pre-training infeasible. To overcome this data scarcity, We introduce $textbfRDB-PFN$, the first relational foundation model trained purely via $textbfsynthetic data$. Inspired by Prior-Data Fitted Networks (PFNs) where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables, we design a $textbfRelational Prior Generator$ to create an infinite stream of diverse RDBs from scratch. Pre-training on $textbfover 2 million$ synthetic single-table and relational tasks, RDB-PFN learns to adapt to any new database instantly via genuine $textbfin-context learning$. Experiments verify RDB-PFN achieves strong few-shot performance on 19 real-world relational prediction tasks, outperforming graph-based and single-table foundation-model baselines (given the same DFS-linearized inputs), while using a lightweight architecture and fast inference. The code is available at https://github.com/MuLabPKU/RDBPFN