PriorBridge: Integrating Language Priors into Prior-Fitted Networks via Rule-Guided Prediction Correction

Prior-fitted networks (PFNs) have become strong predictors for tabular data. They use labeled examples as context to infer target values from feature dependencies. However, their standard inference interface cannot directly accept user-specified knowledge expressed in natural language. This may limit their use in real-world scenarios, especially when the context data does not fully reflect the target distribution or users prioritize compliance with predefined rules over predictive quality. To bridge this gap, we propose PriorBridge, a framework that integrates language priors into PFN predictions through tree-based rules refined by a large language model (LLM). First, we make the PFN's prediction behavior readable as explicit rules. Given a labeled training set $(X_{\mathrm{train}},Y_{\mathrm{train}})$ as context, we query the PFN on target inputs $X_{\mathrm{test}}$ and fit a small tree-based proxy to the resulting predictions ${\hat{Y}}_{\mathrm{test}}$. The proxy's decision paths can then be expressed as natural-language rules. Second, we convert the language prior into input-dependent output ranges. The LLM combines the proxy rules with the natural-language constraint and assigns an output range to each leaf. Finally, we apply the refined ranges to the original PFN predictions. Predictions outside the corresponding ranges are projected to their nearest boundaries. Experiments show that PriorBridge effectively integrates language priors into PFN predictions. It achieves high prior compliance across different PFN backbones. When a prior captures patterns missing from the context data, our method can also use it to improve predictive quality.