acceptodds
Under review as a conference paper at ICLR 2027

PriorBridge: Integrating Language Priors into Prior-Fitted Networks via Rule-Guided Prediction Correction

Abstract

Prior-fitted networks (PFNs) have become strong predictors for tabular data. They use labeled examples as context to infer target values from feature dependencies. However, their standard inference interface cannot directly accept user-specified knowledge expressed in natural language. This may limit their use in real-world scenarios, especially when the context data does not fully reflect the target distribution or users prioritize compliance with predefined rules over predictive quality. To bridge this gap, we propose PriorBridge, a framework that integrates language priors into PFN predictions through tree-based rules refined by a large language model (LLM). First, we make the PFN's prediction behavior readable as explicit rules. Given a labeled training set as context, we query the PFN on target inputs and fit a small tree-based proxy to the resulting predictions . The proxy's decision paths can then be expressed as natural-language rules. Second, we convert the language prior into input-dependent output ranges. The LLM combines the proxy rules with the natural-language constraint and assigns an output range to each leaf. Finally, we apply the refined ranges to the original PFN predictions. Predictions outside the corresponding ranges are projected to their nearest boundaries. Experiments show that PriorBridge effectively integrates language priors into PFN predictions. It achieves high prior compliance across different PFN backbones. When a prior captures patterns missing from the context data, our method can also use it to improve predictive quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
tabular foundation models, llm

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_iJzsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PriorBridge, a framework for incorporating user-specified natural-language priors into the predictions of frozen Prior-Fitted Networks (PFNs) without parameter updates.

The framework first queries a PFN on unlabeled target inputs and fits a tree-based proxy to the resulting predictions. The proxy's decision paths provide a language-readable description of the PFN's local predictive behavior. An LLM then combines these paths with feature descriptions and user-specified priors to generate rule trees whose leaves specify admissible output intervals. At inference time, the intervals obtained from multiple LLM-generated trees are intersected, and the original PFN prediction is projected onto the resulting range. For classification, the projection applies to a designated class probability.

The paper evaluates TabPFN-v2 and LimiX-2M on 21 tabular datasets with 55 constraints, spanning IID and OOD settings. It reports substantial increases in prior compliance, although predictive performance sometimes improves and sometimes deteriorates. Additional experiments include prior-source ablations, cross-model proxy rules, and sample-level correction examples.

Critical strengths

1. Meaningful problem formulation with practical relevance.

The paper addresses an important problem: incorporating human-specified domain knowledge into frozen PFNs without retraining. The distinction between predictive accuracy and prior compliance is meaningful, particularly in OOD settings where domain knowledge may complement insufficient training context.

2. Explicit and inspectable prediction correction.

The framework converts natural-language priors into executable tree-based constraints, allowing individual corrections to be traced through decision paths and output intervals. Experiments across multiple datasets and PFN backbones provide initial evidence of its practical applicability.

Critical weaknesses

1. Insufficient attribution of improvements to the proposed mechanism rather than highly informative priors.

Some supplied priors already contain precise, task-specific numerical constraints, potentially functioning as independent rule-based predictors. Without information-matched prior-only and direct-integration baselines, it is difficult to determine whether performance improvements arise from the proposed framework or simply from the additional predictive information. Moreover, the pipeline resembles existing LLM-assisted rule-manipulation approaches, while its PFN-specific contributions remain insufficiently established.

2. Insufficient characterization of semantic fidelity and system reliability.

The LLM simultaneously interprets human priors, modifies tree structures, and generates refined numerical bounds, making it unclear whether the resulting constraints faithfully preserve the original requirements or introduce unsupported assumptions. In particular, narrower intervals do not necessarily improve correctness, pointwise constraints do not generally guarantee monotonicity, and intersecting multiple LLM-generated intervals may amplify over-constraining errors. More systematic component ablations, sensitivity analyses, and failure-case investigations are needed to establish the reliability of the overall framework.

Questions for authors

Overall

I find the problem relevant, and I appreciate the idea of making prior-guided corrections explicit and executable. However, the current empirical evidence does not yet convince me that the proposed integration mechanism contributes substantially beyond the predictive information already supplied by the priors, or that the generated rules reliably preserve the original constraints. I therefore lean toward a 2 (weak rejection) in this round.

Q1 and Q2 are the most important questions for my assessment, since they concern the provenance and information content of the priors and whether the reported improvements can be attributed to the proposed method. Q3 and Q5 are also potentially score-changing, because they concern the validity of the reported compliance and the observed failures. Clear evidence addressing these concerns could substantially improve my assessment.

Q4, Q6, Q7, and Q8 would strengthen the explanation of the method and its scope, but I do not expect all suggested experiments to be completed in a rebuttal. Q9 is primarily a presentation suggestion and would not, by itself, change my score.

Questions

Q1. Clarification/Evidence: How are the priors constructed, and how much predictive information do they already contain?

Could the authors explain the complete construction protocol for the 55 priors? In particular, how were numerical thresholds, output ranges, feature-specific conditions, and OOD descriptions selected? Which quantities came from independent domain knowledge, which were obtained from the training context, and were any priors revised after observing validation or other performance information?

Appendix C includes relatively precise, region-specific ranges for CCPP. How representative is this level of precision across the evaluated datasets? For classification, what fraction of samples have their final class already determined by the original human priors alone? For regression, how predictive are the supplied bounds without using the PFN or the LLM-refined tree?

I am not suggesting that strong priors are inherently invalid. Rather, these details are necessary to distinguish integration of qualitative knowledge from a setting where the supplied prior already serves as a powerful rule-based predictor. They are also important for interpreting the actual information budget of the experiments.

Could the authors also clarify whether the priors are intended to constrain individual predictions, conditional expectations, physical response bounds, or application-specific decision policies? These are not always equivalent, particularly when a prior is motivated by a causal or physical statement.

Q2. Evidence/Ablation: What is the incremental contribution of PriorBridge under exactly matched prior information?

Could the authors compare against methods given the same original prior instances, including their exact regions, thresholds, numerical bounds, and exceptions?

I would particularly like to see an original-prior-only baseline, direct prior execution on the PFN outputs, and a variant where the LLM receives the same priors and metadata but no proxy rules. A fixed-proxy-structure variant that only generates leaf intervals would further separate structural rewriting from numerical prior assignment.

For explicitly numerical priors, direct execution could be implemented without LLM-based tree rewriting. For monotonic priors, an information-matched comparison could enforce the original ordering relations directly rather than inventing numerical bands from the context. A manually formalized version could be treated as a semantic-oracle ablation, isolating the contribution of language understanding from that of the subsequent correction algorithm.

These comparisons would make it possible to determine whether the main gains arise from the extra domain information, the use of the original PFN predictions, or the specific proxy–LLM–projection mechanism. It would also be important to compare predictive performance at similar levels of original-prior compliance.

Q3. Clarification/Evidence: How is semantic fidelity between the human priors and the generated rules established?

The LLM modifies both the input partitions and the numerical leaf intervals. What conditions guarantee, or at least empirically verify, that the resulting rule tree preserves the requirements expressed by the user?

Could the authors distinguish between restrictions logically implied by the original prior and additional restrictions inferred from the proxy or the LLM's pretrained knowledge? How are unsupported narrower intervals identified? In particular, how should the additional narrowing shown in Figure 3 be interpreted when the original prediction is already within the stated broad numerical range?

I am also concerned about qualitative constraints such as monotonicity. Pointwise interval satisfaction does not, in general, guarantee pairwise ordering of predictions. Is monotonicity evaluated using controlled comparable pairs or through a surrogate based on the generated bands? Could the authors provide the independent evaluation predicates for the original prior semantics, rather than only verifying membership in the generated intervals?

This is central to the claim that the method enforces user-specified knowledge instead of merely enforcing constraints constructed by the LLM itself.

Q4. Clarification: What happens when the PFN, the supplied priors, or the generated rules disagree?

When a PFN prediction strongly contradicts a supplied human prior, does the method always give priority to the generated interval? Does the procedure distinguish a likely PFN error from a potentially incorrect or locally inapplicable prior, or is its intended role strictly to enforce user requirements regardless of predictive accuracy?

What happens when multiple human priors are mutually inconsistent, or when separately generated rule trees produce an empty intersection? Appendix A.2 describes filtering structurally invalid trees, but how are semantically incompatible yet structurally valid trees handled? What happens if no valid candidate remains?

For multiclass classification, how are simultaneous constraints on different class probabilities implemented? In particular, can proportional rescaling for one class violate an already-satisfied constraint on another class, or make the result dependent on the order of corrections?

Finally, is the present framework intended only for hard constraints? It would help to state explicitly whether soft violations, graded prior reliability, or a controllable violation budget are supported. I do not regard support for every such extension as necessary, but the intended semantics should be clear.

Q5. Evidence/Failure analysis: What are the actual sources of unsuccessful or unnecessary corrections?

Could the authors provide a failure analysis organized by correction type, rather than relying mainly on average compliance and predictive metrics?

Several existing results warrant explanation. For LimiX on Gas Turbine NOx, reported compliance is 100% -> 100%, yet MAE worsens from 6.39 to 7.32. For TabPFN on QSAR Biodegradation, corrected compliance remains 75.0%. Conversely, LimiX on Dry Bean improves accuracy from 0.610 to 0.840 under one supplied prior. How much of the latter gain is already determined by the prior, and what accounts for the former failures?

It would be useful to distinguish original-prior violations from generated-interval violations, as well as cases where the base PFN was already compliant but the method nevertheless changed its output. For classification, reporting incorrect-to-correct and correct-to-incorrect changes would make the net accuracy gains more informative. For regression, the distribution of correction magnitudes and the frequency of harmful modifications would be valuable.

A small number of detailed failure traces—from original prior, through proxy path and LLM edits, to final projection—would help identify whether the bottleneck lies in understanding, representation, aggregation, or execution.

Q6. Evidence/Ablation: How do proxy fidelity, tree complexity, and the tree-learning procedure affect downstream correction?

The method relies on a proxy forest whose paths are presented to the LLM. How does correction quality change with maximum tree depth, leaf budget, forest size, tree-learning method, and the complexity of the target data?

A more accurate proxy is not necessarily more useful for applying a domain prior: it may reproduce irrelevant teacher behavior while failing to expose the features or regions needed by the constraint. Conversely, a very detailed tree may increase prompt length and make LLM editing more difficult.

Could the authors measure both teacher–proxy fidelity and downstream correction quality, and investigate whether the former predicts the latter? For classification, what is lost by fitting the proxy to hard labels rather than full PFN probabilities, given that correction is ultimately probability-based?

The cross-model proxy results are interesting in this regard. Could the authors use them to clarify what aspects of the proxy are actually necessary for a successful correction, rather than interpreting cross-model transfer only as backbone flexibility?

Q7. Evidence/Robustness: How does the number of generated rule trees T influence the results?

The method uses T = 5 generated rule trees and intersects their intervals. Because this aggregation takes the maximum lower bound and minimum upper bound, increasing T does not necessarily average away LLM errors; it can instead make the allowed interval progressively more restrictive.

Could the authors report a sensitivity analysis over T, including predictive quality, original-prior compliance, interval width, correction magnitude, the number of valid generated trees, and the frequency of empty intersections?

How much variation arises from independently rerunning the full LLM-based construction procedure? It would be important to distinguish the performance of a fixed generated rule set from the reliability of regenerating that set.

Relatedly, are the corrections stable under semantically equivalent paraphrases, changes in prior order, redundant constraints, and a null or nonrestrictive prior? These tests could help distinguish sensitivity to the actual constraint semantics from sensitivity to incidental prompt wording.

Q8. Clarification: What distinguishes this contribution from existing LLM–tree pipelines, and what is specific to PFNs or the intended deployment setting?

Could the authors provide a more direct comparison with the closest LLM-assisted tree and neuro-symbolic methods, including DeLTa and FORESTLLM? It would be useful to distinguish the information provided to each method, the role of the LLM, the source of the correction signal, and the semantics of the final executable representation.

More specifically, which properties of PFNs are essential here beyond supplying high-quality initial predictions? The proposed operations seem applicable to many black-box predictors. If the method is intentionally model-agnostic, that positioning would be reasonable; if PFN-specificity is central, I would like to understand how context-conditioned adaptation, context coverage, or PFN-specific uncertainty affects the mechanism.

There is also a transductive aspect: the proxy is fitted using predictions on the current unlabeled target inputs. Are the rules intended for a fixed target batch, or can they be reused on independent future inputs? How does performance change if the rules are constructed on one unlabeled batch and then frozen for another? Please also clarify the setup cost and when proxy fitting or LLM generation must be repeated.

Q9. Writing and reproducibility

The overall pipeline is reasonably clear, but a complete worked example would substantially improve the presentation. In particular, a side-by-side visualization of the original proxy tree and the LLM-refined tree would help readers identify what is preserved, what is modified, and what additional information enters the output intervals.

I would also recommend a compact notation table distinguishing the PFN predictive output, proxy prediction, original human-prior feasible set, generated leaf interval, and final projected output. These objects currently appear in different stages of the paper but play importantly different roles.

Finally, the manuscript states that implementation code is supplied in the supplementary materials. Could the authors point to the exact locations of the complete prompts, original-prior definitions, evaluation predicates, and handling of empty or invalid rule sets?

8 comments
AuthorsOpenReview ↗

Response to Reviewer iJzs (1/4)

We thank you for your detailed review and for clearly prioritizing your concerns. We have carefully considered all questions, including those on presentation, reproducibility, and other supporting aspects. Given their breadth and the limited rebuttal period, completing separate experiments and detailed responses for every point is not feasible. Following your prioritization, we focus below on prior provenance and information content (A1), same-prior integration controls (A2), semantic fidelity (A3), and failure analysis (A5). These answers target the central concerns in your two critical weaknesses, with A3 and A5 combined where the same correction analyses address both questions. We will continue addressing the remaining comments and incorporate completed revisions into the revised paper.

[R1 & A1] Thank you for emphasizing the information budget of the priors. We address their construction and predictive information separately below.

Prior construction and information sources.

Which quantities came from independent domain knowledge, which were obtained from the training context, and were any priors revised after observing validation or other performance information?

Regarding prior construction, the information sources include domain relations, training-context statistics, and user-specified requirements. Some priors were constructed with reference to held-out target information. We jointly report compliance and predictive performance to assess how effectively PriorBridge incorporates the supplied prior information. We will clarify the construction protocol and these information sources in the appendix.

PriorBridge aims to improve predictions and satisfy user requirements through additional language priors. The OOD experiments evaluate prior-assisted correction, rather than generalization without extra information. The LLM receives language constraints and PFN-derived rules, not per-example test labels. This reflects practical settings where domain knowledge or user requirements can guide predictions.

How predictive are the priors on their own?

For classification, what fraction of samples have their final class already determined by the original human priors alone? For regression, how predictive are the supplied bounds without using the PFN or the LLM-refined tree?

We have added prior-only baselines that do not use PFN predictions or LLM-generated rules. For regression, the baseline predicts the midpoint of an applicable finite prior interval. For Dry Bean, it predicts the class specified by the prior in the applicable region and uses the training-majority class elsewhere.

DatasetPrior-only performanceOriginal-prior coverage
CCPPMAE 5.46100%
AirfoilMAE 6.69100%
Dry BeanAccuracy 0.36124.14%

The Dry Bean prior specifies a class for 24.14% of the target samples, and that class is correct on 96.29% of the covered subset. It therefore supplies highly useful class information where applicable, while predictions elsewhere still require a model. We report both coverage and covered-subset accuracy to make this information content explicit. For CCPP and Airfoil, [R1 & A2] below compares PriorBridge with these midpoint baselines and direct execution of the same numerical priors.

AuthorsOpenReview ↗

Response to Reviewer iJzs (2/4)

[R1 & A2] Thank you for requesting comparisons under the same supplied priors. We address the simpler baselines and the role of proxy-guided construction separately.

Same-prior baselines and predictive value.

Could the authors compare against methods given the same original prior instances, including their exact regions, thresholds, numerical bounds, and exceptions?

We compare PriorBridge with Direct using the same original numerical priors. Direct clips PFN predictions to the supplied bounds in their corresponding feature regions, without a proxy or an LLM. The prior-only midpoint results are given in [R1 & A1] above. Base and PriorBridge scores follow the results reported in the paper.

DatasetBase MAEOriginal-prior Direct MAEPriorBridge MAEDirect / PriorBridge original-bound compliance
CCPP4.494.553.95100% / 100%
Airfoil5.795.244.74100% / 100%

Direct and PriorBridge both achieve 100% original-bound compliance, while PriorBridge has lower reported MAE than Direct and the prior-only midpoint baseline. On CCPP, Direct's reported MAE is even higher than Base's (4.55 vs. 4.49). Clipping to broad intervals is a coarse correction that does not further refine predictions within those bounds. Thus, we evaluate compliance and predictive quality separately rather than use high compliance alone as evidence of the framework's additional value.

Proxy-guided prior-to-rule construction.

I would particularly like to see an original-prior-only baseline, direct prior execution on the PFN outputs, and a variant where the LLM receives the same priors and metadata but no proxy rules.

We have added a no-proxy ablation that preserves the original priors and metadata but removes the proxy rules. We also remove directional language while retaining the numerical bounds, proxy rules, and metadata. The CCPP results are summarized below, with the paper's Base and full-version scores included as references.

VariantMAEOriginal-bound compliance
Base4.4995.4%
Original-prior Direct4.55100%
Without proxy rules4.55100%
Without directional language4.28100%
Full version3.95100%

Without proxy rules, the reported MAE is essentially the same as Direct's (4.55). Without directional language, the reported MAE is 4.28. Both ablations achieve 100% numerical compliance, yet have higher reported MAE than the full version. These results support the usefulness of proxy context and semantic prior information for local refinement beyond bound enforcement.

The components have complementary roles. The prior specifies the desired behavior, the proxy summarizes the PFN's feature-conditioned predictions, and the LLM translates them into executable correction rules. The proxy is fitted to PFN predictions on the unlabeled inputs to be corrected, without their target labels. The final correction retains the original PFN prediction and projects it into the generated interval, rather than replacing it with the proxy prediction.

AuthorsOpenReview ↗

Response to Reviewer iJzs (3/4)

[R2 & A3 & A5] Thank you for requesting semantic checks and explanations of the highlighted cases. We distinguish evaluation against the original prior, the role of local refinement, and the specific prediction outcomes below.

Independent evaluation of prior semantics (A3).

Could the authors provide the independent evaluation predicates for the original prior semantics, rather than only verifying membership in the generated intervals?

Numerical compliance is checked directly against the original bounds in the applicable feature regions, rather than against the generated leaf intervals. For classification, the check uses the class condition specified by the prior. This evaluates whether the final prediction satisfies the supplied requirement independently of the rule used to produce it.

Qualitative relations require checks on comparable inputs. Our controlled invariance tests use a point-prediction extension and compare final predictions on pairs that differ only in the transformation covered by the prior, such as exchanging equivalent sensors. A pair is compliant when ∣y^(x)−y^(x′)∣≤0.05|\hat{y}(x)-\hat{y}(x')|\leq 0.05. This check does not use target labels. Directional priors instead require an ordering predicate on such pairs, not an interval-membership check. We will make the evaluated prior types and predicates explicit, distinguishing numerical-bound compliance from qualitative relation compliance.

Interpretation of additional local refinement (A3).

In particular, how should the additional narrowing shown in Figure 3 be interpreted when the original prediction is already within the stated broad numerical range?

In Figure 3, the broad range expresses the original requirement, while the narrower leaf interval is a feature-conditioned refinement constructed from the prior and proxy summary. The broad prior defines admissibility, and the local interval supplies an additional predictive estimate. This refinement is not simply a restatement of the original bound. It can change an already-compliant prediction while keeping it within that bound. We assess its usefulness through predictive performance alongside original-prior compliance, as in [R1 & A2]. The NOx case below illustrates why the accuracy of the local interval matters.

Reviewer_rxUBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses how user-defined natural-language priors or constraints can be incorporated into the predictions of prior-fitted networks (PFNs). The authors introduce PriorBridge, a post-hoc method that enforces these priors at inference time without modifying the underlying model. The approach first trains a tree-based proxy to approximate the test predictions of a frozen PFN and extract interpretable decision rules. An LLM then combines these rules with user-provided priors to generate several refined decision trees that define admissible prediction intervals. For each test instance, the applicable intervals are intersected, and the original PFN prediction is projected onto the resulting range if necessary. The method is evaluated on several classification and regression benchmarks, under both IID and OOD settings, using two PFN backbones. The authors report consistent improvements in prior compliance, with improvements in predictive performance in some settings.

Critical strengths
  • The paper addresses an important limitation of PFNs; the lack of an explicit interface for incorporating user-defined or domain-specific priors into frozen models.
  • The correction mechanism is transparent, as the final adjustment is expressed through explicit rules and prediction intervals.
  • The evaluation is broad, covering multiple datasets, classification and regression, IID/OOD settings, and two PFN backbones.
Critical weaknesses
  • The necessity of the proposed framework is unclear. The proposed pipeline introduces considerable complexity without establishing whether its components are needed. Comparisons with simpler alternatives, such as direct constraint projection or synthetic context augmentation, are missing. It is therefore difficult to tell whether the improvements come from the combination of its components or simply from enforcing informative constraints. This is also a limitation of the prior-source ablation, which is used to argue that the method offers benefits beyond generic clipping but does not include a direct projection baseline using the same priors.
  • The experiments do not disentangle the contributions of the proxy, LLM, and user-provided priors. In particular, proxy fidelity and sample requirements are not assessed, the LLM may also bring in additional knowledge beyond the supplied priors. Consequently, it is unclear how much each component contributes to the final prediction intervals.
  • The compliance evaluation provides limited insight. Since the method explicitly projects predictions onto admissible intervals, high compliance is largely expected by construction. The evaluation would be more informative if it examined how often predictions need to be adjusted, the magnitude of these adjustments, how compliance varies across prior types, and why compliance is not perfect when the priors are intended to be explicitly enforced. It is also unclear how the method handles conflicting constraints or empty interval intersections.
  • The construction of the priors raises concerns about evaluation validity. The paper does not sufficiently explain how the numerical bounds used in the priors were determined, or whether they were specified independently of the held-out data. This is particularly important in the OOD experiments, where precise bounds could inadvertently introduce information about the target distribution. Without clarifying this, it is difficult to determine whether the reported gains reflect genuine robustness to distribution shift or simply the use of informative constraints
  • The evaluation only partially tests the stated natural-language prior setting. Many evaluated priors are already highly formalized, containing explicit numerical bounds, conditional rules, probability ranges, or simple monotonic relationships. Although these are expressed in language, much of the executable structure is already specified, making it unclear how well the framework addresses the more challenging problem of translating less structured natural-language knowledge into constraints.
Questions for authors
  • Why is the proxy-tree + LLM pipeline needed? Could you compare PriorBridge against simpler alternatives, such as direct constraint projection or synthetic context augmentation that encode the same prior information? In the current evaluation, many priors are already expressed as explicit bounded rules or simple monotonic relationships, which should be straightforward to encode in substantially simpler baselines.
  • How much of the gain comes from the supplied prior, the proxy-tree structure, and the LLM itself? In particular, how faithful are the proxy trees to the original PFN and how do they affect the final predictions? Since the LLM also receives dataset and task descriptions, how can you distinguish the effect of the provided priors from the LLM’s own knowledge?
  • What does high compliance demonstrate beyond successful projection? Because predictions are explicitly constrained to admissible intervals, could you report the frequency and magnitude of prediction adjustments, as well as compliance broken down by prior type and by prior?
  • How were the numerical bounds in the priors determined? Were they specified independently of the held-out test data, labels, and predictions? In particular, how were the OOD splits and their corresponding priors constructed?
  • How well does the method handle genuinely vague or qualitative priors? Many real-world domain constraints are not expressed as precise numerical rules. How reliably can the proposed approach translate such priors into useful constraints?
5 comments
AuthorsOpenReview ↗

Response to Reviewer rxUB (1/3)

We would like to thank you for your careful review and constructive questions. We provide targeted additional experiments on representative datasets to address the main empirical concerns. Each weakness overlaps its corresponding question, so we respond to R1–R5 together with A1–A5.

[R1 & A1] Thank you for requesting comparisons with simpler alternatives. We compare PriorBridge with Direct and a prior-only midpoint baseline using the same original numerical priors. Direct clips PFN predictions to the original bounds in their corresponding feature regions, without a proxy or an LLM. The prior-only baseline predicts the midpoint of an applicable finite prior interval without using PFN predictions. Base and PriorBridge scores follow the results reported in the paper.

DatasetBase MAEPrior-only midpoint MAEDirect MAEPriorBridge MAE
CCPP4.495.464.553.95
Airfoil5.796.695.244.74

Direct and PriorBridge both achieve 100% original-bound compliance on these two tasks, while PriorBridge has lower reported MAE than both simpler baselines. For numerical priors, compliance is checked directly against the original bounds in the applicable feature regions. Thus, we evaluate compliance and predictive quality separately rather than use high compliance alone as evidence of the framework's additional value.

The additional value is prior-to-rule construction guided by the proxy summary, rather than clipping alone. PriorBridge constructs feature-conditioned correction rules that refine predictions beyond simply enforcing the supplied broad ranges. A fixed numerical clipping baseline assumes that executable bounds have already been specified. Qualitative priors need not supply such bounds, as examined in R5 and A5.

[R2 & A2] Thank you for proposing this concern. We address these two aspects separately below.

Component contributions.

How much of the gain comes from the supplied prior, the proxy-tree structure, and the LLM itself?

Since the LLM also receives dataset and task descriptions, how can you distinguish the effect of the provided priors from the LLM’s own knowledge?

We have added a no-proxy ablation that preserves the original priors and metadata but removes the proxy rules. We compare the following variants on CCPP under the original priors.

VariantMAEOriginal-bound compliance
Base4.4995.4%
Original-prior Direct4.55100%
Without proxy rules4.55100%
Without directional language4.28100%
Full version3.95100%

Direct achieves 100% numerical compliance, but its reported MAE is higher than Base's (4.55 vs. 4.49). Clipping to broad intervals is a coarse correction that offers no further refinement within those bounds. Numerical compliance alone therefore does not guarantee improved prediction accuracy. When proxy rules are removed, the LLM-based correction has essentially the same reported MAE as Direct (4.55). When directional language is removed, the proxy and task metadata remain available, but the reported MAE is 4.28. Both variants satisfy the numerical bounds, yet have higher reported error than the full version (3.95). These results support the usefulness of both the proxy summary and semantic prior information. Their roles are complementary. The supplied priors specify the desired behavior, the proxy describes the PFN's feature-conditioned predictions, and the LLM translates them into executable correction rules. Task metadata provides the feature meanings and units needed to interpret these inputs.

Proxy fidelity.

In particular, how faithful are the proxy trees to the original PFN and how do they affect the final predictions?

The proxy is fitted to PFN predictions on the unlabeled inputs to be corrected, without using their target labels. We measure fidelity on these inputs, matching the proxy's role in summarizing the PFN's behavior for correction. The following examples use five proxy trees, maximum depth 10, and at most 50 leaves per tree. For regression, we report the mean squared difference between proxy and PFN predictions on the preprocessed target scale. For classification, we report their predicted-label agreement.

DatasetUnlabeled inputsProxy–PFN MSE ↓\downarrowPredicted-label agreement ↑\uparrow
CCPP27690.00126—
Airfoil6310.00697—
Concrete2810.0123—
KC279—100%
Gas Drift4364—98.7%

At the current tree budget, the proxy closely reproduces TabPFN's predictions on these inputs, supporting its use as a reliable behavioral summary for the LLM. Its exported tree paths express this behavior in feature-conditioned regions. The final correction still retains the original PFN prediction and projects it into the generated interval, rather than replacing it with the proxy prediction. We will include these fidelity measurements and the fitting protocol in the appendix.

AuthorsOpenReview ↗

Response to Reviewer rxUB (2/3)

[R3 & A3] Given the number of datasets in our evaluation, we use CCPP and Airfoil as illustrative cases here. We report correction frequency, correction outcomes, and per-prior compliance below.

DatasetTest casesDirect changesPriorBridge changesPriorBridge changes to originally compliant casesPriorBridge error-reducing / error-increasing changes
CCPP276938682644547 / 135
Airfoil631140359219245 / 114

Most adjusted predictions have lower absolute error on both tasks. PriorBridge also refines predictions that already satisfy the original broad bounds, rather than only bringing out-of-range predictions back into compliance. For the three original numerical priors, Base per-prior compliance is 100/100/84.17% on CCPP and 100/73.31/10.47% on Airfoil. Direct and PriorBridge reach 100% on all six conditions. The two methods differ in how they refine predictions, even when their numerical compliance is identical. PriorBridge uses the proxy's local predictive structure and LLM refinement to make corrections beyond those required by the original bounds. The error-reduction counts on both tasks support the predictive value of this additional refinement.

[R4 & A4] Thank you for requesting clarification. We address prior construction and OOD split protocols separately below.

Prior construction.

How were the numerical bounds in the priors determined? Were they specified independently of the held-out test data, labels, and predictions?

Regarding prior construction, the information sources include domain relations, training-context statistics, and user-specified requirements. Some priors were constructed with reference to held-out target information. We jointly report compliance and predictive performance to assess how effectively PriorBridge incorporates the supplied prior information. We will clarify the construction protocol and these information sources in the appendix.

PriorBridge aims to improve predictions and satisfy user requirements through additional language priors. The OOD experiments evaluate prior-assisted correction, rather than generalization without extra information. The LLM receives language constraints and PFN-derived rules, not per-example test labels. This reflects practical settings where domain knowledge or user requirements can guide predictions.

OOD splits.

In particular, how were the OOD splits and their corresponding priors constructed?

OOD refers to an explicit difference between context and target subsets, not an intrinsic property of a dataset name. The split protocols are provided below.

DatasetContext subsetTarget subset
AirfoilFrequency ≤2000Frequency >2000
CCPPAT <25°CAT ≥25°C
ConcreteCuring age ≤28 daysCuring age >28 days
Gas Turbine NOxAT ≤15°CAT >25°C (intermediate temperatures excluded)
Seoul BikeTemperature ≤20°CTemperature >20°C
SuperconductivityNumber of elements ≤4Number of elements ≥5
Dry BeanLower-eccentricity groupHigher-eccentricity group (boundary approximately 0.76444)
Bike Sharing20112012
CICIDS2017Monday–WednesdayThursday–Friday, with different attack categories
Gesture PhaseEarlier temporal subsequencesLater temporal subsequences

These splits produce task-relevant feature, time, or category shifts. We will make these split protocols explicit in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer rxUB (3/3)

[R5 & A5] Our prior interface can also accept qualitative directional and invariance relations, not only numerical bounds. These relations specify how predictions should change, or remain unchanged, without user-supplied output ranges. We evaluate qualitative invariance priors on three controlled synthetic regression datasets, with five settings in total. The supplied priors state that changing the acquisition channel should not change the prediction, swapping equivalent sensors should preserve the prediction, and an auxiliary reading should not matter while the machine is idle. Sensor exchange and conditional invariance each use two independent data instances. The user supplies no output values, output bounds, or target labels. Numerical rule values are inferred from the training context and a proxy of unlabeled PFN predictions. For these qualitative evaluations, we use a point-prediction extension of PriorBridge that aggregates five generated predictions by their pointwise median. To compare the same supplied prior without PriorBridge, we introduce a pairwise Direct baseline. It averages the two Base predictions in each applicable pair and leaves active-mode predictions unchanged in the conditional task. Relation compliance is checked on comparable input pairs. Channel pairs flip only the recording-channel sign. Sensor pairs swap the two readings while keeping context fixed. Conditional pairs flip only the auxiliary reading in idle mode. For the applicable pairs P\mathcal{P}, we compute Crelation=1∣P∣∑(x,x′)∈P1 ⁣[∣y^(x)−y^(x′)∣≤0.05]C_{\mathrm{relation}} = \frac{1}{|\mathcal{P}|} \sum_{(x,x')\in\mathcal{P}} \mathbf{1}\!\left[|\hat{y}(x)-\hat{y}(x')|\leq 0.05\right]. This check does not use target labels.

Dataset / settingBase MAESame-prior pairwise Direct MAEOur method MAE
Acquisition-channel shift2.1421.0700.558
Equivalent-sensor exchange, instance 12.4571.2430.045
Equivalent-sensor exchange, instance 22.3761.2060.050
Conditional invariance while idle, instance 12.5221.2790.998
Conditional invariance while idle, instance 22.2261.1410.795

Our method improves MAE over Base and pairwise Direct in all five settings, and its aggregated predictions achieve 100% relation compliance. These results show that qualitative invariance information can improve prediction quality without user-supplied numerical bounds. Together with the directional-language ablation in R2 and A2, they support the usefulness of non-numerical prior information. PriorBridge exploits this information for proxy-guided refinement beyond numerical clipping or pairwise averaging alone.

Reviewer_aBmMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes PriorBridge, a framework for incorporating user-specified domain priors into the prediction of PFNs. The framework first fits a small tree ensemble to train samples with PFN predictions to model the PFN's decision routes. These routes are then exported as readable rules. An LLM receives these rules, along with user-specified priors and returns rule trees whose leaves follow the allowed output intervals. Finally, the PFN predictions are projected onto the intersection of the intervals from all T trees.

Critical strengths
  1. The paper is well-motivated. PFNs are increasingly used but provide no way of incorporating domain knowledge or hard requirements. The use cases in figure one and realistic and interesting.

  2. The method is easy to adopt, and does not require any training. The method treats the PFN as a black box and never touches its weights. It could therefore use any PFN, or any predictor, which makes it easy to adopt.

Critical weaknesses
  1. Main concern: The compliance metric in underspecified and semi-circular. If compliance is computed against the same ranges given to the LLM, then the compliance number mostly measures how faithfully the LLM copies the numbers correctly. In step 2, the LLM converts the priors into a tree. In step 3, the projection forces every prediction into the LLM-copied interval regardless of its original value, and in step 4, the evaluator checks the same interval. After step 3, compliance of ~100% is true by construction. Any sub-100 compliance score simply shows that the LLM made a mistake during copying. It seems like the LLM narrows the bands beyond the specified priors which explains the below 100% compliance in some cases.

  2. The introduction mentions that the goal of PriorBridge is to turning natural language priors into constraints. However, the priors shown in the experiments are already numeric and executable (e.g. When 25 ≤ AT < 28, PE should be in [428, 460].). It is important to use these to clip the PFN output directly into the specified ranges, without using any proxy or LLM. Without this baseline, the value of the proxy and the LLM is not clear. Moreover, the paper does not explain how the priors are written.

Questions for authors
  1. Is compliance computed against the same numeric ranges supplied to the LLM? If so, please explain what the compliance metric measures beyond faithful transcription, and compare against direct projection onto those ranges.

  2. The datasets used for OOD evaluation are not inherently OOD. How did you curate the train and test sets to make sure they have different distributions?

  3. The results show large, unexplained losses (TabPFN’s California Housing and Beijing) . If the priors are valid domain knowledge, enforcing them shouldn’t cost this much. Can you provide more explanations about these drops?

  4. Please discuss related work on incorporating domain knowledge into tabular models, such as LLM-based prior elicitation (e.g., LLM Processes (Requeima et al.), LoID (Rezaeimanesh et al.), AutoElicit (Capstick et. al.)), and how PriorBridge relates to them.

  5. Can you explain how T, proxy, depth, leaf-count, and the LLM and chosen?

  6. Is the performance of PriorBridge sensitive to LLM prompts?

3 comments
AuthorsOpenReview ↗

Response to Reviewer aBmM (1/3)

We would like to thank you for your detailed review and helpful suggestions. We provide targeted additional experiments on representative datasets to address the main empirical concerns. Please see our point-by-point responses to the weaknesses (R) and questions (A) below. For clarity, we have adjusted the order of our responses and discuss the prior types (R2) before compliance and direct projection (R1 and A1).

[R2] Thank you for raising this point. We would like to respond from two aspects: prior construction and qualitative-prior experiments.

Prior Construction.

Our prior interface is not restricted to user-provided numerical output bounds. The priors we consider include numerical requirements and qualitative relations:

  • Numerical requirements: specify an allowable output range under stated feature conditions. For example, power output should lie in [428, 460] MW when 25°C ≤ AT < 28°C.
  • Directional requirements: specify how a prediction should change as a feature increases, with other conditions fixed. For example, increasing ambient temperature should not increase power output.
  • Invariance requirements: specify changes that should leave the prediction unchanged. For example, exchanging equivalent sensors should not change the prediction.

The latter two specify relations between predictions without requiring the user to supply output values or ranges. We have added qualitative-relation experiments to evaluate priors of this form, as detailed below. The numerical examples in the figures make the tree and projection operations easy to visualize. They illustrate one prior type rather than define the full scope of the prior interface. We will make this distinction explicit in the presentation.

A fixed numerical interval-clipping baseline assumes that executable bounds have already been specified. When a qualitative prior provides no output bounds, this baseline cannot directly execute the requirement, such as a directional or invariance relation. PriorBridge uses the natural-language instruction and proxy summary to construct feature-conditioned correction rules. Its contribution is this prior-to-rule construction, rather than clipping alone.

Regarding prior construction, the information sources include domain relations, training-context statistics, and user-specified requirements. Some priors were constructed with reference to held-out target information. We jointly report compliance and predictive performance to assess how effectively PriorBridge incorporates the supplied prior information. We will clarify the construction protocol and these information sources in the appendix.

Qualitative-prior experiments.

We evaluate our method using only qualitative invariance priors. The user supplies no output values, output bounds, or target labels. We use TabPFN on three controlled synthetic regression datasets. They cover acquisition-channel shift, equivalent-sensor exchange, and conditional invariance while idle. The channel task reverses a spurious recording-channel correlation. The sensor task exchanges the roles of two equivalent sensors. The idle-mode task reverses an auxiliary-sensor correlation only while idle. To evaluate the same supplied prior without our method, we introduce a pairwise Direct baseline. It averages the two Base predictions in each applicable pair. In the conditional task, it leaves active-mode predictions unchanged. Relation compliance is checked on comparable input pairs. Channel pairs differ only in the sign of the recording-channel reading. Sensor pairs exchange the two readings but keep context fixed. Conditional pairs flip only the auxiliary reading in idle mode. For the applicable pairs P\mathcal{P}, we compute Crelation=1∣P∣∑(x,x′)∈P1 ⁣[∣y^(x)−y^(x′)∣≤0.05]C_{\mathrm{relation}} = \frac{1}{|\mathcal{P}|} \sum_{(x,x')\in\mathcal{P}} \mathbf{1}\!\left[|\hat{y}(x)-\hat{y}(x')|\leq 0.05\right]. This check does not use target labels.

datasetBase MAESame-prior pairwise Direct MAEOur method MAE
Acquisition-channel shift2.1421.0700.558
Equivalent-sensor exchange, instance 12.4571.2430.045
Equivalent-sensor exchange, instance 22.3761.2060.050
Conditional invariance while idle, instance 12.5221.2790.998
Conditional invariance while idle, instance 22.2261.1410.795

Our method improves MAE over Base and Direct in all five settings. It also achieves 100% relation compliance. Together with the directional-information ablation in A6, these results show that qualitative priors can improve prediction quality. Our method exploits this information for proxy-guided refinement beyond numerical clipping or pairwise averaging alone.

AuthorsOpenReview ↗

Response to Reviewer aBmM (2/3)

[R1 & A1] Thank you for this concern. We distinguish two questions: whether predictions satisfy the specified prior, and whether the correction improves prediction quality beyond directly executing that prior.

Compliance. For numerical priors, we check predictions directly against the original bounds in the corresponding feature regions. For qualitative priors, we check the required relations between predictions on comparable inputs. Dataset-level compliance is the unweighted mean across priors.

Direct projection. For numerical priors, we added a baseline using exactly the original regions and bounds, without a proxy or an LLM. The baseline uses the same test sets and original numerical requirements as the manuscript.

DatasetBase MAEDirect MAEPriorBridge MAEDirect / PriorBridge original-bound compliance
CCPP4.494.553.95100% / 100%
Airfoil5.795.244.74100% / 100%

PriorBridge has lower reported MAE than Direct on these two tasks, while both satisfy the original numerical bounds. Direct already fulfills the supplied numerical ranges. The additional effect is local refinement informed by proxy/LLM context. The CCPP correction audit records 38/2769 changes by Direct and 682 by PriorBridge, including 644 predictions that already satisfy all applicable original bounds. These extra restrictions can improve accuracy, but are not logical consequences of the original broad ranges and can also be harmful, as discussed in A3. Thus, we evaluate compliance and predictive quality separately rather than use high compliance alone as evidence of the framework's additional value.

[A2] OOD refers to an explicit difference between context and target subsets, not an intrinsic property of a dataset name. The feature-based splits are provided as follow.

DatasetContext subsetTarget subset
AirfoilFrequency ≤2000Frequency >2000
CCPPAT <25°CAT ≥25°C
ConcreteCuring age ≤28 daysCuring age >28 days
Gas Turbine NOxAT ≤15°CAT >25°C; intermediate temperatures excluded
Seoul BikeTemperature ≤20°CTemperature >20°C
SuperconductivityNumber of elements ≤4Number of elements ≥5
Dry BeanLower-eccentricity groupHigher-eccentricity group; boundary approximately 0.76444
Bike Sharing20112012
CICIDS2017Monday–WednesdayThursday–Friday, with different attack categories
Gesture PhaseEarlier temporal subsequencesLater temporal subsequences

These splits produce task-relevant feature, time, or category shifts. We have added these split protocols into the revised paper.

[A3] Thanks for your observation. In our method, an inaccurate generated leaf interval can induce an excessive correction, even when the original numerical prior is satisfied. We traced and analyzed the predictions on California Housing and Beijing PM. The results point to overly restrictive generated bounds as an important source of the degradation.

In California Housing, the original prior imposes no numerical bound on a sample, but a generated leaf introduces a lower bound of 200,000. PriorBridge therefore moves the PFN prediction from approximately 51,922 to 200,000, whereas the target is 42,100. A similar effect occurs in Beijing PM. For a sample with no applicable original numerical bound, a generated leaf sets a lower bound of 200, moving the PFN prediction from approximately 10.48 to 200 despite a target value of 6. Therefore, these examples illustrate how inaccurate local intervals can amplify prediction error rather than reduce it.

In future work, we will explore more reliable interval generation and more conservative correction strategies to improve PriorBridge and reduce such errors.

[A4] Thank you for the recommendation. LLM Processes constructs predictive distributions from natural-language descriptions. AutoElicit elicits priors over model parameters, while LoID extracts informative priors for Bayesian logistic regression from LLM token-level predictions. These methods share a focus on obtaining predictive information from LLMs. PriorBridge instead translates user-specified priors into executable rules to correct an existing PFN's predictions, without retraining the predictor. We have added this discussion and the corresponding citations to the Related Work section of the revised paper.

AuthorsOpenReview ↗

Response to Reviewer aBmM (3/3)

[A5] Our proxy configuration follows previous work such as DeLTa [1]. We use a random-forest proxy with five trees, maximum depth 10, and at most 50 leaves per tree, and set T=5. For larger datasets, we increase the leaf limit to 200 to enable the tree model to better approximate the PFN's predictions.

We also evaluate T=1,3,5 on four TabPFN tasks, using the same PFN predictions and proxy and applying the first T refined trees.

DatasetMetricT=1T=3T=5
CCPPMAE ↓\downarrow4.444.233.95
AirfoilMAE ↓\downarrow4.784.754.74
Gas Turbine NOxMAE ↓\downarrow13.912.512.5
QSARAccuracy ↑\uparrow0.8740.8740.874

In these evaluated settings, T=5 achieves the best or tied-best scores. Most tasks show limited gains beyond T=3. This supports T=5 as a practical choice for the current setup.

[A6] We evaluate the effect of directional information through a prompt-content ablation on CCPP dataset. We keep the numerical priors, PFN predictions, and proxy fixed, and remove directional language from the prompt. For the sentence-order comparison, the same prior statements are rearranged without changing their content. For the redundancy comparison, the same statements are repeated without adding new constraints.

Prompt variantMAE ↓\downarrowOriginal-bound compliance
Without directional language4.28100%
Reordered prior statements3.98100%
Redundant repetition4.00100%
Full prompt3.95100%

Removing directional language increases MAE from 3.95 to 4.28, while numerical compliance remains unchanged. These results support the value of directional information for local refinement beyond numerical compliance. The sentence-order and redundancy variants differ from the full prompt by only 0.03 and 0.05 MAE, respectively. This suggests limited sensitivity to these content-preserving prompt changes in our method.

[1] Ye, H., Li, J., Zhao, H., Guo, D., & Chang, Y. (2026). LLM meeting decision trees on tabular data. Advances in Neural Information Processing Systems, 38, 118095-118131.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.