- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PriorBridge, a framework for incorporating user-specified natural-language priors into the predictions of frozen Prior-Fitted Networks (PFNs) without parameter updates.
The framework first queries a PFN on unlabeled target inputs and fits a tree-based proxy to the resulting predictions. The proxy's decision paths provide a language-readable description of the PFN's local predictive behavior. An LLM then combines these paths with feature descriptions and user-specified priors to generate rule trees whose leaves specify admissible output intervals. At inference time, the intervals obtained from multiple LLM-generated trees are intersected, and the original PFN prediction is projected onto the resulting range. For classification, the projection applies to a designated class probability.
The paper evaluates TabPFN-v2 and LimiX-2M on 21 tabular datasets with 55 constraints, spanning IID and OOD settings. It reports substantial increases in prior compliance, although predictive performance sometimes improves and sometimes deteriorates. Additional experiments include prior-source ablations, cross-model proxy rules, and sample-level correction examples.
Critical strengths
1. Meaningful problem formulation with practical relevance.
The paper addresses an important problem: incorporating human-specified domain knowledge into frozen PFNs without retraining. The distinction between predictive accuracy and prior compliance is meaningful, particularly in OOD settings where domain knowledge may complement insufficient training context.
2. Explicit and inspectable prediction correction.
The framework converts natural-language priors into executable tree-based constraints, allowing individual corrections to be traced through decision paths and output intervals. Experiments across multiple datasets and PFN backbones provide initial evidence of its practical applicability.
Critical weaknesses
1. Insufficient attribution of improvements to the proposed mechanism rather than highly informative priors.
Some supplied priors already contain precise, task-specific numerical constraints, potentially functioning as independent rule-based predictors. Without information-matched prior-only and direct-integration baselines, it is difficult to determine whether performance improvements arise from the proposed framework or simply from the additional predictive information. Moreover, the pipeline resembles existing LLM-assisted rule-manipulation approaches, while its PFN-specific contributions remain insufficiently established.
2. Insufficient characterization of semantic fidelity and system reliability.
The LLM simultaneously interprets human priors, modifies tree structures, and generates refined numerical bounds, making it unclear whether the resulting constraints faithfully preserve the original requirements or introduce unsupported assumptions. In particular, narrower intervals do not necessarily improve correctness, pointwise constraints do not generally guarantee monotonicity, and intersecting multiple LLM-generated intervals may amplify over-constraining errors. More systematic component ablations, sensitivity analyses, and failure-case investigations are needed to establish the reliability of the overall framework.
Questions for authors
Overall
I find the problem relevant, and I appreciate the idea of making prior-guided corrections explicit and executable. However, the current empirical evidence does not yet convince me that the proposed integration mechanism contributes substantially beyond the predictive information already supplied by the priors, or that the generated rules reliably preserve the original constraints. I therefore lean toward a 2 (weak rejection) in this round.
Q1 and Q2 are the most important questions for my assessment, since they concern the provenance and information content of the priors and whether the reported improvements can be attributed to the proposed method. Q3 and Q5 are also potentially score-changing, because they concern the validity of the reported compliance and the observed failures. Clear evidence addressing these concerns could substantially improve my assessment.
Q4, Q6, Q7, and Q8 would strengthen the explanation of the method and its scope, but I do not expect all suggested experiments to be completed in a rebuttal. Q9 is primarily a presentation suggestion and would not, by itself, change my score.
Questions
Q1. Clarification/Evidence: How are the priors constructed, and how much predictive information do they already contain?
Could the authors explain the complete construction protocol for the 55 priors? In particular, how were numerical thresholds, output ranges, feature-specific conditions, and OOD descriptions selected? Which quantities came from independent domain knowledge, which were obtained from the training context, and were any priors revised after observing validation or other performance information?
Appendix C includes relatively precise, region-specific ranges for CCPP. How representative is this level of precision across the evaluated datasets? For classification, what fraction of samples have their final class already determined by the original human priors alone? For regression, how predictive are the supplied bounds without using the PFN or the LLM-refined tree?
I am not suggesting that strong priors are inherently invalid. Rather, these details are necessary to distinguish integration of qualitative knowledge from a setting where the supplied prior already serves as a powerful rule-based predictor. They are also important for interpreting the actual information budget of the experiments.
Could the authors also clarify whether the priors are intended to constrain individual predictions, conditional expectations, physical response bounds, or application-specific decision policies? These are not always equivalent, particularly when a prior is motivated by a causal or physical statement.
Q2. Evidence/Ablation: What is the incremental contribution of PriorBridge under exactly matched prior information?
Could the authors compare against methods given the same original prior instances, including their exact regions, thresholds, numerical bounds, and exceptions?
I would particularly like to see an original-prior-only baseline, direct prior execution on the PFN outputs, and a variant where the LLM receives the same priors and metadata but no proxy rules. A fixed-proxy-structure variant that only generates leaf intervals would further separate structural rewriting from numerical prior assignment.
For explicitly numerical priors, direct execution could be implemented without LLM-based tree rewriting. For monotonic priors, an information-matched comparison could enforce the original ordering relations directly rather than inventing numerical bands from the context. A manually formalized version could be treated as a semantic-oracle ablation, isolating the contribution of language understanding from that of the subsequent correction algorithm.
These comparisons would make it possible to determine whether the main gains arise from the extra domain information, the use of the original PFN predictions, or the specific proxy–LLM–projection mechanism. It would also be important to compare predictive performance at similar levels of original-prior compliance.
Q3. Clarification/Evidence: How is semantic fidelity between the human priors and the generated rules established?
The LLM modifies both the input partitions and the numerical leaf intervals. What conditions guarantee, or at least empirically verify, that the resulting rule tree preserves the requirements expressed by the user?
Could the authors distinguish between restrictions logically implied by the original prior and additional restrictions inferred from the proxy or the LLM's pretrained knowledge? How are unsupported narrower intervals identified? In particular, how should the additional narrowing shown in Figure 3 be interpreted when the original prediction is already within the stated broad numerical range?
I am also concerned about qualitative constraints such as monotonicity. Pointwise interval satisfaction does not, in general, guarantee pairwise ordering of predictions. Is monotonicity evaluated using controlled comparable pairs or through a surrogate based on the generated bands? Could the authors provide the independent evaluation predicates for the original prior semantics, rather than only verifying membership in the generated intervals?
This is central to the claim that the method enforces user-specified knowledge instead of merely enforcing constraints constructed by the LLM itself.
Q4. Clarification: What happens when the PFN, the supplied priors, or the generated rules disagree?
When a PFN prediction strongly contradicts a supplied human prior, does the method always give priority to the generated interval? Does the procedure distinguish a likely PFN error from a potentially incorrect or locally inapplicable prior, or is its intended role strictly to enforce user requirements regardless of predictive accuracy?
What happens when multiple human priors are mutually inconsistent, or when separately generated rule trees produce an empty intersection? Appendix A.2 describes filtering structurally invalid trees, but how are semantically incompatible yet structurally valid trees handled? What happens if no valid candidate remains?
For multiclass classification, how are simultaneous constraints on different class probabilities implemented? In particular, can proportional rescaling for one class violate an already-satisfied constraint on another class, or make the result dependent on the order of corrections?
Finally, is the present framework intended only for hard constraints? It would help to state explicitly whether soft violations, graded prior reliability, or a controllable violation budget are supported. I do not regard support for every such extension as necessary, but the intended semantics should be clear.
Q5. Evidence/Failure analysis: What are the actual sources of unsuccessful or unnecessary corrections?
Could the authors provide a failure analysis organized by correction type, rather than relying mainly on average compliance and predictive metrics?
Several existing results warrant explanation. For LimiX on Gas Turbine NOx, reported compliance is 100% -> 100%, yet MAE worsens from 6.39 to 7.32. For TabPFN on QSAR Biodegradation, corrected compliance remains 75.0%. Conversely, LimiX on Dry Bean improves accuracy from 0.610 to 0.840 under one supplied prior. How much of the latter gain is already determined by the prior, and what accounts for the former failures?
It would be useful to distinguish original-prior violations from generated-interval violations, as well as cases where the base PFN was already compliant but the method nevertheless changed its output. For classification, reporting incorrect-to-correct and correct-to-incorrect changes would make the net accuracy gains more informative. For regression, the distribution of correction magnitudes and the frequency of harmful modifications would be valuable.
A small number of detailed failure traces—from original prior, through proxy path and LLM edits, to final projection—would help identify whether the bottleneck lies in understanding, representation, aggregation, or execution.
Q6. Evidence/Ablation: How do proxy fidelity, tree complexity, and the tree-learning procedure affect downstream correction?
The method relies on a proxy forest whose paths are presented to the LLM. How does correction quality change with maximum tree depth, leaf budget, forest size, tree-learning method, and the complexity of the target data?
A more accurate proxy is not necessarily more useful for applying a domain prior: it may reproduce irrelevant teacher behavior while failing to expose the features or regions needed by the constraint. Conversely, a very detailed tree may increase prompt length and make LLM editing more difficult.
Could the authors measure both teacher–proxy fidelity and downstream correction quality, and investigate whether the former predicts the latter? For classification, what is lost by fitting the proxy to hard labels rather than full PFN probabilities, given that correction is ultimately probability-based?
The cross-model proxy results are interesting in this regard. Could the authors use them to clarify what aspects of the proxy are actually necessary for a successful correction, rather than interpreting cross-model transfer only as backbone flexibility?
Q7. Evidence/Robustness: How does the number of generated rule trees T influence the results?
The method uses T = 5 generated rule trees and intersects their intervals. Because this aggregation takes the maximum lower bound and minimum upper bound, increasing T does not necessarily average away LLM errors; it can instead make the allowed interval progressively more restrictive.
Could the authors report a sensitivity analysis over T, including predictive quality, original-prior compliance, interval width, correction magnitude, the number of valid generated trees, and the frequency of empty intersections?
How much variation arises from independently rerunning the full LLM-based construction procedure? It would be important to distinguish the performance of a fixed generated rule set from the reliability of regenerating that set.
Relatedly, are the corrections stable under semantically equivalent paraphrases, changes in prior order, redundant constraints, and a null or nonrestrictive prior? These tests could help distinguish sensitivity to the actual constraint semantics from sensitivity to incidental prompt wording.
Q8. Clarification: What distinguishes this contribution from existing LLM–tree pipelines, and what is specific to PFNs or the intended deployment setting?
Could the authors provide a more direct comparison with the closest LLM-assisted tree and neuro-symbolic methods, including DeLTa and FORESTLLM? It would be useful to distinguish the information provided to each method, the role of the LLM, the source of the correction signal, and the semantics of the final executable representation.
More specifically, which properties of PFNs are essential here beyond supplying high-quality initial predictions? The proposed operations seem applicable to many black-box predictors. If the method is intentionally model-agnostic, that positioning would be reasonable; if PFN-specificity is central, I would like to understand how context-conditioned adaptation, context coverage, or PFN-specific uncertainty affects the mechanism.
There is also a transductive aspect: the proxy is fitted using predictions on the current unlabeled target inputs. Are the rules intended for a fixed target batch, or can they be reused on independent future inputs? How does performance change if the rules are constructed on one unlabeled batch and then frozen for another? Please also clarify the setup cost and when proxy fitting or LLM generation must be repeated.
Q9. Writing and reproducibility
The overall pipeline is reasonably clear, but a complete worked example would substantially improve the presentation. In particular, a side-by-side visualization of the original proxy tree and the LLM-refined tree would help readers identify what is preserved, what is modified, and what additional information enters the output intervals.
I would also recommend a compact notation table distinguishing the PFN predictive output, proxy prediction, original human-prior feasible set, generated leaf interval, and final projected output. These objects currently appear in different stages of the paper but play importantly different roles.
Finally, the manuscript states that implementation code is supplied in the supplementary materials. Could the authors point to the exact locations of the complete prompts, original-prior definitions, evaluation predicates, and handling of empty or invalid rule sets?
8 comments
Response to Reviewer iJzs (2/4)
[R1 & A2] Thank you for requesting comparisons under the same supplied priors. We address the simpler baselines and the role of proxy-guided construction separately.
Same-prior baselines and predictive value.
Could the authors compare against methods given the same original prior instances, including their exact regions, thresholds, numerical bounds, and exceptions?
We compare PriorBridge with Direct using the same original numerical priors. Direct clips PFN predictions to the supplied bounds in their corresponding feature regions, without a proxy or an LLM. The prior-only midpoint results are given in [R1 & A1] above. Base and PriorBridge scores follow the results reported in the paper.
| Dataset | Base MAE | Original-prior Direct MAE | PriorBridge MAE | Direct / PriorBridge original-bound compliance |
|---|---|---|---|---|
| CCPP | 4.49 | 4.55 | 3.95 | 100% / 100% |
| Airfoil | 5.79 | 5.24 | 4.74 | 100% / 100% |
Direct and PriorBridge both achieve 100% original-bound compliance, while PriorBridge has lower reported MAE than Direct and the prior-only midpoint baseline. On CCPP, Direct's reported MAE is even higher than Base's (4.55 vs. 4.49). Clipping to broad intervals is a coarse correction that does not further refine predictions within those bounds. Thus, we evaluate compliance and predictive quality separately rather than use high compliance alone as evidence of the framework's additional value.
Proxy-guided prior-to-rule construction.
I would particularly like to see an original-prior-only baseline, direct prior execution on the PFN outputs, and a variant where the LLM receives the same priors and metadata but no proxy rules.
We have added a no-proxy ablation that preserves the original priors and metadata but removes the proxy rules. We also remove directional language while retaining the numerical bounds, proxy rules, and metadata. The CCPP results are summarized below, with the paper's Base and full-version scores included as references.
| Variant | MAE | Original-bound compliance |
|---|---|---|
| Base | 4.49 | 95.4% |
| Original-prior Direct | 4.55 | 100% |
| Without proxy rules | 4.55 | 100% |
| Without directional language | 4.28 | 100% |
| Full version | 3.95 | 100% |
Without proxy rules, the reported MAE is essentially the same as Direct's (4.55). Without directional language, the reported MAE is 4.28. Both ablations achieve 100% numerical compliance, yet have higher reported MAE than the full version. These results support the usefulness of proxy context and semantic prior information for local refinement beyond bound enforcement.
The components have complementary roles. The prior specifies the desired behavior, the proxy summarizes the PFN's feature-conditioned predictions, and the LLM translates them into executable correction rules. The proxy is fitted to PFN predictions on the unlabeled inputs to be corrected, without their target labels. The final correction retains the original PFN prediction and projects it into the generated interval, rather than replacing it with the proxy prediction.
Response to Reviewer iJzs (3/4)
[R2 & A3 & A5] Thank you for requesting semantic checks and explanations of the highlighted cases. We distinguish evaluation against the original prior, the role of local refinement, and the specific prediction outcomes below.
Independent evaluation of prior semantics (A3).
Could the authors provide the independent evaluation predicates for the original prior semantics, rather than only verifying membership in the generated intervals?
Numerical compliance is checked directly against the original bounds in the applicable feature regions, rather than against the generated leaf intervals. For classification, the check uses the class condition specified by the prior. This evaluates whether the final prediction satisfies the supplied requirement independently of the rule used to produce it.
Qualitative relations require checks on comparable inputs. Our controlled invariance tests use a point-prediction extension and compare final predictions on pairs that differ only in the transformation covered by the prior, such as exchanging equivalent sensors. A pair is compliant when . This check does not use target labels. Directional priors instead require an ordering predicate on such pairs, not an interval-membership check. We will make the evaluated prior types and predicates explicit, distinguishing numerical-bound compliance from qualitative relation compliance.
Interpretation of additional local refinement (A3).
In particular, how should the additional narrowing shown in Figure 3 be interpreted when the original prediction is already within the stated broad numerical range?
In Figure 3, the broad range expresses the original requirement, while the narrower leaf interval is a feature-conditioned refinement constructed from the prior and proxy summary. The broad prior defines admissibility, and the local interval supplies an additional predictive estimate. This refinement is not simply a restatement of the original bound. It can change an already-compliant prediction while keeping it within that bound. We assess its usefulness through predictive performance alongside original-prior compliance, as in [R1 & A2]. The NOx case below illustrates why the accuracy of the local interval matters.
Response to Reviewer iJzs (1/4)
We thank you for your detailed review and for clearly prioritizing your concerns. We have carefully considered all questions, including those on presentation, reproducibility, and other supporting aspects. Given their breadth and the limited rebuttal period, completing separate experiments and detailed responses for every point is not feasible. Following your prioritization, we focus below on prior provenance and information content (A1), same-prior integration controls (A2), semantic fidelity (A3), and failure analysis (A5). These answers target the central concerns in your two critical weaknesses, with A3 and A5 combined where the same correction analyses address both questions. We will continue addressing the remaining comments and incorporate completed revisions into the revised paper.
[R1 & A1] Thank you for emphasizing the information budget of the priors. We address their construction and predictive information separately below.
Prior construction and information sources.
Regarding prior construction, the information sources include domain relations, training-context statistics, and user-specified requirements. Some priors were constructed with reference to held-out target information. We jointly report compliance and predictive performance to assess how effectively PriorBridge incorporates the supplied prior information. We will clarify the construction protocol and these information sources in the appendix.
PriorBridge aims to improve predictions and satisfy user requirements through additional language priors. The OOD experiments evaluate prior-assisted correction, rather than generalization without extra information. The LLM receives language constraints and PFN-derived rules, not per-example test labels. This reflects practical settings where domain knowledge or user requirements can guide predictions.
How predictive are the priors on their own?
We have added prior-only baselines that do not use PFN predictions or LLM-generated rules. For regression, the baseline predicts the midpoint of an applicable finite prior interval. For Dry Bean, it predicts the class specified by the prior in the applicable region and uses the training-majority class elsewhere.
The Dry Bean prior specifies a class for 24.14% of the target samples, and that class is correct on 96.29% of the covered subset. It therefore supplies highly useful class information where applicable, while predictions elsewhere still require a model. We report both coverage and covered-subset accuracy to make this information content explicit. For CCPP and Airfoil, [R1 & A2] below compares PriorBridge with these midpoint baselines and direct execution of the same numerical priors.