acceptodds
Under review as a conference paper at ICLR 2027

XPFN: Learning Zero-Shot Tabular Prediction and Explanation from Prior-Sampled Response Functions

Abstract

Tabular foundation models (TFMs) make predictions in a single forward pass, but explaining these predictions still relies largely on expensive post-hoc attribution. Extending the prior-data fitted network (PFN) paradigm from prediction to explanation is a promising alternative, as it would let a TFM itself act as a cross-task Shapley estimator. However, the cost and error of generating attribution supervision have kept such models far below the pretraining scale of TFMs. We introduce XPFN, a TFM that produces zero-shot regression predictions together with their Shapley attributions in one forward pass. Our key design is to draw attribution supervision from the prior itself. A structural causal model generates the feature columns and a prior-sampled response function generates the target, so the function that defines each synthetic task can be queried under any feature coalition. This yields exact coalition labels without fitting a surrogate model or re-evaluating the TFM. On 40 real datasets, XPFN's direct attributions agree with the Shapley values of its own predictions (r=0.91), and its predictions are competitive with tuned tree ensembles such as XGBoost and CatBoost. Its explanations are up to 90 faster than TreeSHAP on XGBoost and over 1,700 faster than KernelSHAP on TabPFN-v3 and TabICLv2. Controlled ablations further show that our decoupled prior delivers more accurate and cheaper supervision labels than existing methods, which enables efficient pretraining of XPFN on over 10 million synthetic tabular tasks.

Primary area:
interpretability and explainable AI
Keywords:
Tabular Foundation Model, prior-data fitted networks, amortized explanation, explainable AI

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_Go6aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a tabular foundation model that jointly produces predictions and Shapley attributions in a single forward pass. The key idea is to generate attribution supervision from prior-sampled response functions, avoiding expensive surrogate fitting or repeated model evaluations. The model is evaluated on both synthetic and real-world datasets.

Critical strengths
  1. Integrating Shapley attribution into model prediction is novel. The proposed supervision strategy enables efficient joint prediction and explanation without substential additional cost of explanation estimation at inference time.

  2. The experiments are relatively comprehensive. The results demonstrate improvements in explanation efficiency while maintaining competitive predictive performance.

Critical weaknesses
  1. The impact of joint prediction and attribution is insufficiently evaluated. Since XPFN integrates attribution directly into the prediction process, its impact on prediction efficiency and accuracy should be examined in detail. The paper lacks a direct comparison of efficiency and performance with a prediction-only backbone. Moreover, the loss-weight ablation suggests a trade-off between predictive performance and attribution quality, but this is not sufficiently discussed.

  2. The reliability of the approximated attributions lacks guarantees. Although XPFN substantially improves explanation efficiency, the reliability of attribution is always more important in explainable AI. The theoretical result only characterizes the ideal minimizer of the training objective, without guaranteeing that learned attributions faithfully explain the model's own predictions or match their exact Shapley values. The approximation errors for explanation reliability need further investigation.

  3. XPFN is designed specifically for interventional Shapley values with a context-based background distribution. It is unclear whether the framework can support other Shapley formulations or background distributions. The paper should better clarify its applicability and limitations.

Questions for authors
  1. How does incorporating attribution affect prediction accuracy and inference efficiency compared with a prediction-only model?

  2. To what extent can XPFN guarantee attribution fidelity and preserve the theoretical properties of Shapley values beyond the ideal optimization setting?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Go6a

We thank the reviewer for the positive comments and the constructive questions.

W1 and Q1, accuracy. Following the reviewer's suggestion, we trained a prediction-only counterpart of XPFN-6.2M, the model of our supervision ablation, with the same prior, architecture, optimizer, schedule and 720k tasks, differing only in that the attribution losses are removed. It reaches a median Real-40 R² of 0.646, against 0.625 for the joint model XPFN-6.2M. Paired by dataset, the prediction-only model is better on 21 of 39 datasets with a median difference of only 0.002, and a Wilcoxon signed-rank test finds no significant difference (p = 0.57). At earlier checkpoints during the training, from 100k to 600k tasks, the joint model was ahead, significantly so at 400k and 600k tasks (p = 0.02 and 0.03). Incorporating attribution therefore does not consistently reduce predictive accuracy in this controlled comparison. We report this finding in Section 4.5.

W1 and Q1, inference efficiency. One forward pass of our model returns the prediction and all attributions, because the prediction is the base value plus the sum of the attributions. The prediction-only model in our experiment uses the same network, so it has the same inference cost. Compared to a single-output readout for regression only, the two output heads contribute 3.0% of the total parameters. We added this point to Section 4.4.

W1, loss weight. We rewrote the paragraph on the loss weight and moved its table to the main text (Table 5). The weight λ moves a fixed total weight between the coalition and the prediction term. At λ = 2 the prediction term is removed, and the Real-40 R² falls from 0.659 at λ = 0 to 0.601 while the self-fidelity rises from 0.809 to 0.863. The trend is not monotone, as λ = 0.5 reaches the highest R² of 0.663 with a higher self-fidelity than λ = 0 and λ = 1.25 is better than λ = 1 on all four metrics. Each setting was trained once, and λ = 1 was our default. Note that λ = 0 is not a prediction-only model, because all seven models also keep the base loss and the auxiliary attribution loss. The cost of attribution is therefore measured by the prediction-only control above.

W2 and Q2. Every Shapley axiom is a property of the exact values, so a learned estimator satisfies each of them within the error with which it approximates those values. We measured this error. On XAI-Bench we computed the exact Shapley values of XPFN's own predictions by evaluating every feature coalition. The direct attributions agree with these exact values at r = 0.978 and 0.980 for 5 and 10 features (NRMSE 0.149 and 0.155), between 0.975 and 0.982 across feature correlations of 0, 0.5 and 0.9, and between 0.979 and 0.990 across context sizes from 64 to 512 rows. On Real-40 the agreement with KernelSHAP on XPFN's own predictions is r = 0.913 (Table 3). Efficiency holds exactly by construction, and symmetry and the null-player property hold within the measured error. For a finite network there is no guarantee beyond the ideal minimizer, and the new Proposition 2 states what holds in that case. We added these results to Table 1 and Appendix C.2.

W3. XPFN implements interventional Shapley values with the context rows as background. Any value function that can be computed by querying the response function, for example a single baseline row or a background set chosen separately from the context, can supply exact labels in the same way. Conditional Shapley values would need conditional samples that our structural causal model does not provide in closed form. Only the interventional version is trained and tested, and we added a scope paragraph to the conclusion to clarify this.

We hope these revisions address the reviewer's concerns, and we would be grateful if the reviewer would consider raising the score.

Thank you for the authors' responses, which partially address my concerns. However, the results don't sufficiently characterize predictive accuracy and attribution reliability across datasets. It remains unclear how severely predictive accuracy can degrade due to joint learning and how large the attribution errors can be in the worst cases. Or are these drops hidden by aggregated results? Therefore, I maintain my original assessment.

Reviewer_BoHrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies feature attribution in zero-shot tabular prediction and proposes XPFN, a TFM that jointly performs prediction and approximates Shapley values. By incorporating the attribution process into the forward pass, XPFN avoids the repeated model evaluations required by standard post-hoc approaches and therefore achieves significant gains in explanation efficiency. The experiments section provides empirical evidence on synthetic datasets to demonstrate explanation fidelity.

Critical strengths

This paper presents an interesting idea for mechanistic explainability by regulating the output formatting of TFM, which achieves two birds with one stone by leveraging the efficiency axiom of the Shapley Value.

Critical weaknesses
  • Limited applicability with performance degradation. XPFN requires massive training and strictly depends on the capacity of TFMs, making it non-transferable to other models, especially lightweight ones. Besides, the explicit regularization on output formatting (for producing Shapley-like scores) harms the model's predictive performance. While this paper attempts to achieve both explainability and prediction tasks at the same time, it appears that XPFN compromises its performance on both fronts.
  • The experimental design is questionable.
    • Table 1 presents the results of explanation fidelity based on some ground-truth Shapley values known during data generation. However, the to-be-explained model only serves as a proxy for the underlying distribution (which is not revealed to the tested explainers), thus confounding the reported fidelity evaluation. It is unclear whether a lower score in the reported fidelity measure is the fault of the explained model (due to its poor capability) or the explainer itself.
    • Table 3 compares the attribution results across different models. This is even more questionable than the previous point. Given that different models may behave differently, it is unclear what the authors intend to show by reporting the alignment of results in very different contexts.
Questions for authors
  1. Could the authors comment on the applicability of XPFN in practical settings?
  2. Could the authors clarify the experimental designs and the messages they intended to convey through these results?
1 comment
AuthorsOpenReview ↗

Response to Reviewer BoHr

We thank the reviewer for the comments and for appreciating the idea of regulating the output format of TFMs through the efficiency axiom. We would also like to clarify that, besides the synthetic benchmark in Table 1, the paper reports prediction and attribution on our 40 real regression datasets (Tables 2 and 3).

W1, training and transfer. Pretraining is performed once by the model developer, and users only run one forward pass in their applications. Our experiments showed that the attribution supervision adds 11.2% to the per-step time of TabPFN-like regression pretraining (see Figure 3). Besides, XPFN is not tied to its own predictions. Given the context predictions of a frozen external model, it explains XGBoost, LightGBM, CatBoost, a random forest, an MLP, FT-Transformer, TabPFN-v2, TabPFN-v3 and TabICLv2 without any parameter update or retraining (Table 3), including lightweight models. We revised Sections 4.3 and 4.4 to make both points clearer.

W1, prediction. We trained a prediction-only counterpart of XPFN-6.2M, which is identical to XPFN-6.2M except that the attribution losses are removed from training, with the same prior, model architecture and 720k training tasks. It reaches a median Real-40 R² of 0.646, against 0.625 for the joint model XPFN-6.2M. In a paired comparison over the 39 Real-40 datasets, the median difference is only 0.002, and a Wilcoxon signed-rank test finds no significant difference (p = 0.57). At earlier checkpoints, from 100k to 600k tasks, the joint model was ahead. Besides, the additive output does not restrict the predictor, because each feature token attends to all other features before the readout. The attribution objective therefore does not consistently reduce predictive accuracy, and we added this comparison to Section 4.5. TabPFN-v2 has an architecture similar to that of XPFN, and XPFN is slightly more accurate than it in both settings of Table 2 (0.717 against 0.710 and 0.772 against 0.763). The remaining gap to TabPFN-v3 and TabICLv2 is thus unlikely to come mainly from attribution. It more likely reflects the prior and the scale of pretraining, which largely determine the predictive performance of TFMs and differ between XPFN and these models. The prior of TabPFN, for instance, is not fully public.

W2a. We agree that Table 1 mixes the error of the predictor with that of the explainer. For every baseline the explainer is exact at 5 and 10 features, because TreeSHAP is exact for the tree models and KernelSHAP enumerates all coalitions for the other models, so those rows contain predictor error only. We now computed the exact Shapley values of XPFN's own predictions by enumerating all coalitions in the same way. They reach r = 0.987 and 0.985 against the ground truth for 5 and 10 features, which is the predictor error, and the direct attributions reach r = 0.978 and 0.980 against them, which is the explainer error. We added this row to Table 1.

W2b. We thank the reviewer for pointing this out, and we revised the caption of Table 3 to make its design clear. Table 3 does not compare attributions across models. In every row the explainer is XPFN and the reference is the Shapley values of the predictor named in that row, on the same query and background rows. Each row is therefore a separate test of whether XPFN recovers that predictor's own Shapley values.

Q1. We thank the reviewer for this question. XPFN suits regression tables where a prediction and a per-row attribution are needed in one forward pass, and it can also explain an existing regression model from that model's predictions (Section 4.3). A user applies the pretrained model to a new table without any parameter update or retraining, and we now state this in Section 4.4.

Q2. Table 1 asks whether one forward pass on an unseen generating function recovers its attributions as well as a fitted predictor followed by an exact explainer. Table 2 places XPFN's predictions among tuned and pretrained baselines. Table 3 asks whether XPFN recovers the Shapley values of a given predictor, its own or an external one. We revised the opening of Section 4 to state what each experiment tests.

We hope these clarifications and revisions address the reviewer's concerns, and we would be grateful if the reviewer would consider raising the score.

Reviewer_1Er4OpenReview ↗
Rating
3
Confidence
2(+1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes XPFN, a model capable of simultaneously performing zero-sample table regression prediction and feature attribution. The core idea is to split the generation process of synthetic data into two parts: generating features using a structural causal model, and generating target values using independent sampled response functions. Since the response functions are known, during training, the output under different feature combinations can be directly calculated, providing supervision for attribution learning. The authors pre-trained XPFN on approximately 12 million synthetic tasks and evaluated it on synthetic benchmarks and 40 real regression datasets. The results show that the prediction performance of XPFN on real data is close to that of a tuned tree model; its direct output attributions have a high degree of consistency with those obtained by using KernelSHAP on its own predictions, and the explanation speed is significantly faster. The problems addressed in this paper are of practical significance, the method design is clear, and the experiments also show good efficiency potential. However, I am not an expert in this field, and I am unsure if this strategy is a combined improvement, or if the comparison of experimental comparison methods is appropriate. But overall, this is a qualified paper.

Critical strengths
  1. After my research, it is found that the basic model of the table needs to rely on additional post-processing explanation methods, which may result in higher computational costs. This paper attempts to enable the model to simultaneously output predictions and explanations during a single forward propagation.
  2. The authors decoupled feature generation from response function generation, directly querying the response function of the synthetic task, thereby reducing the cost of fitting the proxy model or repeatedly calling the basic table model to generate supervisory signals.
  3. The final experiments covered prediction, explanation, and computational costs, being comprehensive and rich.
Critical weaknesses

This paper combines the table-based model, synthetic data prior, and Shapley attribution method. I think there is a certain element of combination in it.

Questions for authors

I hope the author can provide more details on what new method designs have been brought about by the combination of these parts. And compared to simply combining the existing methods, what is the most significant contribution of this paper?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1Er4

We thank the reviewer for the positive assessment and the kind recommendation.

XPFN does combine known parts. The additive output, the coalition loss and the backbone come from ViaSHAP and ShapPFN, FastSHAP and TabPFN, and the revision credits each of them. The new design is in the data prior. The target of each synthetic task comes from a sampled response function that stays available during training, so every coalition query has an exact answer at an extra cost of 11.2% per pretraining step, without a fitted surrogate or extra passes of the TFM. This is our main contribution, and it allowed us to pretrain on 11 million regression tasks, against 256k for ShapPFN. We revised the introduction to state this contribution more clearly, and corrected the stated pretraining scale to 11 million tasks. We hope these clarifications are helpful.

Thank you for the author's explanation which has solved all my problems. I will maintain a positive attitude and enhance my confidence.

Reviewer_RbWNOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes XPFN, a zero-shot tabular model that jointly outputs regression predictions, a baseline value, and feature-level attributions in a single forward pass. Its main design choice is to decouple feature generation from response-function generation and directly query prior-sampled response functions to obtain coalition supervision, reducing the cost of fitting surrogate models or repeatedly querying prediction functions.

Critical strengths
  1. Directly querying response functions to obtain coalition supervision is a reasonable approach to reducing the cost of attribution pretraining.

  2. The experiments include both generated-function attribution and self-attribution of the learned predictor, and compare supervision paradigms within a common task stream.

Critical weaknesses
  1. The joint prediction-and-explanation framework has direct precedents. FastSHAP uses Shapley-kernel-weighted coalition regression. ViaSHAP learns feature contributions that sum to predictions. ShapPFN uses baseline and attribution heads with coalition-consistency training. ExplainerPFN studies zero-shot attribution across tasks. The clearest contribution of XPFN is therefore the response-function prior and the resulting scale of pretraining. The paper should assess its novelty around this distinction rather than presenting joint prediction and explanation as wholly new.

  2. The connection between generated-function attribution and self-prediction attribution is not fully established for the implemented shared-parameter model. Under ideal fixed-background and posterior-averaging conditions, Shapley linearity can connect the two objectives. However, the paper does not show that its actual shared model, soft endpoint objective, and coalition-mixed inputs satisfy the conditions needed for this connection.

Questions for authors
  1. Beyond response-function supervision, regression adaptation, and scale expansion, what independent methodological contribution distinguishes XPFN from ShapPFN and other joint prediction-attribution approaches?

  2. How do the authors scope the identifiability limitation for external black-box explanations? Does prediction fit control error on the coalition-mixed input distribution?

  3. How do direct attributions and the model's own Shapley values vary with feature correlation and context size?

1 comment
AuthorsOpenReview ↗

Response to Reviewer RbWN

We thank the reviewer for the careful and insightful review. It helped us state our contribution more clearly.

W1 and Q1. We agree that joint prediction and attribution itself is not new, and the revision now states so. FastSHAP, ViaSHAP, ShapPFN and ExplainerPFN have each addressed part of this problem, and we added a table that compares them with XPFN (Appendix E). Our contribution is how the coalition labels are efficiently and effectively computed. As shown in Figure 1, the earlier methods obtain them either from the TFM's own masked predictions (ShapPFN), a fitted surrogate (FastSHAP, ExplainerPFN) or the network being trained (ViaSHAP), whereas our data prior queries the function that generates each synthetic target. The coalition labels are therefore exact, at an extra cost of 11.2% per pretraining step, against 2.18× and 8.38× for ShapPFN-style and ExplainerPFN-style labels (see Figure 3). This allowed us to pretrain on 11 million tasks, against 256k for ShapPFN and hundreds of thousands for ExplainerPFN, which brings attribution pretraining closer to the scale of leading TFMs. We revised the introduction and the related work accordingly to state this contribution more clearly.

W2. We agree that the submission stated the link only for one task, and we added Proposition 2 to Appendix D. For the population minimizer of the training objective, the direct attributions equal the posterior mean of the Shapley values of the generating function, up to a shift shared by all features that comes from the soft endpoints (Proposition 1). They equal the exact Shapley values of the model's own predictions when, in addition, the endpoints are exact, the query carries no further information about the task and the predictor equals the posterior-mean response on coalition-mixed rows. Shared parameters and the random background subsets used in training do not change this minimizer. Training does not enforce the last condition, because the model never receives a coalition-mixed row as input. We therefore state the link for the ideal case and measure the gap. On XAI-Bench the direct attributions agree with the exact Shapley values of XPFN's own predictions at r = 0.978 and 0.980 for 5 and 10 features. On Real-40 the agreement with KernelSHAP on XPFN's own predictions is r = 0.913.

Q2. Prediction fit does not control the error on coalition-mixed rows in general. XPFN sees an external model only through its predictions on the context rows, while interventional Shapley values also depend on the model's values on coalition-mixed rows, which can lie away from the data when features are dependent. Two models with the same fit can then have different Shapley values. Our results agree with this. Across the 360 model and dataset pairs behind Table 3, the rank correlation between fit R² and attribution r is only 0.17. We revised the paragraph on external models in Section 4.3 to state that XPFN returns the Shapley values of its in-context fit of the external model, and added this analysis to Appendix C.3.

Q3. We added an appendix table for this question (Appendix C.2). As the feature correlation grows from 0 to 0.9, the agreement of the direct attributions with XPFN's own exact Shapley values stays between 0.975 and 0.982, while the agreement of those exact values with the ground truth falls from 0.995 to 0.972. As the context shrinks from 512 to 64 rows, the agreement with the own Shapley values stays between 0.979 and 0.990, while the agreement of the own values with the ground truth falls from 0.986 to 0.926. In both cases the loss comes mainly from the predictor.

We hope these revisions address the reviewer's concerns, and we would be grateful if the reviewer would consider raising the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.