- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes ShapFM, an amortised method for estimating the Shapley values of cooperative games arising in machine learning. ShapFM is a prior-fitted, transformer-based model that handles games with up to 32 players. Unlike FastSHAP, which must be trained per model, ShapFM is trained once and can be reused across different games, models and datasets. Unlike ExplainerPFN, it conditions on evaluations of the game itself, so its estimates improve as more coalitions are observed.
ShapFM is trained only on about 960,000 synthetic games with exact Shapley values, drawn from four families: sums of unanimity games, Gaussian processes on the hypercube, TreeSHAP games of tree ensembles fitted on synthetic data, and do-Shapley games on linear causal models.
The input is a coalitions × players grid of observed coalition values, plus two optional inputs: related games and, for tabular data, the explicand with reference rows. One forward pass yields 999 quantiles per player, whose mean satisfies efficiency.
Experiments cover games with up to 24 players (tabular, image, text, hyperparameter importance and SOUM), against sampling-based estimators and the amortised explainers FastSHAP and ExplainerPFN.
Critical strengths
- The framing is interesting: it bridges the foundation-model literature with amortised Shapley estimation, combining reuse across games with refinement from coalition evaluations, without per-model training.
- A single checkpoint can potentially be applied to many cooperative games, not only feature attribution (e.g., hyperparameter importance and SOUM games).
- The method is compared against recent baselines, including OddSHAP, RegressionMSR, ShaplEIG, FastSHAP and ExplainerPFN.
- It outperforms the baselines in the low-budget regime (1.5 to 3.6× lower error at p + 2 evaluations on tabular games), which may matter in applications where evaluations are expensive.
- Its inference runtime is comparable to that of existing estimators (53 ms without context, 123 ms with it) despite the larger context, and far below ShaplEIG's.
Critical weaknesses
- The approach bets that real ML games resemble the synthetic, mostly low-order training games, leaving it open to "adversarial" games that behave very differently.
- The evaluation is limited to games with at most 24 players (training to 32), well below many practical settings.
- The advantage diminishes as the number of sampled coalitions grows, and classical estimators overtake it at larger budgets.
- The method provides no theoretical bounds on its approximation error, unlike, e.g., LeverageSHAP.
- Methods are compared only at an equal number of evaluations; a comparison at an equal time budget would be more informative.
- The FastSHAP comparison may not be entirely fair: it uses a fixed 4,096 training rows, although FastSHAP's error should decrease with more training data, and its very cheap inference lets its training cost be amortised over many explanations.
- The motivation is not entirely clear. It is unclear which real-world applications call for this trade-off: reasonably accurate explanations from very few evaluations, for models with few features, at the cost of relying on a learned prior. This reliance may be undesirable in high-stakes domains such as credit decisioning, where explanations should be faithful to the model rather than shaped by a synthetic prior.
- The uncertainty represented by the 999 output quantiles is never analysed (e.g., calibration), although it is presented as a feature of the method.
Questions for authors
- Regarding the Gaussian process games, the paper states: "The family has no dummy players and no symmetric pair, and all distributions were calibrated on a small set of real precomputed training games." What are these games? If they overlap with the datasets used for evaluation, there could be leakage. How does ShapFM perform without this leakage, if any?
- Could the authors provide more details on the training cost of FastSHAP vs. ShapFM: training time, GPU memory requirements, which GPU was used, etc.?
- Why are no ShaplEIG results reported for the tabular games in Table 1, Figure 4 and elsewhere in the paper? ShaplEIG appears to be the closest competitor (modulo its high computational cost), so it would be valuable to see how it compares with the proposed method on tabular games.
- Beyond the runtime at a fixed evaluation budget, could the authors report the approximation error at a fixed runtime budget, to better compare estimators? Can more light be shed on the training cost of FastSHAP compared to ShapFM?
- ShapFM should work well when the fitted prior (mostly low-order interaction games) matches the game at hand. What happens when it does not?
- Could the authors further motivate which applications would benefit from the method? It is not immediately clear where obtaining amortised Shapley values from very few sampled coalitions with a prior-fitted model is beneficial in real-world applications.
- Connected to the above. Can anything be said about the theoretical guarantees the proposed framework provides?
- How well do the 999 quantiles capture the uncertainty of the Shapley values? Does the predicted distribution widen when an unusual game is provided, either an unusual sample within a dataset or a dataset that is unusual with respect to the learnt prior?
6 comments
Q5: ShapFM should work well when the fitted prior (mostly low-order interaction games) matches the game at hand. What happens when it does not?
Our evaluation already covers this case, since many of the benchmark games differ substantially from the prior. No game in our evaluation was seen in training. Most also differ from the prior in distribution, as we detail below. The two “in-prior” families are less in-prior than they appear. The tree family of the prior is fitted on synthetic data only, and the evaluation uses real datasets only (Section E.2). The sum-of-unanimity (SOUM) games draw 30 or 150 terms of order up to 5 or 15, whereas the prior draws at most 12 terms per block of order at most 6 (Table 3). Thus, the SOUM games lie well outside of what is seen during training. Still, ShapFM has the lowest error on them at 2p (NMSE 0.390 and 0.462 vs. 0.476 and 0.540 for the best estimator, Table 11).
Several evaluation families have no counterpart in the prior at all. The deep learning games already differ from the prior quite substantially in the model class (ResNet, ViT, DINOv2, DistilBERT). Of these six game families, ShapFM is still the most accurate estimator at budget 2p and at 4p on five of six, where only ShaplEIG is ahead on ResNet-18 (Table 11), but with drastically higher computation time. We will further strengthen confidence in our model's generalization capability with more experiments on these deep learning explainability games.
Q6: Could the authors further motivate which applications would benefit from the method? It is not immediately clear where obtaining amortised Shapley values from very few sampled coalitions with a prior-fitted model is beneficial in real-world applications.
Indeed, ShapFM’s main strength is data-scarce applications where no closed-form solution is available. We see our approach as particularly advantageous when observing coalitions is costly and we have access to context such as related games.
Three concrete examples, which we also study in the paper, are: (1) feature importance for ML pipelines (where obtaining coalitions requires refitting the whole pipeline), (2) hyperparameter explanations (where obtaining coalitions requires running complete hyperparameter runs), and of course (3) local explanations for black-box models such as image classifiers, language models, or tabular foundation models (where we often have access to prior explanation games on similar images / text documents / rows).
More generally, we expect that all application areas where Shapley-based explanations are needed for black-box models will benefit from a well-trained foundation explanation model such as our ShapFM. Ultimately, the goal is to have practically useful Shapley values. Meaning (a) a high-quality estimate of the Shapley values and (b) with the least amount of black-box access (since this is the thing that is costly).
Even with advances such as ShaplEIG or other proxy-based techniques like RegressionMSR or OddSHAP, traditional estimators only work well given sufficient sampling budget, as they cannot easily reuse information about the underlying explanation task or machine learning model. The answer, up to now, was to build a FastSHAP model for each individual explanation task from scratch. While technically you would no longer need any black-box access, training a good FastSHAP model is quite finicky, and practical guidance seems to only really exist for local explanations of vision models and some tabular cases.
Foundation models, such as ShapFM, bridge this gap now. Similar to FastSHAP, we can now train the Shapley explanation model on synthetic data and enrich it with as much additional context as possible such that the model already works with very little information, as you can see from our context vs. w/o context ablations (e.g., Figures 3 and 4). Unlike FastSHAP, the performance of the explanations can be assured by a central model provider training on various different game settings and scenarios and by providing additional queries to the black box, reducing the estimation error like a traditional sampling-based approximation technique.
Q7: Connected to the above. Can anything be said about the theoretical guarantees the proposed framework provides?
This is an important point we have talked about in different parts of the paper. Theoretically, ShapFM provides guarantees at two levels.
By construction, for every game and budget, it satisfies efficiency exactly (Eq. 4, Sec. 2.1). As an orthogonal projection onto the efficient hyperplane, Eq. 4 can only reduce the squared error (cf. Jethani et al., 2022, App. B). ShapFM is equivariant to player order (Sec. 2.1, B.1), and its outputs transform correctly under shifts and positive rescalings of the game (Sec. D.1). Under the conditions discussed in the general comment (C2), the optimal solution of our training objective (Eq. 12) is the quantiles of given the observations and context under the training distribution, and our point estimate, the mean of these quantiles, approximates the conditional mean of . As with FastSHAP, whose guarantees also hold relative to its training objective ([2], [3]), this gives no per-game bound.
Importantly, ShapFM is neither unbiased nor provably consistent, dummy and symmetry are only learned through the prior (Sec. C.1), and linearity is not enforced. Classical estimators, in contrast, are consistent, so they are expected to overtake ShapFM at large budgets, as our results show (Sec. 3.2, Figs 5 and 8). This is a deliberate trade-off, as ShapFM targets the low-budget regime.
Q8: How well do the 999 quantiles capture the uncertainty of the Shapley values? Does the predicted distribution widen when an unusual game is provided, either an unusual sample within a dataset or a dataset that is unusual with respect to the learnt prior?
We agree that understanding (and using) these uncertainties offers great opportunities for future research, particularly for uncertainty quantification and calibration behavior. This — from our point of view — is still quite underexplored, also for tabular foundation models.
For the revision we will add a calibration study where we will compare the predicted quantiles with the true Shapley values on a subset of our benchmark games.
We thank you for the detailed and constructive feedback and for your positive assessment of our work. Below, we respond to each question and weakness point by point. Several of your comments call for additional experiments, and we will upload a revised manuscript with these results by the end of the week. Where a response depends on these new results, we describe the planned changes here and will follow up once the revision is online. If you have any follow-up questions or suggestions in the meantime, please let us know so that we can take them into account for the revision!
The GP prior was calibrated on a subset of precomputed games available in shapiq [1], ranging from local explanations to data valuation (always the first instance (seed 1) and restricted to at most 15 players). These games are not the same as the ones used for the evaluation of ShapFM, and hence there is no risk of leakage.
Moreover, we only conduct prior predictive checks, i.e., compute summary statistics per game (e.g., sign share, size-trend) and adjust the prior’s hyperparameters until the real games’ statistics fall inside the 5-95% band of the simulated distribution. So only these statistics enter the procedure, never coalition or Shapley values.
That’s an important question! We agree that the training cost belongs in the paper and will add details to the revision.
The 32-player ShapFM model was trained on a single NVIDIA L40S GPU with 48 GB of memory. It was pretrained once, which took 11 hours and 14 minutes. The larger 128-player ShapFM model was trained for about 7 days on two NVIDIA A100 GPUs with 40 GB each, followed by a final phase of 16 hours on one NVIDIA B300 GPU, which corresponds to about 30 hours on an A100. In total, this is about 350 A100 GPU-hours, with about 27 GB of memory used per GPU.
A direct comparison with FastSHAP is difficult, because the two methods require training for different things. ShapFM is pretrained once and then explains any game without further training. FastSHAP must be trained anew for every game it explains, and its cost grows with that model. On ImageNette with 196 players, reported training times are about 2.4 hours on one Tesla V100 [2], about 19 hours on two RTX 2080 Ti [3] and about 4 hours on one RTX 2080 Ti [4].
Yes, of course! As you mention, ShaplEIG is a very slow but powerful method, and we will report ShapEIG for all experiments at the budgets p+2, 2p and 4p (Table 1, 11 and 12). Due to its computational cost, we cannot cover the full budget range, since a single ShaplEIG explanation of a game with 24 players at a budget of 16p costs about 6 CPU hours, or about 180 CPU hours for all 30 tasks, and the cost grows further up to the largest budget of 32p.
To make sure we understand this question correctly, we kindly ask for a short clarification. Figure 7 reports the time per explanation at a fixed budget of 8p evaluations. Do you mean the reverse view, where every estimator receives the same amount of time and uses as many evaluations as fit into it?