acceptodds
Withdrawn from ICLR 2027

MotionBench-XAI: Benchmarking What Shapes Shapley Explanations in Spatiotemporal Machine Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting outcomes from spatiotemporal data, such as assessing Parkinson's severity from human motion, often requires explainability in addition to accuracy. Practitioners need to know what drove the prediction, whether it be spatial (a body joint), temporal (a specific window), or both. Shapley values are attributions computed by treating input features as players in a cooperative game and measuring how the model output changes as coalitions of players are removed. Computing them involves choosing what counts as players, how absent players are imputed, and how the game is evaluated. How these choices affect explanation quality is poorly understood for spatiotemporal data, and real data carries no ground-truth attributions to check against. We present MotionBench-XAI, a benchmark built on i) five synthetic spatiotemporal datasets with exact ground-truth Shapley values, ii) three real datasets across clinical gait, electrocardiogram, and audio, iii) ten attribution methods across three player sets, and iv) three trained architectures. We show that grouping features into players, across time, space, or both, matters more than the choice of imputer. Generative imputers improve explanations only when their conditional models are accurate. On real data, the standard proxy metrics select contradictory best methods. We distill these findings into practical insights and provide an open-source pipeline.

Primary area:
interpretability and explainable AI
Keywords:
Explainable AI, Shapley, Time-series, Interpretability, Manifold, Benchmarking, Synthetic explanation, Human motion, Audio

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_erLcOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a benchmark, MotionBench-XAI, to evaluate the design choices behind Shapley explanations for spatiotemporal data (player sets, imputers, ground truths and evaluation metrics). It has five synthetic and three real datasets (clinical gait, ECG and audio). The authors vary these choices jointly. The synthetic datasets are carefully constructed so that exact marginal and conditional Shapley values can be computed and each method is graded against the ground truth of its own game. On the real data, where no ground truth exists, they use faithfulness and PlayerAOPC instead. Their main findings are that the choice of players matters more than the choice of imputer, that generative imputers help only when their conditional models are accurate, and that the two real-data metrics often disagree on the best method.

The benchmark covers ten attribution methods, including standard KernelSHAP imputers (zero, mean, marginal, empirical), temporal SHAP variants (WindowSHAP, TimeSHAP), and generative imputers (KS-VAEAC, KS-Gauss and a new flow-matching imputer, KS-Flow). These are evaluated on three player sets, namely time windows, joints, and joint x window cells with up to 68 players. Each synthetic dataset is explained for an MLP, a CNN and a Transformer, and the real datasets use pre-trained or fine-tuned models such as MotionBERT and the Audio Spectrogram Transformer. All methods use the same coalition designs and budgets, and an oracle imputer is included as a reference. Ground truths are exact for up to 12 players, while for larger player sets the authors measure the remaining grading error. Classifiers must also pass an accuracy and overfitting criterion. Finally, a hide-one-reconstruct check with donor and unconditional controls tests whether the generative imputers actually condition on the visible data.

Critical strengths

The main strength is the controlled joint evaluation of the design choices behind Shapley explanations. All methods share the same coalition designs and budgets, ground truths are exact for small player sets, and the remaining grading error is measured for larger ones. Apart from the temporal SHAP baselines, which use their own estimators, differences between methods can therefore be attributed to the imputer and player choices rather than to the estimation. This makes the benchmark a careful and reproducible resource within the Shapley-value literature.

A second strength is game-matched grading. The distinction is known from prior work, but grading each method against the ground truth of its own game makes clear that marginal and conditional Shapley values answer different questions. Result 4 also shows that the common practice of grading everything against the conditional ground truth mixes these up.

Critical weaknesses

The first weakness is that the benchmark measures how accurately each method computes its chosen Shapley game, not whether the attributions are correct explanations. The authors never define what an explanation should recover, the features the model relies on or the mechanism connecting the data to the outcome. The mechanism is usually what clinicians want, and neither Shapley game provides it, since marginal Shapley can credit suppressor variables and conditional Shapley can credit correlated features the model ignores. This disconnect between the clinical motivation and the contribution is the main reason I find the significance limited. Since only the first joint determines the synthetic labels, the authors could address this directly by also reporting whether each method recovers that joint.

The second weakness is that several methodological choices undermine the headline results. The ground truth evaluates the model at the averaged imputation rather than taking the expected prediction, so the protocol rewards low-variance imputers. This is why KS-VAEAC outperforms the exact conditional oracle in Table 3. The authors note in Appendix B that the oracle is not a floor, but this means the ranking reflects how dispersed each imputer's draws are as much as how accurate its conditional model is. This undercuts Result 2. Under the same definition the marginal ground truth is simply the zero baseline, so KS-Marginal's error is mostly the noise of averaging five draws. The synthetic labels depend on the time-averaged mean of a single joint, so there is no temporally localized signal. The synthetic data also lack hard constraints such as bone lengths, so the anatomically impossible imputations that motivate generative imputers cannot occur. On the real data, faithfulness is computed with each method's own imputer on the same coalitions KernelSHAP was fitted on, so it largely measures how additive each method's game is and cannot be compared across imputers. Finally, the claim that grouping matters more than the imputer rests mainly on the periodic datasets, where the conditional ground truth is nearly uniform and EC3 is unstable, and which the authors themselves gray out in the tables.

Questions for authors
  1. Why is the game defined as the model evaluated at the averaged imputation rather than the expected prediction over imputations? How do the rankings in Tables 3-4 change under the expected-prediction game, and does KS-VAEAC still outperform the oracle?
  2. Since only the first joint determines the synthetic labels, does each method recover that joint, for example by ranking it first under the joint and joint x window player sets?
  3. Faithfulness and PlayerAOPC compute v(S) with each method's own imputer. How do the real-data rankings change if both metrics use one common imputer for all methods?
Reviewer_cGUSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

For spatiotemporal dataset, such as CARE-PD (3D skeletons of people walking) to detect Parkinson’s severity from human motion, explanation which part drives the decision is important in critical-safety application. Shapley values is a popular method to explain a model on task on a spatiotemporal dataset by treating input features as players in the cooperative game. The method needs to define what is players, how absent players are imputed and how the game is evaluated. However, the method lacks studies on how choices of players, imputers, and game can affect the explanation as well as the dataset normally does not contain ground truth attribution to check. Therefore, the paper introduces a benchmark called MotionBench-XAI to provide: five synthetic dataset exact ground-truth Shapley values, three real datasets, ten attribution methods across three player sets with three trained architectures. The experiments show that grouping features is more important than imputer, generative imputer only improve when conditional models are correct, and in real dataset, the metric with proxy evaluation protocol can disagree with each other. The paper also distill this to practical recommendation on the real application.

Critical strengths

The motivation of the work is clear: there is no systematic study of choices of players, imputers, game setting for Shapley values based explanation for spatiotemporal dataset.

Exact Shapley ground truth for players as subsets is meaningful to evaluate in practice, and align with how humans prefer explanation, on subset as feature instead of individual time steps or spatial time points.

Insight from experiments lead to key insight to guidance with practical recommendation is meaningful, showing the benchmark can bring new value to applications that need models with explanation.

Critical weaknesses

The synthetic dataset idea is overlapping with the paper [A], which provides synthetic dataset for LIME and SHAP.

The benchmark does not tell which metric the practitioner needs to use. In reality, the dataset does not have ground truth, but the paper does not check if faithfulness and PlayerAOPC is useful on synthetic dataset. There is no test to show the Practitioner decision guide is useful in real applications.

The answer for What is known about the data’s distribution? in Figure 3: Practitioner decision guide, is not clear and the evidence supporting that is not clear also, Two “If” conditions are not clear how to check if the data has that characteristic.

Why KS-VAEAC is better than KS-Oracle in Table 3? Does it mean the imputer in KS-Oracle is not good compare to the learned one in some cases.

Why do the paper only consider use Kernel-SHAP but not other SOTA of SHAP? I suggest can use [B] for the benchmark.

Some weak classifier is used, such as MLP for gait in Table 15, which can affect the explanation method since the model is not general.

The claim “grouping features into players, across time, space, or both, matters more than the choice of imputer. ” is not always supported, for example the same players but different imputer still can have large gap in metric EC3, Table 3, KS-VAEAC and KS-Flow is one example.

Claim “Generative imputers improve explanations only when their conditional models are accurate.” is not always correct. For example, KS-Gauss (closed form) is worse than the KS-VAEAC in the gauss dataset, where the gaussian model is correct.

[A] Liu, Yang, et al. "Synthetic benchmarks for scientific research in explainable machine learning." arXiv preprint arXiv:2106.12543 (2021).

[B] Musco, Christopher, and R. Teal Witter. "Provably accurate shapley value estimation via leverage score sampling." arXiv preprint arXiv:2410.01917 (2024).

Questions for authors

Can the authors compare the work with paper [A]?

Can the authors check if faithfulness and PlayerAOPC is useful on synthetic dataset?

Can the authors provide test to show the Practitioner decision guide is useful in real applications?

Can the authors do some analysis to explain Why KS-VAEAC is better than KS-Oracle in Table 3?

Can the authors explain why The claim “grouping features into players, across time, space, or both, matters more than the choice of imputer. ” is not always supported.

Can the authors explain why “Generative imputers improve explanations only when their conditional models are accurate.” is not always correct? For example, KS-Gauss (closed form) is worse than the KS-VAEAC in the gauss dataset, where the gaussian model is correct.

Reviewer_yboVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Summary: The paper studies how feature grouping, imputation, and evaluation choices affect Shapley explanations for spatiotemporal data, which is a worthwhile question. However, the contribution is mainly an evaluation pipeline, and the restricted ground truth, simple synthetic data, and imputer-dependent real-data evaluation limit its conclusions.

Critical strengths

The paper addresses a worthwhile research question: how feature grouping, imputation strategies, and evaluation choices affect Shapley explanations for spatiotemporal data. Understanding these effects is relevant to the reliable use of explanation methods in practice.

Critical weaknesses

Weaknesses

  1. The paper is difficult to follow because the task configuration and several key concepts are not explained clearly enough. The method categories mix feature grouping, missing-feature imputation, and attribution estimation, making it difficult to understand exactly what each comparison evaluates.
  2. The ground-truth Shapley values are defined for a specific mean-imputation game, which generally differs from averaging model predictions over possible completions. This restricts the scope of the benchmark, and the implications for its conclusions are insufficiently discussed.
  3. The five synthetic dataset families vary their statistical structure but share a simple labeling rule. Consequently, they provide limited evidence that the findings extend to more complex and realistic spatiotemporal prediction tasks.
  4. The real-data evaluation uses each method’s own imputer to measure prediction changes. This makes it difficult to separate attribution quality from the effects of the replacement procedure, weakening comparisons between methods.
  5. Although the research question is worthwhile, the contribution appears primarily to be an evaluation pipeline. The findings largely confirm expected trends, while providing limited insight into the mechanisms or conditions that would make them broadly useful.
Questions for authors

Questions:

  1. How do the conclusions depend on defining the ground truth with the mean-imputation game, rather than averaging predictions over possible completions?
  2. What evidence suggests the findings extend beyond the shared simple labeling rule to more complex, realistic tasks?
  3. How can attribution quality be separated from the effect of each method’s own imputer in the real-data evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.